codepen-puppeteer v1.1.0
╭─╮ ╭─╮ ╭┬╮ ╭─╮ ╭─╮ ╭─╮ ╭╮╭ ┬ ╭─╮
│ │ │ ││ ├┤ ├─╯ ├┤ │││ │ │ │
╰─╯ ╰─╯ ─┴╯ ╰─╯ ┴ ╰─╯ ╯╰╯ o ┴ ╰─╯
╭────╮ ╭──╮╭╮ ╭────╮ ╭────╮ ╭──▞─╮ ╭──╮ ╭────╮ ╭────╮ ╭─┬─╮
│ ╭╮│ │ │││ │ ╭╮│ │ ╭╮│ │ `◯ │ ╭╯ ╰╮ │ ─ │ │ ─ │ │ │
│ ╰╯│ │ ╰╯│ │ ╰╯│ │ ╰╯│ │ │ ╰╮ ╭╯ │ │ │ │ │ ╭╯
│ ╭╯ │ │ │ ╭╯ │ ╭╯ │ ───┤ │ ─┤ │ ───┤ │ ───┤ │ │
╰───╯ ╰───┴╯ ╰───╯ ╰───╯ ╰────╯ ╰──╯ ╰────╯ ╰────╯ ╰──╯ Use Puppeteer to download pens from Codepen.io as single html pages.
Features
- Download example pens as single html pages
- Easy preview with an index page
- Built-in error recovery to resume download
- Skip already downloaded pens
- Easy to debug using screenshots
- Custom template pages
- Easy to follow source code with comments
- Support for loading external resources (i.e.
jquery,google fonts)
Usage
- Clone this project locally,
git clone https://github.com/websemantics/codepen-puppeteer
cd codepen-puppeteer- Install dependencies (
puppeteer),
npm iThis package provide two commands to interact with,
searchcommand to download pens matching search query
penpet search flexboxYou can specify start and end page with -s and -e options
- Browse to
./pens/index.htmlto preview full list of downloads
filecommand to download provided list of pens
penpet file pens.jsonFile pens.json is provided as an example
- For examples and more help, use option
-hwith both commands
Debug
This project is a proof of concept so you might find problematic pens that wouldn't download fully. Turn the debug flag -d with the file command to enable screenshots which might help you debug the issue,
penpet file pens.json -dHint
I find the following command useful to force quit running chromium processes on OSX
pkill -f -- "chromium"Preview Downloads
Resources
- Puppeteer - Headless Chrome Node API
- Getting started with Puppeteer and Chrome Headless for Web Scraping
Support
Need help or have a question? post at StackOverflow.
Please don't use the issue trackers for support/questions.
Star if you find this project useful, to show support or simply for being awesome :)
Contribution
Contributions to this project are accepted in the form of feedback, bugs reports and even better - pull requests.
License
MIT license Copyright (c) Web Semantics, Inc.