@finscrapers/finscrapers NPM

finscrapers

Finscrapers is a Node-based CLI that helps investors to scrape data from websites with Puppeteer. That is how private investors can take sophisticated decisions based on facts, instead of solely gut instincts. Without paying thousands of dollars for a Bloomberg Terminal. Instead Finscrapers will collect data from a financial website and save them to Elasticsearch to display them in any report, or dashboard. The best is, it's free and open source!

Currently supported financial sources:

Yahoo Finance

Installation

Add Finscrapers as global package.

npm install -g @finscrapers/finscrapers

Usage

1. See `--help`

Run the script with Node and you will see flags listed below.

finscrapers --help`

Flag	Required	Description	Example
--elasticsearch, -e	yes	Path to Elasticsearch host, default is http://localhost:9200	https://myelasticsearchserver.com
--symbols, -s	no (defaults to ./symbols.json)	Relative path of the files with the symbols to scrape	`~/symbols.json`
--headless, -h	no	Runs Puppeteer in headless mode (you will not see the browser)
--slowMo, -m	no (defaults to 300)	Slows down Puppeteer operations by the specified amount of milliseconds	1000

2. Start Elasticsearch server

Take the manual installation route with Elasticsearch Reference, easier, but more costly, subscribe Elasticsearch Service or AWS Elasticsearch Service.

3. Configure `symbols.json`

A file containing the symbols needs to be created before executing the script. Let's create a file called symbols.json:

{
  "symbols": ["IBM", "MSFT", "FB", "AMZN"]
}

4. Run Finscrapers:

finscrapers -e https://myelasticsearchserver.com -s symbols.json

5. Run Kibana:

You need to install Kibana, as mentioned in the section Start Elasticsearch server, investigate the data, and finally invest like Warren Buffet. If you get rich, don't forget to buy us a coffee.