0.1.1 • Published 7 years ago

pika-web-crawler-test v0.1.1

Weekly downloads
1
License
MIT
Repository
github
Last release
7 years ago

Pika web crawler test

Introduction

Web crawler test script for web testing with snapshot comparison.

Installing / Getting started

To install the package execute:

npm install -g pika-web-crawler-test

After installation, tou will have access to the 'pika-web-crawler-test' binary in your command line. You can check help with this command:

pika-web-crawler-test --help

Developing

Built With

Prerequisites

The following software must be installed

Folder structure

  • root: Contains the README.md, the main configuration to execute the project such as package.json or any other configuration file.
  • bin: Contains the oclif script run.
  • scripts: Contains the scripts to start/stop Chrome in headless mode or any other system script required to run the tests.
  • src: Contains the source code for crawler test script.
  • node_modules: Contains third party JS libraries used in this project

Setting up Dev

Download the code

git clone https://github.com/pikamachu/pika-web-crawler-test.git
cd pika-web-crawler-test

Install dependencies

npm install

Run web crawler test. Parameters:

  • initial-url: Initial Url to open and parse links.
  • chunks: Number of concurrent url calls.
bin/run [initial-url] [chunks]

Pika commands

All previous command can be executed using pika script

Usage: pika [command]

where [command] is one of:
   run [initial-url] [chunks] - run the crawler test - also starts and stops chrome
     - initial-url: Initial Url to open and parse links.
     - chunks: Number of concurrent url calls. Default 2
   format - auto format project code using prettier

Windows Environment

  • If the project is ran in a Windows environment use the script files to start and stop chrome.
  • Chrome should be installed in c:\Program Files (x86)\Google\Chrome\Application, if it is not the case update the path in processes/start-chrome-headless.bat.