web-scrapping-app v1.1.0

Web scraping APP

This project is a web scraping tool designed to extract data from websites for job seekers. It can be used to gather information from websites to make job application easy and faster.

Prerequisites

Before using this tool, make sure you have the following prerequisites installed:

• Nodejs
• Express
• ngrok
• puppeteer

Installation

1. Clone this repository to your local machine.
2. Install the required nodejs libraries using npm install.

Usage

1. Configure the scraping parameters in the scrape.js file.
2. sign up to ngrok to get authtoken.
3. Run the index.js script to start the scraping process.

URL configuration

This was configued for indeed website and location was configued Nigeria by default. Feel free to modify the baseUrl in scrape.js to the desired website and location.

Data Handling

The scraped data is stored in a jobs.json file by default. You can modify the code to store data in a different format or database.

Contact

For any questions or issues, please contact me at michaelsamuelpedro@gmail.com.