1.0.8 • Published 2 years ago

@brainervirus.dev/web-scraper v1.0.8

Weekly downloads
-
License
ISC
Repository
github
Last release
2 years ago

GPU Scrapper

This is a simple GPU Scrapper that gets data from solotodo.com and submited to a mongodb instance.

Run Locally

Clone the project

  git clone https://github.com/BrainerVirus/node-espress-ts-gpu-scraper.git

Go to the project directory

  cd node-espress-ts-gpu-scraper

Install dependencies

  npm i

Start the server in dev mode

  npm run nodemon

Note : To run dev script you will need to make a nodemon.json file just like the example below

{
  "ignore": [".git", "node_modules", "dist", "build"],
  "watch": ["./src"],
  "exec": "npm run dev",
  "ext": "ts"
}

Start the server

  npm run start

Features

  • Data scrapping
  • Save data retrived
  • Show all the data retrived
  • Manual data insertion

API Reference

GPU model

title: {
    type: String,
    required: true,
  },
  price: {
    type: String,
    required: true,
  },
  img: {
    type: String,
    required: false,
  },
  name: {
    type: String,
    required: true,
  },
  memorySize: {
    type: String,
    required: true,
  },
  coreFrecuency: {
    type: String,
    required: true,
  },
  memoryFrecuency: {
    type: String,
    required: true,
  },
  bus: {
    type: String,
    required: true,
  },
  date: {
    type: String,
    required: true,
  },
  time: {
    type: String,
    required: true,
  },
});

Get all items

  GET /scrapper/

Response

{ message: `You have scraped ${totalDataScrapred} GPUs registries` }

Error Messages

codedescription
1db connection has failed
2db disconnection has failed
3Scrapper internal error
4GPU regestry insertion has failed

Get all items

  GET /gpu/list/

Response

{
    "GPUs": [
        {
            "_id": string,
            "title": string,
            "price": string,
            "img": string,
            "name": string,
            "memorySize": string,
            "coreFrecuency": string,
            "memoryFrecuency": string,
            "bus": string,
            "date": string",
            "time": string,
        },
    ]
}

Error Messages

codedescription
1db connection has failed
2db disconnection has failed
3Internal error
4Cannot find any gpus entries in the db

Create item

  POST /gpu/create/

Response

{ message: "GPU insertion succesfull" }

Error Messages

codedescription
1db connection has failed
2db disconnection has failed
3Internal error
4GPU regestry insertion has failed

Environment Variables

To run this project, you will need to add the following environment variables to your .env file

PORT

URL

MONGO_URI

License

MIT