bgc-crawler-sidekick v1.0.48

BGC Crawler Sidekick

This repository contains some utilities to be used with captcha and crawlers components.

Install

npm install bgc-crawler-sidekick

Captcha Components

• Function to get the base64 from image captchas. import { getImageBase64 } from "bgc-crawler-sidekick"

• Function to get image captcha response. import { getImageCaptchaResponse } from "bgc-crawler-sidekick"

• Recaptcha helper import { recaptcha } from "bgc-crawler-sidekick"

• Asking for refund import { askRefund } from "bgc-crawler-sidekick"

Crawler Components

Crawler helper

• Inside this class you can find many function to help you with your crawler. import { CrawlerHelper } from "bgc-crawler-sidekick"

Functions:

Example of usage:

  const helper = new CrawlerHelper({ state, institution });
  await helper.launchBrowser(proxy);

• _setProxyIfExists
• _getChromeBinary
• _saveFileOnBucket
• _setPDFDownloadBehavior
• _waitForPDFToDowload
• debugBehaviour
• launchBrowser
• generateFolderName
• getEvidence
• getPDFEvidenceLink
• interceptUndesiredRequests
• removeInterceptUndesiredRequests
• clickWithMouse
• removeAccents
• retrieveUsedIP
• fillTextInputs
• downloadPDF
• readPDF
• savePDFEvidence

Errors

• Import: import { errors } from "bgc-crawler-sidekick"

• Example of usage:

    throw new errors.BrowserNotLaunchedError(error);