Rncl NPM | npm.io

retail-notifier

Overview

This project provides a set of command-line tools to scrape retail websites and notify users about new products.

Installation

Clone this repository and run npm install -g rncl to install the necessary dependencies.

Usage

To use the retail-notifier, run the following command in the terminal:

rncl [options] [command]

The following commands are available:

Scrape Commands

To scrape a specific retail website, use the following command:

rncl scrape:<retailId> [options] <config-file>

Supported retailId values are:

abcdin
falabella
hites
lapolar
lider
paris
ripley
entel

Options for the scrape command are:

--dry-run: do not store or notify anything (default: false)
-m, --migrate: notify only products with > 90% discount, prevent spamming on new (default: false)

Run Commands

The following commands are used to run the retail-notifier:

run:proxy: start web proxy for products
run:admin-notifier: notify to admins products checked and ready to be notified to end users that doesn't have the shouldNotify flag set
run:twitter-notifier [options]: notify to twitter products with shouldNotify flag set
run:telegram-notifier: notify to telegram main channel products with shouldNotify flag set
run:remove-product-stored: remove products older than PRODUCT_RETENTION_IN_DAYS env variable
run:check-product-stored: check products that require stock validation before publishing
run:store-product-found: store products found by scrapers
run:handle-admin-events: handle admin commands sent from telegram

Options for the run:twitter-notifier command are:

--use-trends: generate tweet based on current trends on twitter (default: false)

Test Commands

The following commands are used for testing purposes:

test:ai-tweet [options]: create a tweet using AI based on a generic product
test:get-trends: get current trends from twitter
test:fetch-product <url>: fetch product from url

Options for the test:ai-tweet command are:

--use-trends: generate tweet based on current trends on twitter (default: false)

Help Command

To display the help for any command, use the following command:

rncl help [command]

Configuration

Sesitive data should be set via env variables (.env.dist). Here is a brief explanation of some of the variables:

PORT: This variable defines the port on which the proxy will run. By default is 7174.
PRODUCT_RETENTION_IN_DAYS: This variable is used to set the number of days after which a product is considered to be old and can be removed from the system. By default is 60.
ELASTICSEARCH_USERNAME and ELASTICSEARCH_PASSWORD: These variables define the Elasticsearch username and password that the application uses to connect to the Elasticsearch server.
ELASTICSEARCH_INDEX: This variable defines the name of the Elasticsearch index that the application uses to store product data. By default is products.
ELASTICSEARCH_HOST: This variable defines the host address of the Elasticsearch server.
RABBIT_FQDN: This variable defines the fully qualified domain name of the RabbitMQ server.
HASHTAG: This variable defines the default hashtags that will be used when creating tweets.
IGNORE_BRAND: This variable defines the brand name of products that should be ignored by the application.
TWITTER_API_KEY, TWITTER_API_KEY_SECRET, TWITTER_ACCESS_TOKEN, and TWITTER_ACCESS_TOKEN_SECRET: These variables are used to authenticate the application with the Twitter API.
TELEGRAM_BOT_TOKEN, TELEGRAM_CHANNEL_ID, and TELEGRAM_ADMIN_CHANNEL_ID: These variables are used to authenticate the application with the Telegram API and define the chat IDs of the channels that the application uses to send notifications.
OPENAI_API_KEY: This variable is used to authenticate the application with the OpenAI API.