Turkish-columnist-crawler NPM

Crawl the columns on newspaper websites.

🏠 Homepage

Install

npm i turkish-columnist-crawler

Usage

This is a simple use case. See the Documentation section for details.

const crawler = require('turkish-columnist-crawler')

let urls = SAMPLE_URL

crawler(url, {
    date: '23 Nisan 1920', // Activate if you want to scan articles written after this date.
    last: false, // Activate only if you want the last article.
    externalParams: null, //Allows you to add things that need to be added to the return data.
    saveDisk: true, //Saves outputs to disk
    limit: -1, //Determines the maximum count of articles to be scanned. If you give -1, it scans a maximum of 100 articles.
    filePath: __dirname + '/texts', // If you set the saveDisk parameter to true, you have to give the entire Path of the folder where the files will be saved.
    strOp: false // If you use this parameter, you will see that there are spaces between all punctuation and words.
})