screx v1.0.0

• general screx is a thin wrapper around knexjs / requestjs / cheeriojs / rate-limit

• downloads the html body and optionally wraps it in cheerio
• saves body and url to a local table to cache repeated requests to the same url
• employs a queue to limit the maximum number of requests per second (interval between requests in ms)

• api #+BEGIN_SRC js /* You should only use one instance of screx in your app - otherwise there will be multiple queues and rate limiting won't work. Create a screxfile somewhere and require that:

  ./screxfile.js

  var screx = require('screx'); //var knexConfig = require('./knexfile'); var yourKnexConfig = {}; module.exports = screx(yourKnexConfig, 'request_body', 300);

,*/

// if no table name is provided the table 'response_body' is used var screx = require('screx')(knexConfig, table, interval); // setup the table on the first run screx.createTable().then(function() { console.log('done'); });

// on subsequennt runs: screx.getBody('url').then(function(body) { /* the first time an url is requested it's retrieved from the server after that the body is retrieved from the table

,*/ }); // or use loadBody to get the body prewrapped by cheerio screx.loadBody('url').then(function($) { // ... });

// to reset the table screx.resetTable().then(function() { console.log('done'); }); #+END_SRC

• running the test suite #+BEGIN_SRC sh export KNEX_TEST='/path/to/your/knex_config.js'

relative to the spec folder, so a file in the spec folder would have the path './foo'

depending on your config you also need to install the relevant database module (e.g. pg/mysql) (there'll be an error telling you what to do)

npm test #+END_SRC