Custom-rss NPM

Custom RSS

:satellite: Filtering RSS because Zapier is too expensive (and Pipes is gone).

Features

Blacklist filtering
Repost guarding
Skipped entry logging
No database required
Uses JSON for configuration
Feeds as middleware functions
As few dependencies as possible
ES5, so compatible with older Node versions

Usage

The sample app is what's in the root directory. It's a barebones Connect app, with personal configuration in config.json. This is foremost for personal use, but what's in /src should be reusable with any Connect-like web framework. Or to use as-is:

$ npm run develop
$ subl config.json # continued below
$ subl src/feeds/feedd.js # continued below
$ npm run deploy

$ ssh <production>
$ cd <site> # or `mkdir <site>; cd <site>`
$ git pull origin master # or `git clone <repo>`
$ npm start # or touch tmp/restart.txt
$ exit

$ curl <site-url>/feedd

// config.json
{
  "feeds": [
    // ...
    {
      "name": "feedd",
      "filters": [
        { "name": "tired-topics", "type": "blacklist", "tokens": [ "Foo", "Bar", "Baz" ] }
      ]
    }
  ]
}

// src/feeds/feedd.js
var fetchFeed = require('../fetch-feed');
var filterFeed = require('../filter-feed');
var url = require('url');

module.exports = function(config, request, response) {
  config.originalURL = 'http://feedd.com/rss.xml';
  config.url = url.format({
    protocol: 'http', host: request.headers.host, pathname: config.name
  });
  fetchFeed({
    url: config.originalURL,
    onResponse: function(resFetch, data) {
      response.setHeader('Content-Type', resFetch.headers['content-type']);
      filterFeed({
        config: config,
        data: data,
        onDone: function(data) { response.end(data); }
      });
    },
    onError: function(e) { response.end(e.message); }
  });
};

Feeds

Hacker News (via hnapp)
Quartz
NYTimes Business
Ray Wenderlich
Gama Sutra

Implementation

Some shared hosting providers, including mine, refuse to have NPM installed on their system. So dependencies need to be few to none, unless they're small enough to version. No XML parser or writer is used; a much lighter hand-rolled transformer does basic regex parsing. No MySQL client is used; data is stored with limits in plain files and manipulated in buffers (memory). No logger or mailer is used for feedback; custom loggers are handrolled as needed, with utilities on top of fs. The test-runner is handrolled (only because not a lot is required). Connect is the only dependency (but not really, see usage). This core constraint also yields the opportunity to learn Node fundamentals.

custom-rss