Feathers-hooks-rediscache NPM

Redis Cache

Cache any route with redis

Releases

Versions 1.x.x are compatible with Feathersjs 3.x.x
Versions 0.x.x are compatible with Feathersjs 2.x.x -> this branch will not be updated anymore

Installation

Feathers 3.x.x

npm install feathers-hooks-rediscache --save

Feathers 2.x.x

If you do not use nested routes you can install version 1.x.x if not:

  npm install feathers-hooks-rediscache@0.3.6 --save-exact

Purpose

The purpose of these hooks is to provide redis caching for APIs endpoints. Using redis is a very good option for clusturing your API. As soon as a request is cached it is available to all the other nodes in the cluster, which is not true for usual in memory cache as each node has its own memory allocated. This means that each node has to cache all requests individually.

Each request to an endpoint can be cached. Route variables and params are cached on a per request base. If a param to call is set to true and then to false two responses will be cached.

The cache can be purged for an individual route, but also for a group of routes. This is very useful if you have an API endpoint that creates a list of articles, and an endpoint that returns an individual article. If the article is modified, the list of articles should, most likely, be purged as well. This can be done by calling one endpoint.

Routes exemples

In the same fashion if you have many variants of the same endpoint that return similar content based on parameters you can bust the whole group as well:

'/articles' // list
'/articles/article' //individual item
'/articles/article?markdown=true' // variant

Clearing cache

These are all listed in a redis list under group-articles and can be busted by calling /cache/clear/group/article or /cache/clear/group/articles it does not matter. All urls keys will be purged.

You can also purge single cached paths as by doing GET requests on

'/cache/clear/single/articles'
'/cache/clear/single/articles/article'
'/cache/clear/single/articles/article?markdown=true' // works with query strings too

It was meant to be used over HTTP, not yet tested with sockets.

Available hooks

More details and example use bellow

Before

redisBeforeHook - retrives the data from redis

After

hookCache - set defaults caching duration, an object can be passed with the duration in seconds
redisAfterHook - saves to redis
hookRemoveCacheInformation - removes the cache object from responses (does not clear from Redis)

Documentation

Add the different hooks. The order matters (see below). A cache object will be added to your response. This is useful as other systems can use this object to purge the cache if needed.

If the cache object is not needed/wanted it can be removed with the after hook hookRemoveCacheInformation()

Configuration

Redis

To configure the redis connection the feathers configuration system can be used.

//config/default.json
{
  "host": "localhost",
  "port": 3030,
  "redis": {
    "host": "my-redis-service.example.com",
    "port": 1234
  }
}

if no config is provided, default config from the redis module is used

Hooks Configuration

A redisCache object can be added to the default feathers configuration

//config/default.json

  "redisCache" : {
    "defaultDuration": 3600,
    "parseNestedRoutes": true,
    "removePathFromCacheKey": true,
    "env": "NODE_ENV"
  };

defaultDuration

The default duration can be configured by passing the duration in seconds to the property defaultDuration. This can be overridden at the hook level (see the full example bellow)

parseNestedRoutes

If your API uses nested routes like /author/:authorId/book you should turn on the option parseNestedRoutes. Otherwise you could have conflicting cache keys.

removePathFromCacheKey

removePathFromCacheKey is an option that is useful when working with content and slugs. If when this option is turned on you can have the following issue. If your routes use IDs then you could have a conflict and the cache might return the wrong value:

  'user/123'
  'article/123'

both items with id 123 would be saved under the same cache key... thus replacing each other and returning one for the other, thus by default the key includes the path to diferenciate them. when working with content you could have an external system busting the cache that is not aware of your API routes. That system would know the slug, but cannot bust the cache as it would have to call /cache/clear/single/:path/target, with this option that system can simply call :target which would be the slug/alias of the article.

env

The default environement is production, but it is anoying when running test as the hooks output information to the console. Therefore if you youse this option, you can set test as an environement and the hooks will not output anything to the console. if you use NODE_ENV it will pick up the process.env.NODE_ENV variable. This is useful for CI or CLI.

immediateCacheKey

By default the redis cache key gets determined in redisAfterHook based on the path. However if you're doing a lot of query manipulation you might want to set the cache key before anything else to keep its size as small as possible. You can achieve this by setting immediateCacheKey: true what will set the cache key in the redisBeforeHook. Then your hooks might look similar to:

{
  before: {
    find: [redisBefore({ immediateCacheKey: true }), someQueryManipulation()]
  },
  after: {
    find: [cache(), redisAfter()]
  }
}

Available routes:

// this route is disable as I noticed issues when redis has many keys, 
// I will put it back when I have a more robust solution
// '/cache/index' // returns an array with all the keys
'/cache/clear' // clears the whole cache
'/cache/clear/single/:target' // clears a single route if you want to purge a route with params just adds them target?param=1
'/cache/clear/group/:target' // clears a group

Complete Example

Here's an example of a Feathers server that uses feathers-hooks-rediscache.

const feathers = require('feathers');
const rest = require('feathers-rest');
const hooks = require('feathers-hooks');
const bodyParser = require('body-parser');
const errorHandler = require('feathers-errors/handler');
const routes = require('feathers-hooks-rediscache').cacheRoutes;
const redisClient = require('feathers-hooks-rediscache').redisClient;

// Initialize the application
const app = feathers()
  .configure(rest())
  .configure(hooks())
  // configure the redis client
  .configure(redisClient)

  // Needed for parsing bodies (login)
  .use(bodyParser.json())
  .use(bodyParser.urlencoded({ extended: true }))
  // add the cache routes (endpoints) to the app
  .use('/cache', routes(app))
  .use(errorHandler());

app.listen(3030);

console.log('Feathers app started on 127.0.0.1:3030');

Add hooks on the routes that need caching

//services/<service>.hooks.js

const redisBefore = require('feathers-hooks-rediscache').redisBeforeHook;
const redisAfter = require('feathers-hooks-rediscache').redisAfterHook;
const cache = require('feathers-hooks-rediscache').hookCache;


module.exports = {
  before: {
    all: [],
    find: [redisBefore()],
    get: [redisBefore()],
    create: [],
    update: [],
    patch: [],
    remove: []
  },

  after: {
    all: [],
    find: [cache({duration: 3600 * 24 * 7}), redisAfter()],
    get: [cache({duration: 3600 * 24 * 7}), redisAfter()],
    create: [],
    update: [],
    patch: [],
    remove: []
  },

  error: {
    all: [],
    find: [],
    get: [],
    create: [],
    update: [],
    patch: [],
    remove: []
  }
};

the duration is in seconds and will automatically expire
you may just use cache() without specifying a duration, any request will be cached for a day or with the global configured value (see configuration above).

License

Licensed under the MIT license.

Change log

v.1.1

Fixes #23. Feathers works slightly differently, now the change is reflected in the hooks. Tests were adapted.
Fixes #24. When it can Webpack 4 tries to evaluate code and remove "dead" code, therefore a condition testing for the environement was being evaluated, the value was set to the build environement... To bypass that, the hooks consider that they run in production mode. If you want to set a different one add a property env: "NODE_ENV" to the rediscache object, it will pick up your node environement and pass it to the hooks.

v1.1.0

The /index path as well as the scan methods have been removed for now. In fact, testing on a Redis instance with more than 30k keys, it might bring down your server. I need to find a better way to return keys, or to search for them. So to prevent any problem I have removed it. (the code is commented out).