Subscribe-for-data NPM

subscribe-for-data

Fast implementation of multiple related models properties fetching & mixing to your model.

DB agnostic.
flexible query condition
one DB query per subscription.
all queries running parallel, starting at same moment, you choose then
No loops, stream based.
zero dependencies

Workflow description

You create subscription around your related model with makeSubscription(model, options)

You can create any count of subscriptions you need.

Then you can to subscription.add(target) target objects you want to mix in properties from related model data.

After you've added all needed targets to all subscriptions you can anytime run fillSubscriptions()

fillSubscriptions() assigns data as it goes via stream with auto parallelization if multiple subscriptions created. One query per subscription is executed.

It generates mongo condition. If you return from options.getCondition(target) scalar value then is generated $in query. I
to query your source,

Mongo query generation is just default behavior, you can alter it as you want.

Installation

npm i subscribe-for-data subscribe-for-data-from-mongoose

Import

const mongoosePlugin = require('subscribe-for-data-from-mongoose');
const { 
  makeSubscription, fillSubscriptions 
} = require('subscribe-for-data').use(mongoosePlugin);

Example

By default it works with mongoose. This behavior can be easily overriden by setting custom getStream option callback.

const mongoosePlugin = require('subscribe-for-data-from-mongoose');
const { makeSubscription, fillSubscriptions } = require('subscribe-for-data').use(mongoosePlugin);
const RootModel = require('./MainModel');
const RelatedModel = require('./RelatedModel');
const AnotherRelatedModel = require('./AnotherRelatedModel');

(async () => {
  const relatedSubscription = makeSubscription(RelatedModel, { // single field attach
    targetField: 'position', // key of property to be created on roots
    foreignField: 'root_id', // field to build condition against
    sourceField: 'position'
  });
  const anotherRelatedSubscription = makeSubscription(AnotherRelatedModel, { // something completely different
    getCondition({ mysteriousTimestamp, type }) {
      return { type, updatedAt: { $gt: mysteriousTimestamp} };
    }, 
    assignData(root, { someField, otherType }) {
      (root.someFields = root.someField || []).push(someField);
      (root.otherTypes = root.otherTypes || new Set()).add(otherType);
    },
  });
  const roots = [];
  await RootModel.find({}).cursor().eachAsync((root) => {
    [relatedSubscription, anotherRelatedSubscription]
      .forEach(subscription => subscription.add(root)); // subscribed
    roots.push(root);
  });
  await fillSubscriptions(); // 2 DB queries executed in parallel, no loops then
  console.log(roots[0]);
})();

Expected output:

{
  "_id": "000000",
  "position": 42,
  "someFields": [2, 5, 8, 5],
  "otherTypes": [23, 42, 78]
}

API Reference

SubscribeForData : object
- .makeSubscription(source, options) ⇒ Object
- .fillSubscriptions() ⇒ Promise
- .assignDefaultOptions(mixin)

assignData : function
getKey ⇒ *
extractKey ⇒ *
getCondition ⇒ *
getDataHandler ⇒ function
getAddingMethod ⇒ function
getStream : function

SubscribeForData : object

subscribe-for-data

SubscribeForData : object
- .makeSubscription(source, options) ⇒ Object
- .fillSubscriptions() ⇒ Promise
- .assignDefaultOptions(mixin)

SubscribeForData.makeSubscription(source, options) ⇒ Object

Creates subscription for related model data

Param	Type	Description
source	Object	Source model
options	Object	Options
options.targetField	String	field data to be saved into (optional)
options.baseCondition	Object	Base condition
options.defaultValue	*	Default value for field
options.getKey	getKey	Callback which returns unique key from target model (model.id by default)
options.getCondition	getCondition	returns condition, using target model (model.id by default)
options.extractKey	extractKey	returns unique key of target model from foreign model
options.isMultiple	Boolean	if one to many relation
options.useEachAsync	Boolean	only for mongoose cursor
options.parallel	Number	parallel parameter for `eachAsync` if `useEachAsync` is `true`
options.foreignField	String	If `getCondition` returns scalar values this field will be used for $in
options.sourceField	String	field to use of foreign model
options.assignData	assignData	Do model filling by itself, otherwise use `targetField`
options.getStream	getStream	returns stream from source and condition (using mongoose model by default)
options.getDataHandler	getDataHandler	Get data handler for processing related models
options.getAddingMethod	getAddingMethod	Get `add()` method of future `subscription`

SubscribeForData.fillSubscriptions() ⇒ Promise

Fill subscribed targets

SubscribeForData.assignDefaultOptions(mixin)

change default options

Param
mixin

assignData : function

Assigns data from foreign model to target

Param	Type	Description
target	Object	your target model
foreign	Object	foreign model

getKey ⇒ *

get unique identifier of target for internal indexing

Returns: * - target identifier

Param	Type	Description
target	Object	your target model

extractKey ⇒ *

get unique identifier of target from foreign model

Returns: * - target identifier

Param	Type	Description
foreign	Object	Foreign model data

getCondition ⇒ *

get condition

Returns: * - condition, can be scalar or object

Param	Type	Description
target	Object	your target model

getDataHandler ⇒ function

get foreign data handler

Returns: function - Callback handling data assignment

Param	Type	Description
options	Object	Options
options.targets	Object	targets index
options.targetField	String	field data to be saved into
options.extractKey	extractKey	returns unique key of target model from foreign model
options.isMultiple	Boolean	if one to many relation
options.sourceField	String	field to use of foreign model
options.assignData	assignData	Do model filling by itself, otherwise use `targetField`

getAddingMethod ⇒ function

get future subscription.add() method

Returns: function - Callback handling data assignment

Param	Type	Description
options	Object	Options
options.targets	Object	targets index
options.getKey	getKey	Callback which returns unique key from target model (model.id by default)
options.getCondition	getCondition	returns condition, using target model (model.id by default)
options.defaultValue	*	Default value for field
options.targetField	String	field data to be saved into
options.condition	object	DB Query condition, being prepared
options.extractKey	extractKey	returns unique key of target model from foreign model
options.foreignField	String	If `getCondition` returns scalar values this field will be used for $in
options.inner	Array	Internal array for condition storing