Sl-triggers NPM

Datasource
Create and subscribe for trigger
Event processing
Triggers API

Datasource

sequenceDiagram   
    participant triggers
    participant adapter
    participant datasource               
    datasource->>adapter: sends raw data
    adapter->>adapter: converts raw data into standard events
    adapter->>triggers: pushes standard events to  http interface

datasource streams events to adapter
adapter being aware of the logic of the datasource converts raw data into internal standard events.
adapter pushes standard events into http interface of triggers

Each standard event represents an entity from this table: https://docs.google.com/spreadsheets/d/1YBeSAcBpC596bcqI0M_mkXIqGHSMgbGh4myCF4Er89k/edit#gid=452767957

All events extend the base standard event

interface Event {
    // unique raw event identifier from datasource
    id: string 
    datasource: "sportradar"
    // scope of event
    scope: "game"
    // unique scope identifier, e.g. d8539eb6-3e27-40c8-906f-9cd1736321d8, adapter takes it from datasource raw data
    scopeId: string
    // event name
    name: string    
    // event value
    value: string | number
    // timestamp, unixtime millis
    timestamp: number
}

Example of events derived from Event

interface GameLevelEvent extends Event {
    name: "game.level" 
    value: "game.start" | "game.quarter.start" | "game.half.start" | "game.quarter.end" | "game.half.end" | "game.end" | "game.under_review"
}

interface GameHomePointsEvent extends Event {
    name: "game.home_points"    
    value: number
}

interface GameAwayPointsEvent extends Event {
    name: "game.away_points"    
    value: number
}

the event objects will be like

{   
    datasource: "sportradar",
    scope: "game",
    scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",    
    id: "ab362b76-b89d-4cd8-80e3-340b048f98c8",
    name: "game.level",
    value: "game.start",
    timestamp: 1686054201993
} as GameLevelEvent

{
   datasource: "sportradar",
   scope: "game",
   scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",
   id: "ab362b76-b89d-4cd8-80e3-340b048f98c8",
   name: "game.home_points",         
   value: 100,
   timestamp: 1686054201993
} as GameHomePointsEvent

Create and subscribe for trigger

To use service one should create trigger and subscribe for trigger to be notified of its activation. It is assumed that Studio should be used for this purpose.

sequenceDiagram       
    participant glossary
    participant studio            
    participant triggers
    studio->>glossary: get list of NBA games, players etc
    glossary->>studio: list of reference data
    studio->>triggers: get list of triggers for game_id
    triggers->>studio: list of triggers <br/>[ { trigger_id, conditions, description } ]
    studio->>triggers: create new trigger if none suits
    triggers->>studio: trigger object { trigger_id }
    studio->>triggers: subscribe service <br/>{ trigger_id, route, <br/>payload, service: "polls", <br/>entity: "question", entity_id }
    triggers->>studio: ok, subscribed

studio fetches datasource nomenclature from glossary to show/create triggers; e.g. "sl-sports" can be used as glossary, since it has sync with sportradar;
studio matches "game.id" of interest from glossary and stores it as scope and scopeId somewhere inside the moderation object;
studio fetches a list of existing triggers by using { datasource, scope, scopeId};
if none of the existing triggers suits, studio creates new trigger with certain conditions;
studio subscribes the destination service (e.g. "polls") for a trigger; to do so studio presents following arguments:
- callback route, e.g. "studio.questions.activate"
- callback payload to be passed into the route, e.g. { id } with question id

Trigger is the main entity inside service triggers

interface Trigger {
   // uuid generated for each trigger
   id: string
   // human-readable name just for description purposes
   name: string
   // human-readable name just for description purposes
   description: string
   datasource: "sportradar"
   scope: "game"
   scopeId: string   
   conditions: TriggerCondition[]
}

interface TriggerCondition {
   // uuid generated for each condition
   id: string
   event: string   
   // comparison operation, should be used to compare "current" and "target" and return a boolean
   compare: "eq","lt","gt","le","ge"   
   // type of condition
   type: "set-and-compare"
   // target value of the event, threshold value to compare with
   target: string | number
   // current value read from event
   current: string | number
   // true when compare(target, current) == true
   activated: boolean
   // log of events consumed by condition
   log: Map<string, Event>
}

example of trigger object

{
   id: "{uuid}",
   name: "home points 30+",
   description: "should trigger when home points reach 30 or more",
   datasource: "sportradar",
   scope: "game",
   scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",
   conditions: [
      {
         id: "{uuid}",
         event: "game.home_points",
         type: "set-and-compare",
         compare: "ge",
         target: 30,         
         current: 10,
         log: {
            "event_1-{uuid}": {
               datasource: "sportradar",
               scope: "game",
               scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",
               id: "ab362b76-b89d-4cd8-80e3-340b048f98c8",
               name: "game.home_points",         
               value: 0,
               timestamp: 1686054201993
            },
            "event_2-{uuid}": {
               datasource: "sportradar",
               scope: "game",
               scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",
               id: "ab362b76-b89d-4cd8-80e3-340b048f98c8",
               name: "game.home_points",         
               value: 10,
               timestamp: 1686054201994
            },
         }
      }
   ]
}

When trigger is created, it is stored in collection "triggers" as redis hash.

triggers/{trigger-id as uuid}:   
   id: "{uuid}",
   name: "home points 30+"
   description: "should trigger when home points reach 30 or more"
   datasource: "sportradar"
   scope: "game"
   scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8"

Trigger conditions are stored into separate collection "triggers/conditions" as redis hash.

triggers/{trigger-id as uuid}/conditions/{condition-id as uuid}:
   id: "{uuid}"
   event: "game.home_points"
   compare: "ge"
   target: 30
   type: "set-and-compare"
   current: 0

Every event consumed by trigger condition is stored in the log collection as redis hash

triggers/{trigger-id as uuid}/conditions/{condition-id as uuid}/logs:
   "event_id {uuid}": "{ JSON document of the event }"
   ... as many rows as events received while condition is listening...

The list of conditions are linked to a trigger as redis set.

triggers/{trigger-id}/conditions:      
   - "condition-id-1 as {uuid}"
   - "condition-id-2 as {uuid}"
   ...

Trigger is added into event subscribers as redis set.

events/{event-name}/subscribers:      
   - "trigger-id-1 as {uuid}"
   - "trigger-id-2 as {uuid}"
   ...

When trigger is created it can be subscribed. Trigger subscribers are notified upon trigger activation. To create a subscription studio passes subscription object in the request "triggers.subscribe"

interface TriggerSubsription {
   // uuid generated on create
   id: string
   // linked trigger
   triggerId: string  
   // name of destination service, e.g. "polls"
   service: string
   // entity, e.g. "question"
   entity: string
   // entity id
   entityId: string
   route: string
   payload: object
}

Trigger subscription is stored as redis hash.

triggers/{trigger-id as uuid}/subscriptions/{subscription-id as uuid}:
   id: "subscription {uuid}"
   triggerId: "trigger {uuid}"
   service: "polls"
   entity: "question"
   entityId: "123"
   route: "studio.questions.activate"
   payload: "{ JSON of payload object }"

To match subscription to trigger its ID is stored in redis set collection under the trigger-id key.

triggers/{trigger-id as uuid}/subscriptions:
   - "subscription-id-1 as {uuid}"
   - "subscription-id-2 as {uuid}"
   ...

Also, to search for subscriptions by entity, entity_id or service we should use RedisSearch extension.

Event processing

sequenceDiagram
    participant service    
    participant triggers
    participant adapter    
    adapter->>triggers: pushes standard events to  http interface    
    triggers->>triggers: get list of triggers subscribed on standard event
    loop For every trigger
        triggers->>triggers: apply event on the trigger
    end    
    triggers->>triggers: trigger is activated if all conditions activated
    loop For every subscriber
        triggers->>service: send message to "route" { payload }
    end       
    triggers->>triggers: remove activated triggers

diagram description:

adapter pushes standard events into http interface of triggers

When event "game.home_points" is received from adapter,

{
   datasource: "sportradar",
   scope: "game",
   scopeId: "d8539eb6-3e27-40c8-906f-9cd1736321d8",
   id: "ab362b76-b89d-4cd8-80e3-340b048f98c8",
   name: "game.home_points",         
   value: 10,
   timestamp: 1686054201993
}

it has a list of ids of subscribed triggers

SMEMBERS events/game.home_points/subscribers

We fetch a trigger by ID and apply event on the trigger. To do so, we proceed with the flow:

get a list of conditions for the trigger

SMEMBERS triggers/{trigger-id}/conditions

for each condition one should run lua script according to condition's "type" field, e.g. for { type: "set-and-compare" } run lua script which should:
- set current value to 10 replacing the old one
- compare current and target values and store bool result into activated , e.g condition.activated = condition.target >= condition.current
- put event JSON into log collection
- return condition state activated in the result

There are as many Lua scripts as conditions types, e.g:

set-and-compare: to store current value passed by the event and compare with target value
incr-and-compare: to increment some accumulator and compare with target value

Collect results of all conditions inside the trigger.

If all conditions are activated then the trigger becomes activated. And here we should notify all trigger subsribers about it.

we take a list of subsription IDs from

SMEMBERS triggers/{trigger-id as uuid}/subscriptions

get the subscription by ID

HGETALL triggers/{trigger-id as uuid}/subscriptions/{subscription-id as uuid}

e.g.

{   
   id: "{uuid}"   
   triggerId: "{uuid}"   
   service: "polls",   
   entity: "question",   
   entityId: "123",
   route: "studio.questions.activate",
   payload: { id: 123 }
}

and send message to route = "studio.questions.activate" with payload = { id: 123 }

After all subscribers are notified we should do a clean up and remove trigger and related entities.

Triggers API

"triggers.create"

Creates trigger.

interface Trigger {
   // uuid generated for each trigger
   id: string
   // human-readable name just for description purposes
   name: string
   // human-readable name just for description purposes
   description: string
   datasource: "sportradar"
   scope: "game"
   scopeId: string   
   conditions: TriggerCondition[]
}

"triggers.list"

List of triggers by {scope, scopeId} or other fields. RedisSearch is used to match hash fields in trigger object.

"triggers.subscribe"

Subscribes for trigger.

interface TriggerSubsription {
   // uuid generated on create
   id: string
   // linked trigger
   triggerId: string  
   // name of destination service, e.g. "polls"
   service: string
   // entity, e.g. "question"
   entity: string
   // entity id
   entityId: string
   route: string
   payload: object
}