stepster v1.0.5

Stepster

Build testable, production ready step function applications.

• How do I use Stepster inside my AWS Lambda Function handler?

  You don't, Stepster becomes your AWS Lambda function handler.

• How does Stepster isolate the logic of the state machine from the logic of the step function?

  Simple, each step in your state-machine consumes a data object that was produced by the previous iteration (step) of the state machine. Before Stepster, those step functions were likely written "smartly", aware that they were apart of this State machine (or a complex outer jig function might decompose the output onto a final return object). With Stepster though, "step functions" become "step objects" and your smartly written logic can become simple and generic functions.

Usage

The simpliest (and most useless) way you could use Stepster would be to create a single step state-machine like this

exports.handler = new Stepster().handler

The single step is called the terminal step and it is present in any Stepster state machine. It essentially says, "No steps match the current conditions of the state machine and so we are done here."

Adding a step

Adding a step is simple, at minimum a step requires a condition function and a step function.

exports.handler = new Stepster()
  .addStep({
    condition: data => typeof data === 'undefined',
    step: data => {
      return 1
    }
  })
  .handler  

We can add types to our steps, and even add more steps by chaining off of our Stepster instance.

Step conditions are evaluated in the order they are added to your Stepster instance.

exports.handler = new Stepster()
  .addStep({
    type: 'FIRST_STEP',
    condition: data => typeof data === 'undefined',
    step: data => {
      return 1
    }
  })
  .addStep({
    type: 'INCREMENT_PREV_STEP',
    condition: data => data < 3,
    step: data => {
      return data + 1
    }
  })
  .handler

Now we have a step function that will run 4 times and output the following responses (a single response per step)

{
  "errors": {"consecutive": 0, "total": 0},
  "action": "FIRST_STEP",
  "complete": false,
  "data": 1
}

{
  "errors": {"consecutive": 0, "total": 0},
  "action": "INCREMENT_PREV_STEP",
  "complete": false,
  "data": 2
}

{
  "errors": {"consecutive": 0, "total": 0},
  "action": "INCREMENT_PREV_STEP",
  "complete": false,
  "data": 3
}

{
  "errors": {"consecutive": 0, "total": 0},
  "action": "TERMINATED",
  "complete": true,
  "data": 3
}

Note: The functions you write simply take as input and return as output the values that become the data property. The other properties on the object returned to the Lambda function are used by Stepster and also your AWS Step Function state machine to determine when the state machine has finished.

We could rewrite the example above to not include that 4th step TERMINATED by adding a completion condition to our second step:

exports.handler = new Stepster()
  .addStep({
    type: 'FIRST_STEP',
    condition: Stepster.condition.initialStep,
    step: data => {
      return 1
    }
  })
  .addStep({
    type: 'INCREMENT_PREV_STEP',
    condition: data => data < 3,
    completion: data => data === 3,
    step: data => {
      return data + 1
    }
  })
  .handler

In this example condition and completion appear to do the same thing, however one is ran to determine which step is be be ran and the other (completion) as to whether the entire state-machine should stop running after this step.

Anatomy of a Step

The steps in Stepster have a small set of configuration options:

{
  type: string?,             // (defaults to empty string)
  complete: bool?|function?, // (defaults to false)
  condition: function,
  step: function,            // feel free to use `async function` for step
  
  /* use config below only if running as a max-compute engine */
  breakLeft: bool?,
  breakRight: bool?,
  breakStart: bool?,
  breakEnd: bool?,
  breakBuffer: number?,
}

The important properties really are just type, complete, condition and step

Hooking into the Steps

For those who want to run logging functions, and you should. Simply chain off the Stepster instance with onSuccess and onError, passing in functions to do your logging. If you simply need to run something before the steps kick off, then use the beforeEachStep function which is good for 1 time pre-setup of the environment.

exports.handler = new Stepster()
  .beforeEachStep(() => {
    s3 = new AWS.S3()
  })
  .onSuccess((event) => {
    Logger.log(event)
  })
  .onError((event) => {
    Logger.error(event)
  })
  .addStep({ ...someStep1 })
  .addStep({ ...someStep2 })
  .addStep({ ...someStep3 })
  .handler

Condition and Completion

A condition is a simple method that tells if current data object state should invoke the step.

condition => (data, prevAction) => {
  /*
  data: return value from the previous step function
  type: previous step name, undefined if initial step. 
  */  

  // do some logic here
  return true
}

These are optional functions that make it easier to read what is being done in the condition.

Information about non-idempotent steps

The options breakBuffer, breakRight, breakLeft, breakStart or breakEnd control whether or not a step can be ran in a sequence. None of these conditions have effects on the first step ran in a particular Lambda execution. So if your step was configured to have a buffer of 30000 (30 seconds) and someone changed your lambda to have a max run time of 15 seconds, it would not break the state machine, that step would just wait until it could run as the first step in an execution

These configuration options translate roughly to, "return from the Lambda function immediately when...": - breakBuffer: int there are less than n milliseconds of Lambda execution - breakRight: bool this step finishes (stops even if next step has the same type as this one) - breakStart: bool the previous step was of a different type than this one - breakEnd: bool the next step is different type from this one This model is easy to think about if you imagine a state-machine with steps from top to bottom, but steps can also be ran multiple times in a row (in which case we imagine it moving left to right)

START                 - State machine starts
STEP: A1              - Step A runs
STEP: B1, B2, B3      - Step B runs 3 times
STEP: C1, C2          - Step C runs 2 times
STEP: D1              - Step D runs once
END                   - State machine ends

In the model above, Stepster could either run in 1 - 7 successful lambda functions. If steps of type B were configured to breakStart, then that changes to 2 - 7 successful lambda functions, since the first lambda must exit after A1 because steps of type B can run sequentially only when they are the starting function. If Bs were also breakRight then it would take 6 - 7 successful lambda functions, as the steps B1, B2, B3 all must break immediately after they complete.