experimentation-js-sdk v1.0.1

Experimentation JS SDK

[Experimentation is a modular Feature Flagging and Experimentation platform.

This is the Javascript client library that lets you evaluate feature flags and run experiments (A/B tests) within a Javascript application.

• Lightweight and fast
• Supports both browsers and nodejs
• Local targeting and evaluation, no HTTP requests
• No flickering when running A/B tests

Installation

yarn add @turingenterprises/experimentation-js-sdk

or

npm i --save @turingenterprises/experimentation-js-sdk

Steps to configure if npm install fails: 1. Create GitHub access token. Make sure to select "read:packages" scope. 2. Run the following commands:

npm config set "@turingenterprises:registry" "https://npm.pkg.github.com/turingenterprises"
npm config set //npm.pkg.github.com/:_authToken {access-token}

Quick Usage

import { TuringExperimentation } from "@turingenterprises/experimentation-js-sdk";

// Create a TuringExperimentation context and pass two arguments
// init params like apiKey, projectId, env, polling, ttl
// Pass GB context object to initilize GB instance
// Export instance and use anywhere in application
const initParams = {
    apiKey: 'apiKey',
    projectId: 'projectId',
    env: 'env',
    polling: 'bool',
    ttl: 'number',
  };
const context = {
      trackingCallback: (experiment, result) => {
      console.log('experiment', experiment);
      console.log('result', result);
      console.log('result', experimentation.getAttributes());
   };
export const experimentation = new TuringExperimentation(initParams, context);

// Simple boolean (on/off) feature flag
if (experimentation.isOn("my-feature")) {
  console.log("Feature enabled!");
}

// Get the value of a non-boolean feature with a fallback
const color = experimentation.getFeatureValue("button-color", "blue");

The TuringExperimentation Context (Second argument in TuringExperimentation)

The TuringExperimentation constructor takes a number of optional settings.

Features

If you already have features loaded as a JSON object, you can pass them into the constructor with the features field:

new TuringExperimentation(initParams,
{
  features: {
    "feature-1": {...},
    "feature-2": {...},
    "another-feature": {...},
  }
})

Attributes

You can specify attributes about the current user and request. These are used for two things:

1. Feature targeting (e.g. paid users get one value, free users get another)
2. Assigning persistent variations in A/B tests (e.g. user id "123" always gets variation B)

The following are some comonly used attributes, but use whatever makes sense for your application.

new TuringExperimentation(initParams,
{
  attributes: {
    id: "123",
    environment: "prod",
    loggedIn: true,
    deviceId: "abc123def456",
    company: "acme",
    paid: false,
    url: "/pricing",
    browser: "chrome",
    mobile: false,
    country: "US",
  },
});

If you need to set or update attributes asynchronously, you can do so with setAttributes(). This will completely overwrite the attributes object with whatever you pass in. Also, be aware that changing attributes may change the assigned feature values. This can be disorienting to users if not handled carefully.

Tracking Callback

Any time an experiment is run to determine the value of a feature, we call a function so you can record the assigned value in your event tracking or analytics system of choice.

new TuringExperimentation({initParams},
{
  trackingCallback: (experiment, result) => {
    // Example using Segment.io
    turingLogger.info("Experiment Evaluate", {
      experimentId: experiment.key,
      variationId: result.variationId,
    }, 'Experiments');
  },
});

Feature Usage Callback

TuringExperimentation can fire a callback whenever a feature is evaluated for a user. This can be useful to update 3rd party tools like NewRelic or DataDog.

new TuringExperimentation({initParams},{
  onFeatureUsage: (featureKey, result) => {
    console.log("feature", featureKey, "has value", result.value);
  },
});

The result argument is the same thing returned from experimentation.evalFeature.

Note: If you evaluate the same feature multiple times (and the value doesn't change), the callback will only be fired the first time.

Using Features

Every feature has a "value" which is assigned to a user. This value can be any JSON data type. If a feature doesn't exist, the value will be null.

There are 4 main methods for evaluating features:

if (experimentation.isOn("my-feature")) {
  // Value is truthy
}

if (experimentation.isOff("my-feature")) {
  // Value is falsy (null, 0, "", or false)
}

// Get the value with a fallback for when it's null
const value = experimentation.getFeatureValue("my-feature", 123);

// Get detailed information about the feature evaluation
const result = experimentation.evalFeature("my-feature");

The evalFeature method returns a FeatureResult object with more info about why the feature was assigned to the user. It has the following properties:

• value - The value of the feature (or null if not defined)
• source - Why the value was assigned to the user. One of override, unknownFeature, defaultValue, force, or experiment
• ruleId - The string id of the rule (if any) which was used to assign the value to the user
• experiment - Information about the experiment (if any) which was used to assign the value to the user
• experimentResult - The result of the experiment (if any) which was used to assign the value to the user

Feature Definitions

The feature definition JSON file contains information about all of the features in your application.

Each feature consists of a unique key, a list of possible values, and rules for how to assign those values to users.

{
  "feature-1": {...},
  "feature-2": {...},
  "another-feature": {...},
}

Basic Feature

An empty feature always has the value null:

{
  "my-feature": {}
}

Default Values

You can change the default assigned value with the defaultValue property:

{
  "my-feature": {
    defaultValue: "green"
  }
}

Override Rules

You can override the default value with rules.

Rules give you fine-grained control over how feature values are assigned to users. There are 2 types of feature rules: force and experiment. Force rules give the same value to everyone. Experiment rules assign values to users randomly.

Rule Ids

Rules can specify a unique identifier with the id property. This can help with debugging and QA by letting you see exactly why a specific value was assigned to a user.

Rule Conditions

Rules can optionally define targeting conditions that limit which users the rule applies to. These conditions are evaluated against the attributes passed into the TuringExperimentation context. The syntax for conditions is based on the MongoDB query syntax and is straightforward to read and write.

For example, if the attributes are:

{
  "id": "123",
  "browser": {
    "vendor": "firefox",
    "version": 94
  },
  "country": "CA"
}

The following condition would evaluate to true:

{
  "browser.vendor": "firefox",
  "country": {
    "$in": ["US", "CA", "IN"]
  }
}

If a condition evaluates to false, the rule will be skipped. This means you can chain rules together with different conditions to support even the most complex use cases.

Force Rules

Force rules do what you'd expect - force a specific value for the feature

// Firefox users in the US or Canada get "green"
// Everyone else gets the default "blue"
{
  "button-color": {
    defaultValue: "blue",
    rules: [
      {
        id: "rule-123",
        condition: {
          browser: "firefox",
          country: {
            $in: ["US", "CA"]
          }
        },
        force: "green"
      }
    ],
  }
}

Gradual Rollouts

You can specify a coverage value for your rule, which is a number between 0 and 1 and represents what percent of users will get the rule applied to them. Users who do not get the rule applied will fall through to the next matching rule (or default value).

This is useful for gradually rolling out features to users (start coverage at 0 and slowly increase towards 1 as you watch metrics).

// 20% of users will get the new feature
{
  "new-feature": {
    defaultValue: false,
    rules: [
      {
        force: true,
        coverage: 0.2
      }
    ]
  }
}

In order to figure out if a user is included or not, we use deterministic hashing. By default, we use the user attribute id for this, but you can override this by specifying hashAttribute for the rule:

// 20% of companies will get the new feature
// Users in the same company will always get the same value (either true or false)
{
  "new-feature": {
    defaultValue: false,
    rules: [
      {
        force: true,
        coverage: 0.2,
        hashAttribute: "company"
      }
    ]
  }
}

Experiment Rules

Experiment rules let you adjust the percent of users who get randomly assigned to each variation. This can either be used for hypothesis-driven A/B tests or to simply mitigate risk by gradually rolling out new features to your users.

// Each variation gets assigned to a random 1/3rd of users
{
  "image-size": {
    rules: [
      {
        variations: ["small", "medium", "large"]
      }
    ]
  }
}

Weights

You can use the weights setting to control what percent of users get assigned to each variation. Weights determine the traffic split between variations and must add to 1.

{
  "results-per-page": {
    rules: [
      {
        variations: ["small", "medium", "large"],
        // 50% of users will get "small" (index 0)
        // 30% will get "medium" (index 1)
        // 20% will get "large" (index 2)
        weights: [0.5, 0.3, 0.2]
      }
    ]
  }
}

Tracking Key

When a user is assigned a variation, we call the trackingCallback function so you can record the exposure with your analytics event tracking system. By default, we use the feature id to identify the experiment, but this can be overridden if needed with the key setting:

{
  "feature-1": {
    rules: [
      {
        // Use "my-experiment" as the key instead of "feature-1"
        key: "my-experiment",
        variations: ["A", "B"]
      }
    ]
  },
}

Hash Attribute

We use deterministic hashing to make sure the same user always gets assigned the same value. By default, we use the attribute id, but this can be overridden with the hashAttribute setting:

const experimentation = new TuringExperimentation({initParams}, {
  attributes: {
    id: "123",
    company: "acme",
  },
  features: {
    "my-feature": {
      rules: [
        // All users with the same "company" value
        // will be assigned the same variation
        {
          variations: ["A", "B"],
          hashAttribute: "company",
        },
        // If "company" is empty for the user (e.g. if they are logged out)
        // The experiment will be skipped and fall through to this next rule
        {
          force: "A",
        },
      ],
    },
  },
});

Coverage

You can use the coverage setting to introduce sampling and reduce the percent of users who are included in your experiment. Coverage must be between 0 and 1 and defaults to 1 (everyone included). This feature uses deterministic hashing to ensure consistent sampling.

{
  "my-feature": {
    rules: [
      // 80% of users will be included in the experiment
      {
        variations: [false, true],
        coverage: 0.8
      },
      // The remaining 20% will fall through to this next matching rule
      {
        force: false
      }
    ]
  }
}

Namespaces

Sometimes you want to run multiple conflicting experiments at the same time. You can use the namespace setting to run mutually exclusive experiments.

We do this using deterministic hashing to assign users a value between 0 and 1 for each namespace. Experiments can specify which namespace it is in and what part of the range 0,1 it should include. If the ranges for two experiments in a namespace don't overlap, they will be mutually exclusive.

{
  "feature1": {
    rules: [
      // Will include 60% of users - ones with a hash between 0 and 0.6
      {
        variations: [false, true],
        namespace: ["pricing", 0, 0.6]
      }
    ]
  },
  "feature2": {
    rules: [
      // Will include the other 40% of users - ones with a hash between 0.6 and 1
      {
        variations: [false, true],
        namespace: ["pricing", 0.6, 1]
      },
    ]
  }
}

Note - If a user is excluded from an experiment due to the namespace range, the rule will be skipped and the next matching rule will be used instead.

Typescript

When using getFeatureValue, the type of the feature is inferred from the fallback value you provide.

// color will be type "string"
const color = experimentation.getFeatureValue("button-color", "blue");

When using evalFeature, the value has type any by default, but you can specify a more restrictive type:

// result.value will be type "number" now
const result = experimentation.evalFeature<number>("button-size");