@sogody/experiment-framework NPM

Experiment Framework

The experiment framework is a collection of libraries that facilitates the process of building tests using Javascript, SCSS and other modern technologies. Since some of these technologies are not natively supported by the browser, the code needs to go through a build process before you’re able to preview changes in the console or paste them into the experimentation tool (Adobe Target). This framework provides a somewhat opinionated approach to structuring the code for your experiments, while enabling you to use cutting-edge front-end technologies and retaining backwards compatibility.

Minimum System Requirements

Node 14
Yarn

Creating a blank experiment

In order to use the framework, you need to run the following:

npx @sogody/create-experiment [EXPERIMENT_NAME]

This will create a new folder with a basic structure for building experiments, along with the dependencies on the framework.

Your first experiment

After the above command is completed, the new experiment is set up inside the folder with the name you assigned above.

You will be writing your code within files like src/js/v1/index.js. Most frequently, this corresponds to a variation in your experiment.

Each file that you need to build, should be listed in the entryPoints array within config.json. Usually, an entrypoint corresponds to a variation, but there might be other things listed as entrypoints, such as common tracking code, shared code, etc.

In terms of structure, you are free to write whatever JavaScript code within your JS files. However, to reap the benefits of the framework, we suggest that you enclose everything within a runScript call.

Building and previewing your experiment

As a start, keep in mind that there are two parts to the process:

Building your code as you're building the experiment. This is facilitated by running yarn run start -e[ENTRYPOINT_INDEX] once, and keeping the process running. This basically watches your *.js, *.scss files for changes, and as soon as it detects a change, it builds and bundles the code. Additionally, to aid your development workflow, it copies the output to your clipboard so that you can paste it directly into the target page and observe the changes.
Building the code when you want to deploy the experiment to your desired experimentation tool. This is facilitated by running yarn run build, which generates a separate file for each entrypoint in your dist/ folder, according to your configuration in config.json.
(Optional) Building your code and refreshing a browser window as you're working on the experiment. This is very similar to 1., except that the commmand you need to run is yarn run dev -e[ENTRYPOINT_INDEX]

Installing on Linux-based systems

If you are using any of Linux distributions, make sure to install xclip in order to be able to use copy to clipboard feature. This can be installed with the following command sudo apt-get install xclip.

Code Structure

JavaScript

We use @babel/preset-env so that we can use the latest JavaScript features while still ensuring that our code works across all modern browsers. For example, you can make use of the ES6 modules to better organize your code instead of having to have all your code within a single file.

Styling

One of the fundamental features of the framework is the ability to write stylesheets within its own files, while still ensuring that they get bundled within a single JS function that you can paste into the browser console.

We use SCSS instead of CSS. The framework transpiles SCSS into CSS and bundles it together with the final output. The SCSS syntax uses the file extension .scss. With a few small exceptions, it’s a superset of CSS, which means essentially all valid CSS is valid SCSS as well. Because of its similarity to CSS, it’s the easiest syntax to get used to and the most popular.

We use CSS Modules to ensure proper scoping of stylesheets and class names. This is to avoid clasesh with existing classes in the pages you're targeting with your experiment. Hence, adding those classes to your DOM would require the following extra steps:

// This becomes: your-experiment-name--someClass
.someClass { 
    color: blue; 
}

import styles from './styles.scss';

...

// Adding a class
document.querySelector('.someSelector').classList.add(styles.someClass);

// Querying by a scoped class 
document.querySelector(`.${styles.someClass}`);

If you ever need to target an already existing DOM element using CSS, simply enclose your styles within a :global scope like:

:global { 
    .existing-class-name { 
        display: none;
    }
}

React

While you can build experiments by using standard DOM APIs, for more complex ones, we recommend that you resort to a component-based framework like Preact for better code organization, faster development cycles, and (possibly) less bugs stemming from poor state management. Our go-to approach is to include Preact in all of our mid-to-large experiments, hence, we've baked it inside the framework. To use Preact within your experiment:

1. Write your component(s). We suggest you keep them in src/components so that you can reuse them across variations (and possibly across experiments as well):

const Banner = ({ title }) => { 
    return <h1>{title}</h1>;
}

export default Banner;

2. Render them in your variation like:

import { render, h } from 'preact';
import Banner from '../components/Banner';

runScript(() => { 
    withPreact(() => { 
        const container = document.createElement('div');
        document.body.insertBefore(container, document.body.firstElementChild);

        render(<Banner />, container);
    });

});

Note: We wrap our React render code within withPreact in order to ensure that Preact is in the scope when the component is rendered. This is especially useful when loading Preact from the CDN, rather than bundling it. If you'll always gonna bundle it, feel free to not enclose your render calls within withPreact.

ESLint

This project has ESLint enabled along with some useful rules. On every build (either development or production), ESLint checks your files for conformity to the enabled rules. It mostly follows Airbnb Javascript Style Guide, with minor modifications that can be observed in .eslintrc. If there's an error, the build does not succeed. In addition, the production build (yarn run build) does not succeed if there are warnings. We suggest that you stick to the ESlint rules to aid your consistency across different team members and experiments. However, if you ever need to customize it, you can edit the .eslintrc configuration file.

As most of the errors are auto-fixable, it is recommended that you install some plugin for your favorite editor to make this easier. See below some recommended plugins for different editors/IDEs:

VSCode - see this. Make sure, you opt-in to use the ESLint from node_modules. After installing it, VS Code will clearly show errors and provide ways to fix them.
Sublime - TBD
Atom - TBD

Stylelint

This project has Stylelint enabled along with some useful rules. On every build (either development or production), Stylelint checks your files for conformity to the enabled rules. It mostly follows Airbnb CSS Style Guide, with minor modifications that can be observed in .stylelintrc. If there's an error, the build does not succeed. In addition, the production build (yarn run build) does not succeed if there are warnings. Please do not stylelint-ignore errors and warnings without discussion / approval.

As most of the errors are auto-fixable, it is recommended that you install some plugin for your favorite editor to make this easier. See below some recommended plugins for different editors/IDEs:

VSCode - see this. Make sure, you opt-in to use the Stylelint from node_modules. After installing it, VS Code will clearly show errors and provide ways to fix them.
Sublime - TBD
Atom - TBD

Customizing the framework using `config.json`

You can customize experiment framework according to your needs by altering the configuration options in the config.json file.

targetUrl - the page that should be loaded in the browser when using yarn run dev development feature.
entryPoints - entry files to your experiment. You can think of this concept as variations. You would likely have each variation into a separate file like src/v1/index.js, src/v2/index.js. Needless to say, there's no restriction in following that structure, and also no restriction in using this for experiments only, hence, entryPoints as a more generic name.
runtime - a collection of attributes related to experiment during runtime.
- globalObject - each experiment is bound to a single global object (window), which makes it easy to debug during runtime, as well, as enables feature such as ensuring only a single instance of the experiment code ever exists and runs in the page. The default name for this object is sgd, i.e. window.sgd and your experiment code would be bound to window.sgd[EXPERIMENT_NAME]. However, you cna freely change this to something of your liking, like your company or team abbreviation.
- includeEmergencyBrake - set this to true if you need to be able to put an emergency brake to your experiment code. This is applicable only in Adobe Target and is useful in scenarios where the developers do not have the necessary access to deactivate tests, yet, they notice some urgent break of functionality that needs to be addressed.
- bundlePreact - you can choose whether you want to bundle preact with your generated code, or whether you want to load it from CDN. For performance reasons, it might make sense to try and load Preact from JSDelivr during runtime.
development / production - Using these, you can override configuration values listed above for different environments. For instance, you might choose to bundle preact on a development build, but not a production one. Similarly, you might choose to have terser for production but not development.
- terser - You can customize minification, by supplying configuration options for terser. If you leave it null, it means that minification is disabled.

Helper functions

The framework provides some helper functions that you may find useful while building your experiments. You can see them in src/js/helpers.js.

Async support

To use async/await please ensure you import 'regenerator-runtime' in each script, which utilized asynchronous calls. However be aware that this increases the payload significantly (up to 30Kb). In cases where payload size is important it is recommended to resort to promise calls.