@zeus-technology/zeus-prime NPM

zeus-primejs

Scripts for publisher integration with Zeus Prime.

This repo holds the code for all of the Zeus Prime clientside javascript that publishers will need to install. This includes the wrapperless js file hosted at https://primejs.zeustechnology.com/ and the npm package for @zeus-technology/zeus-prime.

This repo also includes the code and code generators for the Prebid module that can be copied to the WPMedia/Prebid.js fork of Prebid. From there the script can be submitted as a PR to Prebid.

All this code is built from the same core functions provided in this repo and is tested in a sample page running in Cypress using olympus helpers. This ensures equivalent functionality across each artifact for all our Zeus Prime customers.

Local Development

For local development, this repo mostly will be interacted through testing with Jest and Cypress and basic yarn builds.

yarn install
yarn build
yarn test
yarn start:site:script // Run in a separate terminal from yarn cy
yarn cy

Install

This repository is using the latest yarn version, current 3.X (or berry) with the pnp system and what they call zero install. This means the contents of the entire dependency tree is actually checked in via the .yarn/cache directory. Unlike node_modules, pnp with the cache directory is significantly smaller in size and since all dependencies are checked, installs take only a few seconds and can be performed totally offline (the install process is only required to setup a couple of symlinks for bridging non pnp packages to work with pnp).

yarn install

Because of this setup, any upgrades or new packages that are installed will not only update the yarn.lock and package.json files, they will also update the .pnp files at the top level and the .yarn/cache. All of these files and folders should be committed.

Upgrading packages

This repository has a devDependency on @wpmedia/zeus-olympus for the cypress testing. This requires authentication with the Github package registry to upgrade this package. To do so, you will need to create a personal access token in github with at least read:packages permissions. See GitHub docs on personal access tokens for more information.

Once you have the token, you can apply it and upgrade the package using the following:

sh scripts/upgrade-olympus.sh 'YOUR GITHUB TOKEN'

If you need to install a specific tag or version, you can use a second param to specify that:

$ sh scripts/upgrade-olympus.sh 'YOUR GITHUB TOKEN' @next
$ sh scripts/upgrade-olympus.sh 'YOUR GITHUB TOKEN' @3.0.5

IMPORTANT: Do not commit any changes to the .yarnrc.yml file or you can accidentally leak your personal token. If there are changes in that file from running this command (like due to a failure) then just use git to revert all changes to this file.

Builds

yarn build

The build for primejs uses parcel. This lets us easily create multiple export targets, which currently are the wrapperless prime.js file, the Prebid module file, and the npm library package. The configuration for these targets are in the package.json. See https://parceljs.org/features/targets/ for more information on multiple targets.

When running build, the wrapperless script files will be created in dist/wrapperless and the Prebid build will be created in dist/prebid. The npm module when published will be all of dist folder except for dist/prebid, *.test.js files, and *.map files.

Prebid Build

The Prebid build can also be run independently using yarn build:prebid. The prebid build uses rollup instead of parcel because with Parcel we can't create human readable source files. Parcel will always mangle at least function names, so we use rollup because it will generate human readable files with the module: "es" option. This is required since we are copying the generated file to Prebid.js repo as source code for PR Review by Prebid.

In the rollup configuration we are also making a couple of changes to the source files. This is done because Prebid has different logging mechanisms we need to use, the API requests need to run through their ajaxBuilder instead of fetch, and we need to remove all our internal code comments in the file.

You can see the modifications to the script in the rollup.config.mjs file replace section, where we replace strings in the source file during processing. We replace the console.*() calls and instead bind to the Prebid log*() functions. We change the log format from [Zeus Prime]: to zeusPrimeRtdProvider: which matches the name expected in Prebid logs. Finally we change the import of sendRequest from topics.js to sendPrebidRequest. sendRequest is a simple wrapper around fetch, which is what NPM and vanilla PrimeJS script use, but the sendPrebidRequest uses the Prebid ajaxBuilder function which is a wrapper around XMLHttpRequest. That function then takes the responses from the ajax call to ensure it acts like the fetch call does so topics.js can work in all scripts.

We also remove comments from all our code and then run the final output through prettier to ensure the code is formatted correctly (the comment removal generates a lot of empty lines). Any comment with the jsdoc tag @preserve will be left in the final output. This is configured in the rollup configuration file cleanup and prettier plugins.

Last, there is a banner inserted into the top of the generated script as well with a comment header and the required imports from Prebid that will be required in our script. That is inserted from the rollup config output.banner property.

Browser Compatibility

We also use a browserlist configuration specifically for the browser prime.js file. This ensures parcel will transpile with all required code to run on the browsers specified, in our case in any browser with greater than 4% market share and the last two versions of every browser. We will exclude however all Opera mini browsers and IE 11. This targets about 91% of the market based on stats provided by browsers list: https://browserslist.dev/?q=PiAwLjQlLCBsYXN0IDIgdmVyc2lvbnMsIG5vdCBpZSAxMSwgbm90IG9wX21pbmkgYWxsLCBub3QgZGVhZA%3D%3D

If required, we can add additional targets for wrapperless easily to capture legacy browsers. By adding the following target to the targets list we can build a version that covers more browsers at the expense of file size and it will generate a new dist/wrapperless-legacy with the output code:

    "wrapperless-legacy": {
      "source": "src/prime.js",
      "engines": {
        "browsers": "> 0.1%"
      }
    },

Note: the s3 syncing script and release workflow file would need to be updated to support and sync the new folder as well under a legacy folder in s3.

Tests

This repo uses jest and cypress for testing. Unit tests of all functions will be done in jest and then integration tests of the primejs artifacts are in cypress.

Jest

All unittests are run through Jest, and during the CI/CD coverage reports are also uploaded to codecov. The goal is to aim for as close to 100% as possible on all functions in the repo.

To run the jest tests, use the yarn test command.

yarn test

When developing you can also use the jest watch command to put Jest in watch mode.

yarn test --watch

Note: The test command will pass all options right to Jest, so you can use any Jest option as well.

Creating Test files

Each js file that has code will have a test file in the same folder, named after the file being tested with the .test.js file extension. For example, for the file src/setTargeting.js the test file would be src/setTargeting.test.js.

Each test file should have at least on describe function call that has the path to the function being tested. If a file has multiple functions being tested, then each function should have it's own describe block.

import setTargeting, { setTopicsTargeting } from "./setTargeting"

describe("src/setTargeting:", () => {
    describe("setTargeting()", () => {
        // it() tests go here
    }

    describe("setTopicsTargeting()", () => {
        // it() tests go here
    }
})

From there add your it tests as needed for that function.

Cypress

This repo uses Cypress for integration tests. These tests are run against all variations of our scripts automatically in CI/CD on every Pull Request and push to main and production.

To run them locally, you will first need to serve one one of the example html files then run the cypress command itself.

To run the tests against the prime.js script version for instance, you would run:

yarn start:site:script

to start the scripts. Then in another terminal run cypress:

yarn cy

This will load cypress in a new window and ask which browser you would like to test against. Select that browser for cypress to automatically open a new instance of that browser with the cypress tests. Now you can select the spec file you want to run.

You can also go directly to a browser you want to test:

yarn cy --browser chrome

If you want to run the entire test suite (like is done in CI/CD), use the yarn cy:run command.

For Prebid, we also need to include two environment variables due to a couple differences in how Prebid runs versus the other vanilla PrimeJS scripts. Namely, Prebid runs logging with extra prefix strings and the name of the Zeus Prime prefix is changed to match the name in Prebid and it doesn't use debug messages so our debug messages are printed out to console.info instead of console.debug in Prebid. The two variables are CYPRESS_TEST_ENV=prebid and CYPRESS_LOG_PREFIX="zeusPrimeRtdProvider: ". These are set by default to script and [Zeus Prime]: which are what the NPM and script versions use. Note that the environment variables in the test files themselves are TEST_ENV and LOG_PREFIX, but when used on the command line the CYPRESS_ prefix is required.

So the full commands to run cypress interactive mode and the ci run mode are:

CYPRESS_TEST_ENV=prebid CYPRESS_LOG_PREFIX="zeusPrimeRtdProvider: " yarn cy
CYPRESS_TEST_ENV=prebid CYPRESS_LOG_PREFIX="zeusPrimeRtdProvider: " yarn cy:run

Test Targets and Running Locally

We currently run the same suite of Cypress tests against both the PrimeJS script and the NPM module in CI/CD. This is done by generating a running server using Parcel based on either build and then run the suite against that served html file. Since the html file is always served the same for both PrimeJS, Prebid, and the NPM module, Cypress is able to run without modification of the source, though Prebid does require two extra environment variables.

You can run these tests locally as well. The NPM module and Prebid module requires a build step to be run first, but the PrimeJS script does not require any builds. To build the NPM module, we will generate a folder that contains a simple node project. This is done through a shell script run at the root of the project:

sh ./scripts/generate-npm-test.sh

For Prebid, we would run a similar script, though before running we need to ensure the prebid module is generated first so the script can access it (the NPM module doesnt not require a prebuild).

yarn build:prebid
sh ./scripts/generate-prebid-test.sh

These scripts will create sister folders to your repo by default called prime-js-npm-cypress-test and prime-js-prebid-cypress-test, though you can use other folders by providing a single parameter that is the path to the folder you want to be created, eg: sh ./scripts/generate-npm-test.sh ~/temp/npm-test. The NPM generator will generate the following files:

- package.json
- src/
  - index.js
  - index.html
- .parcel

Then it will run the yarn installs to install all the required packages and link the project locally to zeus-primejs to simulate downloading from NPM. It will also create a symlink to the folder in your repo called ./node-test. What results is a simple project that imports Prime as a node module and then calls run from the html file in that created project.

For the Prebid generator, it will first clone the entire Prebid.js repo into that generated folder and run the npm install. Then it will copy the build Prebid module, dist/prebid/zeusPrimeRtdProvider.js into the Prebid cloned repo folder modules/ and execute a build on Prebid. The build is run with three modules included in Prebid: zeusPrimeRtdProvider our script, rtdModule which is required code for Rtd Providers like us, and appnexusBidAdapter because Prebid requires at least one bid adapter (though we do not configure it nor do we render an ad in the test files). Once the prebid.js bundle is created we copy that file back into the zeus-primejs repo as dist/prebid/prebid.js. The src/test-sites/prebid.site.html will import that file to run prebid on the test site (and includes a full prebid config embedded in that html page as well).

Once the test target is prepped, you will need to run the local server to serve the html file Cypress will use to test. There are three start scripts in the package.json to start the PrimeJS script and the NPM module html files that serve the test page:

yarn start:site:script
yarn start:site:npm
yarn start:site:prebid

You can now run Cypress as explained in the previous section (yarn cy, yarn cy:run, or for Prebid CYPRESS_TEST_ENV=prebid CYPRESS_LOG_PREFIX="zeusPrimeRtdProvider: " yarn cy).

Note: We cannot run the generate scripts for NPM and Prebid as a yarn script, the runs will fail because of some path and import issues with Yarn. So they must be run as shell scripts.

Publisher Installation and Usage

Installation instructions for Zeus Prime can be found in the Install.md file. It is the source of truth for the documentation that will live on the Zeus Prime onboarding site.

Release Process

This repo uses semantic-release with the matching github action. This means that releases are automatic and tied to pushes to specified branches.

We use semantic versioning, so we follow the major.minor.patch version scheme. Deciding what version number to bump when a release occurs is automatically handled by semantic-release which checks for all commits between the last release and the new one, looking for conventional commit messages that describe the changes. More on commit structure in the Commits section below. It will change the parts of the version in the following conditions:

major: The major version is bumped if any commit ends with the line BREAKING CHANGE: <description>
minor: The minor version is bumped if any commit message starts the type feat.
patch: The patch version is bumped if any commit message starts with the type fix or perf.

Production Releases

To create a production release, make a branch from main and submit a PR into the production branch that specifies this is a production release PR. Once approved, merge the PR into production to create the release.

Any push to production branch will automatically run the semantic-release process which analyzes all the commits since the last tag on that branch and checks if those commits should generate a release (see Commits below). If a release should be generated it will generate a new version number, tag the release in github and upload release notes, and deploy the code to S3 (for the JS wrapperless script) and publish to NPM. The PrimeJS wrapperless and Prebid script files are also uploaded as artifacts to the Github release page.

In S3, the release action will upload the prime.js script to multiple places as well, and do so for both the production and staging S3 buckets: zeus-primejs-production and zeus-primejs-staging. The push will overwrite the script at /latest/prime.jswith the new version and also update the/latest/latest_version.txtfile to display the new version number. The major version folder will also update as well with a file with the exact version number, such as/v2/prime-2.0.0.js. Then the major version head andlatest_version.txt` is also updated, so in the 2.0.0 example the following files will be updated in S3 for both buckets:

latest/prime.js
latest/latest_version.txt
v2/prime-2.0.0.js
v2/prime.js
v2/latest_version.txt

You can get the production releases on the following URLs and npm commands:

yarn add @zeus-technology/zeus-prime
npm install @zeus-technology/zeus-prime

For Prebid, you will also need to go to the Github release page for the created release and download the zeusPrimeRtdModule.js file. Place that file into the modules directory of the Prebid fork and create a PR on a branch named release-vX.Y.Z where X.Y.Z are the numbers for the Prime JS release. Once merged into Prebid fork we will follow the Prebid guidelines to create a PR from our fork into Prebid itself.

Post Release

Once a production release is made, we will need to update the ZeusPrime plugin in zeus-lib. To upgrade, clone the zeus-lib repo and navigate to the zeus-core and zeus-neo packages. In each package, run yarn upgrade @zeus-technology/zeus-prime@latest and commit the changes to package.json and yarn.lock files. Create a PR for this change to merge into main so the next Zeus Lib release will include the new Zeus Prime version.

Maintenance Releases

If an already published version needs a bug fix, then we can deploy a fix specifically for that version. This can only be done for versions that have another major or minor version after the tag that needs to be updated. For instance if production is 2.1.4 You can update any version prior to 2.1 but you cannot push a fix to 2.1.X as that would have to come from main.

To create a maintenance release, you will create a branch from the tag you need to update, for example maybe 1.4.5. We will name the branch the same as the version but replace the patch version with an x instead. So the branch in this case would be 1.4.x.

Now make any fix: commits required on a PR from that branch (feat or BREAKING CHANGE commits will be blocked from releasing. See Commits for more information on the commits). Once the PR is approved and merged into the branch 1.4.x semantic release will issue a patch release to that version. It will tag, build, and deploy as normal with two exceptions. NPM will not update the latest tag to push this as the latest release. And for S3 push, ONLY the single version file will update, for instance: /v1/prime-1.4.5.js. It will not update the prime.js files in the latest or vX folders.

Once the deploy is done you can safely remove the branch.

Pre-Releases

All other releases will be pre-releases. Meaning they will follow the same exception for npm as the maintenance releases where they do not updated the latest tag in npm but they will mark the release as a prerelease and update the tag for whatever branch is configured to push as, ie dev, beta, or rc. The tag in github will also be suffixed with the tag, ie 2.4.3-dev.

In S3, these will also only deploy to the staging bucket and they will only deploy to the configured tag folder for the release, ie. zeus-prime-staging/dev/prime.js. This will not update latest or add any versions to the major version folder.

The following branches will deploy pre-releases:

main - Any push to the main branch will deploy on the dev tag, so npm install @zeus-technology/zeus-prime@dev or https://primejs-test.zeustechnology.io/dev/prime.js
beta - Any push to a branch named beta will deploy on the beta tag, so npm install @zeus-technology/zeus-prime@beta or https://primejs-test.zeustechnology.io/beta/prime.js
rc - Any push to a branch named rc will deploy on the rc tag, so npm install @zeus-technology/zeus-prime@rc or https://primejs-test.zeustechnology.io/rc/prime.js

Contributing

To contribute to the project, you will need to follow a few systems to ensure things work smoothly and releases include the correct commits in the automatically generated changelog.

Commits

We use Conventional Commits in this repo. Using this format for our commits ensures that semantic-release can properly generate a changelog for each release. This also helps to ensure consistent and clear commit messages.

This format is generally:

<type>[optional scope]: <description>

[optional body]

[optional footer(s)]

Type is one of the following: (anything marked No Release will skip releasing when committed against a releasable branch)

feat - A new feature. This will bump the minor version.
fix - A bug fix. This will bump the patch version.
perf - Performance improvement. This will bump the patch version.
docs - Update to documentation. No release.
chore - Catch all for other actions that don't affect code or other systems. No Release.
build - Update related to the build system or dependencies. No Release.
ci - Update related to CI/CD configurations and scripts. No Release.
refactor - Refactor commit. No Release.
style - Update to code style with no changes to meaning of the code. No Release.
test - Adding or updating test files. No Release

The scopes we use for primejs can also be one of the following (scope is not required but it helps to separate things in the changelog and additional scopes can be added):

wrapperless
npm
prebid
README
jest
cy

Description is the description of the commit, this is what is used in the automatically generated changelog so it should be clear and concise.

You can provide additional details on the commit as well to help clarify usage or the description by adding additional paragraphs below the first commit line.

If this is a breaking change and should increment the major version number, then you MUST provide the following as the last line of the commit:

BREAKING CHANGE: <description of change>

Git Commit Linter

Each commit, through a git hook, is run through a linter to ensure the commits match the proper format. This will ensure all commits are formatted correctly. If the commits do not follow the guidelines of convential commits, then it will block the commit and an error will be displayed that tells you what parts of the commit did not pass the linter.

It might look something like this:

⧗   input: Added docs for release and commit linter
✖   subject may not be empty [subject-empty]
✖   type may not be empty [type-empty]

✖   found 2 problems, 0 warnings
ⓘ   Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint

VSCode Plugin

There is a VSCode plugin to assist with users committing via VSCode. This will guide you through the process of selecting the type of commit, scopes, and other information as needed.

If this is not automatically installed, then you can find the extension at https://marketplace.visualstudio.com/items?itemName=vivaxy.vscode-conventional-commits

Git Command Line (Commitizen)

Commitizen is a helper that when installed will bootstrap git with helpers for generating conventional commits via the command line. This is installed via husky as a commit hook that when using the command line and it will guide you through crafting a conventional commit. Its a fully interactive shell program and will look similar to the following:

? Select the type of change that you're committing: (Use arrow keys)
❯ feat:     A new feature
  fix:      A bug fix
  docs:     Documentation only changes
  style:    Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
  refactor: A code change that neither fixes a bug nor adds a feature
  perf:     A code change that improves performance
  test:     Adding missing tests or correcting existing tests
(Move up and down to reveal more choices)
? Select the type of change that you're committing: docs:     Documentation only changes
? What is the scope of this change (e.g. component or file name): (press enter to skip) readme
? Write a short, imperative tense description of the change (max 86 chars):
 (60) updated with info on commitizen/commitlint and prod releases
? Provide a longer description of the change: (press enter to skip)
 Also added a release bage to the top of the readme.
? Are there any breaking changes? No
? Does this change affect any open issues? No

One commitizen is done it will open the editor usually used for commit messages with the formatted message prefilled in so you can verify and make any adjustments.

If you already supplied a valid message and commitizen still runs, you can CMD+C once to exit commitizen and complete the commit (provided the commitlinter passes after commitizen is exited).

Husky

Husky is used for our precommit hooks. These hooks ensure that things like the linter and tests are run before every commit. Currently we do the following on every commit:

Run eslint
Run jest
Run prettier
Run commitizen (if command line)
Run commitlint

Currently, the husky configuration is in the .husky folder. There is a shell script in that folder called pre-commit that is installed as a git hook once yarn install is complete.

Because husky is setup on postinstall script, it needs to be disabled when we build for npm, otherwise we would throw errors trying to install husky in the npm package. To avoid this, we use pist package to disable and enable the install when packing for npm. See https://typicode.github.io/husky/#/?id=yarn-2 for more information.

Note: if you run git commit and find that there is a failure related to husky, you can run yarn install once to reinstall the scripts and amend the install.

You can disable the hooks for a commit (usually only on a merge commit with conflicts) using the --no-verify flag in git.

git commit -m "No husky" --no-verify

Pull Requests

Like most teams, we use a Pull Request (PR) process to merge code with reviews and checks required to merge code. Most development PRs will target the main branch which is where active development code lives. Our PR review process is also based heavily on the zeus-console process, so that will be useful as well for additional information on pull requests: https://main--606c656f9e0b0a0021a7ce10.chromatic.com/?path=/story/documentation-processes-pr-review-process--page#peer-review-process.

When creating a PR, please review and follow the template provided in the details of the PR. This helps ensure all of the information required for a quick review are already provided to avoid delays in gathering data for things like how to test and what has actually changed.

When the PR is submitted it will automatically start running any checks required to pass for merge and notify the zeus-prime team that a PR is available for review. Once you have at least 1 approval, all required checks have passed, and all discussions are resolved, then you can merge the PR in.

Please be sure to add the correct milestone and labels to the request as well when submitting. Also, add yourself as the assignee of the PR.

Size Limit Check

We also deploy a project called size-limit in this repo which is run during the CI/CD process for a Pull Request. This will check the current branch being submitted with the target branch and measure the size of the compiled and minified scripts and library bundles. It will then check the differences in sizes to ensure that the PR does not exceed threshold limits for script size. This is to ensure that we are aware and keep file sizes small since the goal of Prime is to be extremely lightweight and fast. If this check fails then either the PR must be refactored to lower the sizes, or a special exemption will need to be added or made by the Zeus Prime team which will require a team discussion.

core-js

@infinitebrahmanuniverse/nolb-_zeu

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago