Stsc-tsc NPM | npm.io

Smart Tariff, Smart Comparison Library

The smart tariff, smart comparison (STSC) project uses smart meter data along with tariffs to calculate cost and transform data to be used in presentation to user interfaces. The objective is to help consumers select the best tariff, track a current tariff forward and understand how in changing their energy consumption may effect the cost of energy.

This library provides Javascript and Typescript support for data access, calculations and general framework elements. It is intended to be used in browser and node.js environments - although the current demonstration of the library is through an Angular web application executing stsc-tsc code in the browser. This means the testing and structure are probably more mature for use in the browser, however both environments shoud run identically.

Example code and use as a stand alone library is available on ObservableHQ @joshuacooper. https://observablehq.com/collection/@joshuacooper/smart-tariff-smart-comparison There are a number of tutorials on how to use the library in the collections found there.

A simple server side example to demonstrate how the library can be used on a server is on Runkit https://runkit.com/ijosh/stsc-energy-example/2.0.2 This is not meant to be exhaustive as through the front end code we are trying to demonstrate the use of the library.

As a general direction of travel, smart meter data access is through Glowmarkt, a service platform run by Hildebrand. It is not necessary to use this data access route to use the library, however you may need to put more effort in the registration and management of your users, as well as a number of the supporting data elements like GSP codes.

Release POC4

This library works with a fourth proof of concept (POC4), supporting the data models for presentation within a larger STSC front end website. These Javascript libraries could be used to embed calculations within your own website and offer some insight into how the backend data is structured.

For this release the use cases are around using half hourly smart meter data from the DCC and a number of hypothetical tariffs taken from public sources or from energy suppliers. The source of data is Hildebrand's Glowmarkt data service, providing a binary file of consumption data.

Tariffs in scope are of three types:

Flat rate tariffs - these are typical single rate tariffs in pence per kilowatt hour
Time of Use (TOU) tariffs - rates (up to 48 per day) that change throughout the day, but are the same for every day. This mirrors many of the ECO7, ECO10 rates as well as rates targeted at electric vehicle owners. Natural progression would be for different day of the week rates until they begin to look more like truly dynamic rates.
Dynamic - each half hour has a specific rate that changes, the rate may only be known a day or two ahead, therefore there is little predictability in the price.

Each of these usually has an additional standing charge expressed in pence per day that is locked in for a longer contract period.

Calculating cost for various time periods is a key output of the library. To do this, it is a relatively simple application of a tariff structure to energy consumption data. This can be used for a number of purposes:

optimisation (at least historically) of tariff based on lowest cost; a brute force calculation is fairly quick within a browser however becomes more challenging as the number and complexity of the tariff (see calculation complexity for tariff types below)
tracking/monitoring - possibly save off the results and then come back to track if the savings happened in reality
scenarios - change the energy consumption and see what happens to the costs

if you would like to dive in and have a look at some examples in code, the ObservableHQ notebook https://observablehq.com/@joshuacooper/tariff-calculations works through some examples of the three different tariff structures.

Compute complexity and memory analysis

The implementation of the library is interesting to talk about in having sets of tariff types that will use the same energy input to calculate cost. For instance there may be 10 flat rate tariffs, 20 TOU and 30 dynamic tariffs to evaluate in order to chose the best tariff for the consumer. If the energy profile is changing, say I would like to consider what a smart EV charger may do to my cost given the same set of tariffs to evaluate, fast calculation becomes desirable if not necessary. Very complex and interacting scenarios challenge this further.

Although the scenarios themselves may not be fully coded in the library, the structure of the library should be considerate of the computational challenges and it is a goal to accommodate software like this on top of this library.

Calculating a flat rate tariff usually involves summarising data for a time period, such as one year or for 12 calendar months. The summary operation is an O(n) computation with n probably maximal for 13 months (396 days * 48 values = 19,008). Once the summary is complete, then new tariff calculations are O(1). For 10 flat rates, the complexity is 9 x O(1) + O(n).

TOU is fairly similar in complexity with a first pass of the data to put it into time buckets and then rates applied. In order to be as flexible as possible, we've opted for summarising the time period of interest into 48 half hourly buckets. For example a year's worth of data is summed within its half hourly slot. This allows for simple 48 x O(1) calculations for any TOU tariff.

Dynamic is more challenging in that every half hour has a different rate. There are not really any short cuts for calculation. Each tariff to be evaluated has an O(n) computational cost. The main approach has been to calculate all of the results at once for each tariff half hourly slot; this saves iterating for each tariff.

Timeseries

This is a base class that is used for storing and aggregating time series data. It assumes "perfect" data as higher level classes will know how to fill in the blanks. There are utility functions that calculate statistics within the time buckets so that when the aggregate function is called it knows how to reduce the data.

The timeseries class is used as a base class for many of the STSC classes as they are time series data. Even within the timeseries, when range is called, it returns a Timeseries, so then the timeseries functions are all available on a new instance of the data rather than mutating the underlying raw data that may be relied on by other parts of an application.

Full documentation of Timeseries will be done later as it is more important to see how the Timeseries is used in the higher level objects, therefore the concept is introduced here.

Loader functions for large timeseries, specifically Energy and Dynamic Tariffs are implemented in their respective classes as the data types and validation assumptions are specific to that higher level semantic.

At the moment to keep the API understandable and testable, each aggregate function is called atomically. This is a known inefficiency in the structure of the code, albeit very minor. A guture implementation may actually call a number of the aggregation functions as the main work is the loop, not the calculation. Presenting the aggregates back (sum, count, mean, variance) as an object is more efficient in CPU cycles only if those statistics are actually required together as a group.

STSC

At an application level we have implented an STSC class that can log on to the Glow platform service and retrieve a token that can be used for further calls on classes. Tokens are Java Web Tokens or JWTs as a structure.

Energy

Energy is a Timeseries that is populated from a binary object sourced from Hildebrand's Glow service. The binary is used because of the size of JSON that would be required, plus benefits in being portable and helping the backend systems service fewer requests.

The Energy object takes has a token property that is passed through to a server request. For testing and simplicity a resource_test token can be used that will return a known, but real smart meter data set. This populates data that methods can then act on. A simple example:

// require through local npm install - make sure your project has npm i stsc-tsc in package.json
var mystsc = require("stsc-tsc");

let energy = new mystsc.Energy();
energy._token = 'resource_test';
await energy.load();

// time stamped values, note the default aggregation is being used.
console.log(energy.byMonth());
console.log(energy.max());

The async call to load is to make sure the data has been loaded from the server request before any calculations are attempted. The other timeseries and energy methods are then available.

For example, max, min and mean are all against the complete set of energy data that has been loaded.

As an example (following on using the energy object made above), the consumption method helps extract and aggregate for displaying into a graph.

let  from = '2020-05-01T00:00:00+00:00';
let  to = '2020-06-01T00:00:00+00:00';

let newTS = energy.consumption({
	from:  from,
	to:  to,
	interval:  'PT30M',
	aggregateFunction:  mystsc.Timeseries.AGG_SUM,
});

console.log(newTS.export());

From and to are used to pass in start and end dates for processing. They are ISO formatted strings with timezone information. When the API uses start and end semantics those are expressed in milliseconds since epoch. Slot numbers if referring to half hours are 0 - 47 with 0 equal to midnight, the beginning of the day. Days are indicated by a daySlot, 0-6 with Monday as zero.

You can set different intervals to aggregate with using the ISO standards. In reality, use the PT not the P type ISO durations as localtime is used with P designators, whereas PT are relative to the initial timestamp and in UTC (this is Luxon not handling the P intervals correctly). Also make sure to keep the timezone information on the ISO date passed in to the object as you will get strange behaviour with local day light savings without it.

Aggregation functions are defined in the Timeseries base class and are referred to through the instance name of the library, the Timeseries class and then the aggregate function macro name e.g. mystsc.Timeseries.AGG_SUM.

Build

The build of the project is using Typescript and ES5 syntax in coding as classes. The package.json file has a couple of scripts, make sure to run npm install so they are available.

The index.ts exports the classes, with transpiling to index.js for ES5

git clone https://ijosh@bitbucket.org/LDNSFO/stsc-tsc.git
cd stsc-tsc
npm install
npm run build

This should build source code into a distribution folder, lib, that can be included in your projects or used in a bundled single file. For bundling we are using an external service, https://bundle.run/stsc-tsc@1.0.9 for instance. This uses rollup and browserify at the heart of its implementation. Just pass the relavent version number into Bundlerun to get a neat version for use.

Jest tests

Jest is used as the test framework with tests found by class in:

src/__tests__/

Each of the classes has regression tests associated with it. They are run as a part of the release scripts in npm. If tests fail, then a release can not be made.

To run the jest tests use the script in npm (look in package.json for the detailed command) by running:

npm run test

Transpile

We are using the tsc transpiler to Javascript. The typescript configuration is basic, but targets a module style that can be used across a browser and server environment.

{
  "compilerOptions": {
    "allowJs": true,
    "target": "es2015",
    "module": "commonjs",
    "moduleResolution" : "node",
    "declaration": true,
    "outDir": "./lib",
    "strict": true
  },
  "include": ["src"],
  "exclude": ["node_modules", "**/__tests__/*"]
}

NPM availability

Code is released for use in npm. Semantic versioning is used with POC4 being 1.0.x and patched updates replacing the x.

https://www.npmjs.com/package/stsc-tsc

Publication

This is published as a part of the BEIS sponsored Smart Tariff, Smart Comparison programme.

Copyright

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Glow Service

Please contact Hildebrand to gain access to smart meter data on a B2B basis. As a consumer you should be able to use the public data service to get access to your data.