@colabo-transform/i-json v0.2.3

Intro

@colabo-transform/i-json is a colabo utils puzzle.

It provides isomorphic part of the tool for transforming data (json) objects

To transform an object file you have to provide the object and transformation.

If you want to transform JSON files content you should look into @colabo-transform/b-json

The tool fully uses human friendly, extended JSON5, rather then the standard JSON format, which makes it possible to have easier management of json described transformations and config files, like commenting-out currently unnecessary sources, etc.

Language

It is based on the format of JSON Pointer RFC6901 but it extends it to support additional features (like addressing and iterating through array elements, etc)

Walking down algorithm

Algorithm follows paths, like:

• selection/id
• subfilters[*]/rel

It walks down the path parts, separated with / and matches (currently?) single source path, with one or many destination paths and applies transformations.

When it matches path part which has iterative part [*] it will spawn multiple sub-paths for each found iterative element in the source (or destination if source doesn't exist, as in the case of the ETransformOperation.ADD operation). It can have multiple iterative parts, like: subfilters[*]/rel/id[*].first

Examples

A lot of examples can be found in the unit-tests file: transform.test.ts but here is just a gist:

{
			title: "progressives eye texas race as test for toppling incumbents",
			text:
				"Henry Cuellar, D-Laredo, speaks during press ... more than month until the primary, the political implications for Cuellar are uncertain... \\n ",
			id: "dc9b097ba27dc155",
			time: "2022-01-23T14:00:00Z",
			hash: "785f3db1e4ad90bae2074a31492a8d62",
			url: ["https://apnews.com/article/henry-cuellar-texas-congress-crime-race-and-ethnicity-f98f3ee482f2cb9aefd9ed9edf7b6c52"],
			image: "https://storage.googleapis.com/afs-prod/media/4abd4b8001784114a5238361b0f66a27/2000.jpeg",
			subfilters: [
				{
					id: "61b76db98c90be3d859898a8",
					rel: 1,
					tags: [{ name: "tag-1" }],
				},
				{
					id: "61b76db98c90be3d85989877",
					rel: 0.8,
					tags: [{ name: "tag-1" }, { name: "tag-2" }],
				},
				{
					id: "61b76db98c90be3d85989885",
					rel: 0.9,
					tags: [{ name: "tag-1" }, { name: "tag-2" }, { name: "tag-3" }],
				},
			],
			tags: ["institutional-adoption,energy-stocks"],
			subfilter: "61b76db98c90be3d859898a8",
			selection: {
				id: "61b76db98c90be3d859898a8",
				title: "Institutional Adoption",
				type: "CRYPTOCURRENCY_TOPIC",
			},
		}

In TypeScript:

const transform5: ITransform = {
	src: "subfilters[*]/tags[*]/name",
	dest: "subfilters[*]/tags[*]/newName",
	operation: ETransformOperation.MOVE,
};

or JSON5:

{
	src: "subfilters[*]/tags[*]/name",
	dest: "subfilters[*]/tags[*]/newName",
	operation: ETransformOperation.MOVE,
}

or JSON:

{
	"src": "subfilters[*]/tags[*]/name",
	"dest": "subfilters[*]/tags[*]/newName",
	"operation": "MOVE",
}

will span over two level iterations (subfilters[*] array and tags[*] array) and iterate over all identified iterable elements of the arrays and at the final leaves will rename name property into newName property.

Operations

There are currently 4 operations:

• ETransformOperation.MOVE - moves elements from a src to dest paths
• ETransformOperation.DELETE - deletes elements from a src path (paths in future)
• ETransformOperation.ADD - adds value to the dest path (paths in future)
• ETransformOperation.CODE - runs a random code (TS/JS) to manipulate with the leaf sub-objects (NOTE: the protocol is still in beta as we explore all possible scenarios)

Transformations format

Transformation is described with ITransformation interface which (among others) has a list of transforms (ITransform) that should be applied.

TODO

path.lastValue not fully implemented

After last extensions path.lastValue is not fully implemented

iterative element as last path part (at the end of the path)

NOTE: It seems it is implemented to work well!

transformation-v0.0.2.transformations.ts

name: "upgrade-src-&-dest-path",

{
	name: "upgrade-src-&-dest-path",
	operation: "code",
	src: "transforms[*]",
	dest: "transforms[*]",
	// eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types, @typescript-eslint/no-explicit-any
	reference: async (sourcePathContext: IPathTransformationContext, destinationPathContext: IPathTransformationContext, transform: ITransform): Promise<any> => {
		console.log(`\tTransforming \`src\` and \`dest\` formats to JSON Pointer [RFC6901](https://datatracker.ietf.org/doc/html/rfc6901)`);
		console.log(`\t transform: ${JSON.stringify(transform)}`);
		console.log(`\t destinationPathContext.dataSubObject: ${JSON.stringify(destinationPathContext.dataSubObject)}`);

		if (destinationPathContext.dataSubObject.src) destinationPathContext.dataSubObject.src = transformPointer(destinationPathContext.dataSubObject.src);
		if (destinationPathContext.dataSubObject.dest) destinationPathContext.dataSubObject.dest = transformPointer(destinationPathContext.dataSubObject.dest);

		return destinationPathContext;
	},
},

Solution would be to add a newly crated value for IPathTransformationContext.currentPathPart that is NOT part of IPathTransformationContext.pathPartsAspects but it is a separate one pointing to the iterative element we are currently processing (for example transforms[1]):

export interface IPathPart {
	text: "transforms[1]";
	selector: "1";
	isIterable: false;
	index: number;
}

and IPathTransformationContext.dataSubObject referencing transforms.

we should probably also add a flag if we are in such a virtual state:

IPathTransformationContext.state = EPathTransformationContextState.ITERABLE_VIRTUAL_CURRENT