@x12i/helpers

@x12i/helpers

Small helper utilities for x12i projects.

Install

npm i @x12i/helpers

Helpers (library of abilities)

Each helper has a focused doc page under helpers-docs/:

• Objects mapper: helpers-docs/objects-mapper.md
• Record transform (protocol v2.0): helpers-docs/record-transform.md (v1 acceptance: helpers-docs/v1-acceptance-checklist.md)
• Record key analysis (identifier discovery): helpers-docs/record-key-analysis.md
• Link-based enrichment: helpers-docs/enrichment.md
• HTTP tools: helpers-docs/http-tools.md
• API contracts: helpers-docs/api-contracts.md
• API mapper / protocol bridge: helpers-docs/api-mapper.md
• Firebase RTDB (mongo-like): helpers-docs/firebase-rtdb.md
• Firebase Firestore (admin init + databaseId): helpers-docs/firebase-firestore.md
• Google Cloud Storage (object I/O): helpers-docs/gcs.md
• MongoDB (mongo-like collection API): helpers-docs/mongo.md
• Amazon S3 (object I/O, GCS-shaped API): helpers-docs/s3.md

Usage

Objects mapper

const { mapObject, createMapper, mapArray, deepGet, deepSet } = require("@x12i/helpers");

const source = { user: { first: "Jane", last: "Doe" } };

const mapping = [
  "user.first -> profile.firstName",
  "user.last -> profile.lastName",
  { template: "{user.first} {user.last}", to: "profile.displayName" }
];

console.log(mapObject(source, mapping));
// { profile: { firstName: 'Jane', lastName: 'Doe', displayName: 'Jane Doe' } }

You can also import the mapper directly:

const { mapObject } = require("@x12i/helpers/objects-mapper");

Record transform (protocol v2.0)

Declarative JSON record transformer — apply a versioned spec to reshape, validate, and enrich a single object.

Docs: helpers-docs/record-transform.md · Acceptance checklist: helpers-docs/v1-acceptance-checklist.md

const { transform, validateSpec, TransformError } = require("@x12i/helpers/record-transform");
// also available from the package root: require("@x12i/helpers")

const spec = {
  version: "1.0",
  operations: [
    { op: "move", from: "details.sev", to: "metadata.severity" },
    {
      op: "mapValue",
      path: "metadata.severity",
      mapping: { 0: "none", 1: "low", 2: "medium", 3: "high", 4: "critical" },
      default: "unknown",
    },
    {
      op: "rule",
      when: {
        type: "number",
        path: "metadata.priority",
        operator: "gte",
        compareTo: { value: 8 },
      },
      then: { action: "set", path: "metadata.escalate", value: true },
    },
  ],
};

const check = validateSpec(spec);
if (!check.valid) throw new Error(check.errors.join("; "));

const { record, valid, violations } = transform(
  { details: { sev: 3 }, metadata: { priority: 9 } },
  spec
);
// record.metadata.severity === "high"
// record.metadata.escalate === true

API

Export	Description
transform(record, spec)	Returns { record, valid, violations }. Input is never mutated.
validateSpec(spec)	Pre-flight structural validation → { valid, errors }.
TransformError	Thrown for invalid specs, onMissing: "error", strict mapValue, cast onError: "error", etc.

Version policy: only "1.0" is supported. validateSpec rejects other versions; transform throws TransformError before execution.

Operations: move, rename, mapValue, cast, drop, rule, validate. Rule actions (v1): set with value or valueFrom.

Tests: npm test -- test/recordTransform.acceptance.test.js

Record key analysis (identifier discovery)

Three-tier heuristics for finding identifier-like fields and cross-collection relationships. Tier 1 scoring is JSON-driven (keywords, regex, anti-keywords) via defaultHeuristics.json.

Docs: helpers-docs/record-key-analysis.md

const {
  analyzeSingleRecordKeys,
  analyzeCollectionKeys,
  mapCrossCollectionRelationships,
} = require("@x12i/helpers/record-key-analysis");

const candidates = analyzeSingleRecordKeys({
  _id: "507f1f77bcf86cd799439011",
  user_id: "a1b2c3d4-e5f6-4789-a012-3456789abcde",
  name: "Jane",
});

const classified = analyzeCollectionKeys(rows, candidates.map((c) => c.property));

const joins = mapCrossCollectionRelationships(orders, "user_id", {
  users: { primaryKeys: ["id"], records: users },
});

Tests: npm test -- test/recordKeyAnalysis.test.js

Link-based enrichment

Generic join helper for linking enrichment collections onto target records — by IP, normalized username, id, etc. Two layers: enrichLink (core matcher) and enrichFromMetadata (metadata-driven rule runner).

Docs: helpers-docs/enrichment.md

const { enrichLink, enrichFromMetadata, applyPatch } = require("@x12i/helpers/enrichment");

const linkedUsers = enrichLink({
  enrichmentCollection: usersArray,
  enrichmentLinkKey: "ip",
  target: someAsset,
  targetLinkKey: "data.assetIp",
  matchMode: "exact",
  fieldMap: [
    { from: "user", to: "user" },
    { from: "vsys", to: "vsys" },
  ],
});

const patch = enrichFromMetadata(someAsset, { objectType: "assets", dataDir: "./data" });
const merged = applyPatch(someAsset, patch);

Tests: npm test -- test/enrichment.test.js

HTTP tools (request ⇄ curl, base URLs)

const {
  requestToCurl,
  curlToRequest,
  buildUrl,
  createBaseUrlClient,
} = require("@x12i/helpers");

const curl = requestToCurl({
  method: "POST",
  url: "https://api.example.com/v1/users",
  headers: { authorization: "Bearer TOKEN" },
  body: { name: "Jane" },
});

// curl -X POST -H 'authorization: Bearer TOKEN' -H 'content-type: application/json' --data-raw '{"name":"Jane"}' 'https://api.example.com/v1/users'
console.log(curl);

const req = curlToRequest(curl);
// { method: 'POST', url: 'https://api.example.com/v1/users', headers: { authorization: 'Bearer TOKEN', 'content-type': 'application/json' }, body: '{"name":"Jane"}' }
console.log(req);

console.log(buildUrl("https://api.example.com/", "/v1/users", { limit: 10, tags: ["a", "b"] }));
// https://api.example.com/v1/users?limit=10&tags=a&tags=b

const api = createBaseUrlClient("https://api.example.com/", {
  headers: { authorization: "Bearer TOKEN" },
});
console.log(api.curl("/v1/health"));

Direct import:

const { requestToCurl } = require("@x12i/helpers/http-tools");

API contracts + protocol bridge

Define protocol-aware API contracts (REST/GraphQL/JSON-RPC/SOAP/custom) with optional request/response validation and auth application.

const { defineContract, createRegistry } = require("@x12i/helpers/api-contracts");
const { createBridge } = require("@x12i/helpers/api-mapper");

const registry = createRegistry();

registry.register({
  name: "restGetUser",
  protocol: "rest",
  endpoint: "/api/users/{id}",
  method: "GET",
  auth: { type: "bearer" },
});

registry.register({
  name: "gqlGetUser",
  protocol: "graphql",
  endpoint: "https://api.example.com/graphql",
  query: "query GetUser($userId: ID!) { user(id: $userId) { id fullName } }",
  auth: { type: "bearer" },
});

const bridge = createBridge({
  source: "restGetUser",
  target: "gqlGetUser",
  registry,
  requestMapping: [{ from: "pathParams.id", to: "variables.userId" }],
  responseMapping: ["user.id", "user.fullName -> user.name"],
  credentials: { target: { token: "TOKEN" } },
});

// bridge.call({ pathParams: { id: "42" } })

Direct imports:

const { createBridge, batchCall, chainBridges } = require("@x12i/helpers/api-mapper");
const { defineContract, createRegistry } = require("@x12i/helpers/api-contracts");

Firebase Realtime Database (native) — mongo-like helper

This helper uses the Firebase Admin SDK and wraps RTDB with a familiar Mongo-ish API (findOne, insertOne, updateOne, etc.).

This is not a Firestore helper. If your upstream app uses Firestore, you should initialize firebase-admin with service account credentials and use admin.firestore() (Firestore does not use FIREBASE_DATABASE_URL).

1. Create .env (see .env.example) and provide:

• GOOGLE_SERVICE_ACCOUNT_BASE64: Base64-encoded Google service account JSON
• FIREBASE_DATABASE_URL: the exact Realtime Database URL for that same Firebase project (copied from Firebase Console → Realtime Database → Data)

Credentials (Base64 only)

All helpers use a Base64-encoded Google service account JSON.

Why:

• Works in serverless / edge environments
• No filesystem dependency
• Safer secret handling

How to generate:

cat service-account.json | base64

Then set:

GOOGLE_SERVICE_ACCOUNT_BASE64=...

Migration

Before:

FIREBASE_SERVICE_ACCOUNT_PATH=.secrets/firebase-service-account.json

After:

cat .secrets/firebase-service-account.json | base64

GOOGLE_SERVICE_ACCOUNT_BASE64=<result>

1. Use it:

const { initFirebaseRtdb } = require("@x12i/helpers/firebase-rtdb");

const fb = initFirebaseRtdb(); // loads .env by default
const users = fb.collection("users");

const { insertedId } = await users.insertOne({ email: "a@b.com", name: "Ami" });
const user = await users.findOne({ _id: insertedId });

await users.updateOne({ _id: insertedId }, { $set: { name: "Ami N." }, $inc: { loginCount: 1 } });
await users.deleteOne({ _id: insertedId });

Important:

• Service account + RTDB URL must match the same Firebase project.
• Do not guess the RTDB URL (firebaseio.com vs firebasedatabase.app, region, etc.). In Firebase Console, open the project referenced by your service account JSON (project_id), then copy the Database URL shown under Realtime Database → Data and set FIREBASE_DATABASE_URL to that exact value.

Optional: to fail fast (avoid long hangs in test suites) set FIREBASE_CONNECT_TIMEOUT_MS or pass connectTimeoutMs:

const fb = initFirebaseRtdb({ connectTimeoutMs: 5000 });

If you want native RTDB query features (server-side indexed filtering), use query():

const result = await users
  .query({ orderByChild: "email", equalTo: "a@b.com", limitToFirst: 10 })
  .find();

Firebase Firestore — admin helper

If you want Firestore (not RTDB), use this helper. Firestore does not require a database URL; it authenticates via service account credentials.

1. Set GOOGLE_SERVICE_ACCOUNT_BASE64 (Base64-encoded service account JSON).

2. Ensure Firestore exists for that project:

• Firebase Console → Firestore Database → Create database (this creates the default database for the project)

1. Use it:

const { initFirebaseFirestore } = require("@x12i/helpers/firebase-firestore");

const { firestore } = initFirebaseFirestore(); // loads .env by default
const docRef = firestore.doc("catalogs/appId");

await docRef.set({ hello: "world" }, { merge: true });
const snap = await docRef.get();
console.log(snap.exists, snap.data());

Optional: fail fast (avoid long hangs in test suites) set FIRESTORE_CONNECT_TIMEOUT_MS (or FIREBASE_CONNECT_TIMEOUT_MS) or pass connectTimeoutMs:

const fb = initFirebaseFirestore({ connectTimeoutMs: 5000 });

Defaults:

• FIRESTORE_DATABASE_ID defaults to catalox
• FIREBASE_PROJECT_ID defaults to x12i (normally inferred from the service account’s project_id, but you can override it)

If you want a different Firestore database id, set FIRESTORE_DATABASE_ID or pass databaseId:

initFirebaseFirestore({ databaseId: "catalox" });

Google Cloud Storage (GCS)

Server-side object storage only: import the subpath @x12i/helpers/gcs so the main package entry does not load @google-cloud/storage.

1. Set GOOGLE_SERVICE_ACCOUNT_BASE64 (Base64-encoded service account JSON).

2. Set a bucket name (defaults to storage_bucket; STORAGE_BUCKET / GCS_BUCKET also supported) or pass { bucket }.

3. Minimal usage:

const { createGcsClient } = require("@x12i/helpers/gcs");

const gcs = createGcsClient({
  // envPath defaults to ".env"; loads GCS_BUCKET / GCS_OBJECT_PREFIX if set
  prefix: "my-app/uploads",
});

await gcs.uploadObject("user/1/avatar.png", buffer, { contentType: "image/png" });
const data = await gcs.readObjectBuffer("user/1/avatar.png", { maxBytes: 2 * 1024 * 1024 });
const { items, nextPageToken } = await gcs.listObjects({ prefix: "user/1/", maxResults: 50 });

When to use this helper instead of @google-cloud/storage directly:

• You want shared x12i conventions for loading .env and service account material (same Base64-only credential mechanism as RTDB/Firestore helpers).
• You want small, stable helpers for upload, buffered read (with a max size), list pagination, existence checks, and metadata, plus typed errors (GcsNotFoundError, GcsPermissionDeniedError, GcsFailedPreconditionError) for mapping to your own issue types.

Optional emulator (CI or local): set STORAGE_EMULATOR_HOST (for example with fake-gcs-server). See test/gcs.emulator.test.js for an example command line.

Live tests in this repo: set STORAGE_LIVE_TESTS=1 (or legacy GCS_LIVE_TESTS=1) and STORAGE_BUCKET (or GCS_BUCKET) to a dedicated test bucket; see test/gcs.live.test.js.

MongoDB (mongo-like, RTDB-shaped API)

Subpath: @x12i/helpers/mongo. Exposes collection, ref, db, and client with the same mongo-like surface as @x12i/helpers/firebase-rtdb (insertOne, find, findOne, updateOne, query, per-doc get/set/update/delete, etc.). Uses MONGO_URI (or MONGODB_URI) from .env after loading via the same env loader as other helpers.

initMongoDb is async (MongoDB must connect over the network) unlike synchronous Firebase RTDB init:

const { initMongoDb, getMongoDb } = require("@x12i/helpers/mongo");

const mongo = await initMongoDb(); // reads MONGO_URI from .env
const users = mongo.collection("app/users");
const { insertedId } = await users.insertOne({ email: "a@b.com" });
await mongo.ref("app").remove(); // drops collections under path prefix (see docs)

Live tests: MONGO_LIVE_TESTS=1 — see test/mongo.live.test.js.

Amazon S3 (object I/O)

Subpath: @x12i/helpers/s3. Same method names as GCS: createS3Client, uploadObject, openReadStream, readObjectBuffer, listObjects, objectExists, deleteObject, getObjectMetadata, setObjectMetadata, plus matching error classes (S3NotFoundError, …). Configure S3_BUCKET, AWS_REGION, optional S3_ENDPOINT / S3_FORCE_PATH_STYLE for MinIO or LocalStack, and standard AWS credentials.

const { createS3Client } = require("@x12i/helpers/s3");

const s3 = createS3Client({ prefix: "my-app/uploads" });
await s3.uploadObject("k.txt", "hi", { contentType: "text/plain" });

Live tests: S3_LIVE_TESTS=1 — see test/s3.live.test.js.

Helper documentation

See helpers-docs/ for per-helper docs (what it is, why it exists, usage, and any .env keys it reads).

API

• mapObject(source, mapping, opts)
• createMapper(mapping, opts)
• mapArray(sourceArray, mapping, opts)
• deepGet(obj, path)
• deepSet(obj, path, value)
• requestToCurl(req)
• curlToRequest(curlCommand)
• buildUrl(baseUrl, pathOrUrl, query)
• createBaseUrlClient(baseUrl, defaults)
• defineContract(config)
• createRegistry()
• createBridge(config)
• batchCall(bridge, paramsList, opts)
• chainBridges(...bridges)
• initFirebaseRtdb(options)
• getFirebaseRtdb()
• initFirebaseFirestore(options)
• getFirebaseFirestore()
• enrichLink(options)
• defaultNormalize(value)
• enrichFromMetadata(target, options?)
• applyPatch(target, patch)
• getByPath(obj, path) / setByPath(obj, path, value) / stripArrayMarker(path)

Subpath only (not re-exported from the root package):

• @x12i/helpers/gcs: createGcsClient(options), GcsHelperError, GcsNotFoundError, GcsPermissionDeniedError, GcsFailedPreconditionError, GcsObjectTooLargeError
• @x12i/helpers/mongo: initMongoDb(options), getMongoDb()
• @x12i/helpers/s3: createS3Client(options), S3HelperError, S3NotFoundError, S3PermissionDeniedError, S3FailedPreconditionError, S3ObjectTooLargeError