0.0.12 • Published 3 years ago
@bstruct/gcp-pubsub-to-bigquery-example-transformer v0.0.12
gcp-pubsub-to-bigquery-example-transformer
Example how to customize the pubsub message transformation step in component gcp-pubsub-to-bigquery
One of the exported classes must implement the interface IPubsubMessageTransformer
so that the transformation can be implemented inside the transform
method.
The class must contain a public constructor with no parameters expected.
Here's the example in the file exampleTransformerSync:
import { IPubsubMessage, IPubsubMessageTransformer } from '@bstruct/pubsub-message-types';
import { MyNewMessageType } from './types/myNewMessageType';
import { MessageBody } from './types/messageBody';
export class ExampleTransformerSync implements IPubsubMessageTransformer<MyNewMessageType> {
transform(message: IPubsubMessage): MyNewMessageType[] {
const body = JSON.parse(message.data.toString()) as MessageBody;
if (!body.orderId) {
throw Error("Mandatory property 'OrderId' is defined in the message body.");
}
const newMessage = {
orderId: body.orderId,
status: body.status,
date: body.date,
__messageId: message.messageId
} as MyNewMessageType;
return [newMessage];
}
}
The method can be syncronous or asynchronous, as demonstrated in the file exampleTransformerAsync.ts.
Deployment
This code is meant to be deployed to a npm package registry. In the file github-artifacts-publish.yml is how this package is deployed to github's package registry.
name: Publish package
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: 16
registry-url: https://npm.pkg.github.com/
scope: '@bstruct'
- run: npm install
- run: tsc
- run: jq "del(.devDependencies)" package.json > package2.json ## do not install development packages
- run: rm package.json
- run: mv package2.json package.json
- run: cp package.json ./dist/src
- run: cp LICENSE ./dist/src
- run: cp README.md ./dist/src
- run: npm ci
- run: npm publish ./dist/src/ --access public ## only publish the code files, not the test ones
env:
NODE_AUTH_TOKEN: ${{secrets.GITHUB_TOKEN}}
0.0.12
3 years ago