sproutward-demeter-lima v1.0.0
In order to run this locally, it is necessary to get two JWT files. Either use existing default names and location in project root or customize names and location of these files. If customizing location and name, update .local_env.sh with the new name and location of these files. Each file represents a JWT file for service accounts in demeter-247120. Additionally, if you update the name and the location remains in the project, update .gitignore so they are not checked in; these files should never be checked in.
Service accounts required for development:
- default demeter service account: demeter-app-credentials.json
- demeter secrets service account: demeter-secrets-credentials.json
Setup environment variables running .local_env.sh: cmd> source ./.local_env.sh
Before running lima locally, pick a function for lima to process. This is done by setting environment variables for the function. cmd> source ./.local_enc_converse_marketmappings.sh
Run lima locally Pay attention to port and use ifconfig to get IP for local OS: cmd> node --inspect node_modules/@google-cloud/functions-framework --target=limaProcessingPubSub --signature-type=event
How to create a request to functions?
- Use template to build payloads and assign data payload to data.data
- See these tests for an idea of how to pass a payload: https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/master/functions/helloworld/test/sample.integration.pubsub.test.js#L26-L58
- This link is to section in function framework describing other parts of a payload handled like context: https://github.com/GoogleCloudPlatform/functions-framework-nodejs/blob/de6ef3874ae703084d5b7566ecde743a8f80bd36/src/invoker.ts#L431-L447
- marketing mappings data requires being Base64 encoded. This is an online utility to encode payloads: https://www.base64encode.org/
Marketing Mappings:
- runs as pubsub subscriber
- Jobs from postman will send to emulator which will pass on "message" to function
{ "data": { "data": "<>" } }
For a marketing mapping job, something like this: Job without encoding is: {"config": "prod/config_marketing_mappings","publish": false} Base64 encoding is: eyJjb25maWciOiAicHJvZC9jb25maWdfbWFya2V0aW5nX21hcHBpbmdzIiwicHVibGlzaCI6IGZhbHNlfQ== { "data": { "data": "eyJjb25maWciOiAicHJvZC9jb25maWdfbWFya2V0aW5nX21hcHBpbmdzIiwicHVibGlzaCI6IGZhbHNlfQ==" } }
For Hopper Manual or SFTP, something like this: For a marketing mapping job, something like this: { "data": { "data": { "name": "ConverseNA_Adobe_LastTouch_20200516-20200522.zip", "bucket": "converse.staging.demeter-sftp.hy66h7.sproutward.com" } } }
Production deploy command: gcloud functions deploy preprocessMarketingPubSub --trigger-resource preprocess-marketing-mappings --trigger-event google.pubsub.topic.publish --env-vars-file .env.yaml --region=us-central1 --runtime=nodejs10 --timeout=540
4 years ago