Transmetrix-payments NPM

Trans Metrix

What is Trans Metrix?

Trans Metrix is a information dashboard provided for logistics, shipping and transportation companies. It offers insight into most recent details about transportation situation in Europe.

Architecture

Project is split into two different repositories:

Backend application (this repository) - https://git.dss.lt.trans/trans-metrix/transmetrix-api
Frontend application - https://git.dss.lt.trans/trans-metrix/radar

alt text

Backend Application

This project is written in Typescript and runs on Nodejs. Server is an instance of apollo-server with Express running under the hood. It exposes both REST API and GraphQL Endpoint. GraphQL is being used for fetching data from MySQL.

Main responsibilities of the service:

Querying data and passing it to client application (GraphQL + SQL)
Managing user authentication (JWT)
Managing payments and invoices (Przelewy24 + Fakturownia)
Managing subscription system
Sending notification emails (Sendgrid)

Data sources

Databases - DEV:

Mysql: host: db-dev-externals-mysql.cebownpwwkzg.eu-west-1.rds.amazonaws.com database: db_dev_trans_metrix

Databases - RC:

Mysql: host: db-rc-externals-mysql.cebownpwwkzg.eu-west-1.rds.amazonaws.com database: db_rc_trans_metrix

Databases - PROD:

Mysql : host: db-prod-etl-dm.cebownpwwkzg.eu-west-1.rds.amazonaws.com database: db_prod_etl_tm

Tech Stack (npm packages worth mentioning):

Server - express, apollo-server-express
Authentication - passport, jsonwebtoken, bcryptjs
SQL - mysql2, TypeOrm
GraphQL - graphql, type-graphql
Emailing - @sendgrid/helpers, @sendgrid/mail
Scheduling tasks - cron
Requests validation - joi
Logging - winston, morgan
Testing - mocha, chai, chai-http, sinon, faker, nyc
Documentation - swagger-jsdoc, swagger-ui-express
Development tools - typescript, tsc, eslint, husky, nodemon, genversion, cross-env
Metrics - appmetrics, lynx, lynx-express, statsd, event-loop-monitor
Security - helmet, express-rate-limit
Caching - redis
Security - cors, helmet, rate-limit-redis, yieldable-json
Utility - dotenv (env variables), date-fns (workign with dates)

Full list of used packages can be found here.

Installation

Before installation, you need to specify environment variables in env.dev file. File itself is not saved in git system, ask project developers for more details.

The easiest way to run the application on your local machine is to use yarn. Additionally you will need to run a mysql docker container.

docker-compose -f docker-compose.test.yml up

yarn install
yarn build && yarn dev

Or via npm:

npm install

npm run build && npm run dev

You can also user docker and provided Dockerfile. Build image:

docker image build -t trans-metrix .

Run image:

docker run -p 4000:4000 trans-metrix

Now you can check if application is up: http://localhost:4000/status

To deploy application on DEV or PROD, first you need to update and push project version and corresponding tag. Remember to include code changes in separate commits. All env variables for DEV/PROD environments are set in separate project: https://git.dss.lt.trans/trans-metrix/terraform Now all you have to do is to update the project version and push the changes:

yarn version --[patch/minor/major]
git push --follow-tags

For patch version update you can simply run following script (Version Update and PUBlish):

yarn vupub

Then publish new version from CI/CD by choosing deploy DEV and deploy PROD respectively.

API REST documentation

Swagger After running the application, you can see all the available endpoints by accessing swagger: http://localhost:4000/swagger

Debugging

The project is already properly configured for debugging in terms of TS/JS files mapping etc. To debug the application locally you need to create Node.js configuration and specify entry file and ENV variables:

entry file:  build/index.js
env variables: NODE_ENV=dev

To attach to a remote Node.js process and debug it, you need to create "Attach to Node.js/Chrome" configuration (webstorm/IntelliJ), specifying host and port. Local:

localhost:9229

Dev - Node.js prevents debuggers from attaching to host specified by fqdn (hostname). First option is to create a ssh tunnel and map local port to the port we want to attach to, but due to trans restrictions, ssh access cannot be granted (multiple services available on single aws machine). Second option is to attach by directly specifying IP address. It is dynamic and changes daily, but can be queried with below command:

IP: echo "$(host trans-metrix-api-debug.service.dev-eu-west-1.aws.lt.trans | head -1 | awk '{print $NF}'):9229"
port: 9229

Testing

To run tests, type:

yarn build && yarn test

Things worth mentioning:

TypeORM is not actively supported anymore. Suggesting switiching to other ORM in the future.
Graphql has hard time dealing with big responses validation which additionally blocks the event loop - https://github.com/graphql/graphql-js/issues/723#issuecomment-379090872 Suggested solution is to use scalar types in gql queries and pass selected fields as query arguments.
Redis saves Typeorm results as strings. There is a bug with cached results of raw typeorm queries. It returns date as string which is not deserialized to JS object. Graphql complains about it and throws error when cached data is being returned. Simple fix is to check if returned date instance is Date object and if not create Date object based on its string representation. - https://github.com/typeorm/typeorm/issues/2794

Migrations

After changing SQL schemas of any models located in models/sql-security folder, you need to create migration which will autmatically update DB schema during next deployment. First step is to navigate to src/config/security.mysql.ts file, comment lines 25-29 (host-...-password that user env variables) and uncomment lines 31-35 (plaintext configuration of local db). Remember that local db must be running (check Installation section) and all previous migrations must be present there before you want to discover what changed (you can run yarn local BEFORE you make schema changes and local DB will be updated with current schemas). Then run:

yarn typeorm migration:generate -n <YourMigrationName>

Finally, add generated migration to migrations section in src/config/security.mysql.ts file. Remember to revert changes from src/config/security.mysql.ts before committing!

Notice! Remember to update documentation if made some changes!