@icgc-argo/program-service-proto NPM

Program Service

Objective

The central point to create and manage programs and maintain their metadata.

Development

Protocol Buffers

The Protocol Buffer file that define the GRPC interface can be found at ./src/main/proto/ProgramService.proto

NPM Program-Service-Proto Package

This service is supporting an NPM package which provides the proto file for NodeJS clients. The package is found here @icgc-argo/program-service-proto. This package should published for every Program-Service release with a matching version number. Whenever a proto change is made, it is important to update the version number for the npm package and for Program-Service.

To publish, you need to have a NPMJS account and be a member of the ICGC-Argo organization. With this, publish using make publish-npm. Make sure you have updated the version number in the package.json file.

Generate Protocol buffers Java files

./mvnw compile

Running the Service

Local

Database

Run postgres at port 5432

docker run --name postgres -d -p 5432:5432 postgres

Create database program_db

psql -h localhost -p 5432 -U postgres -c 'create database program_db'

Set up database schema (although it could be done automatically)

./fly.sh info
./fly.sh migrate

Run Spring Boot

./mvnw spring-boot:run

Docker

Getting Started

Since the EGO service is a dependency, having a local running docker instance can help the developement of unit tests and application code. Here are the steps for local developement:

Run make dev-start to start all the services needed for developement. These services DO NOT have security enabled.
Run the program-service in the debug profile. This is already configured to listen to the ports forwarded by the previous step. To find the correct port forwarding, run make dev-ps.

Additional Commands

Run make dev-ps to see a list of all forwarded ports
Run make dev-logs to see the logs of all the running dev services
Run make dev-stop to kill all services and remove the containers
Run make fresh-ego to restart a fresh and empty EGO service. Useful for when your EGO service contains alot of junk and you just want a clean slate.

Useful Services

Mail server
A mail server called MailHog is used to capture all email activity from the program service. The default url of the MailHog UI is http://localhost:9025.
Spring Boot Admin server
A spring boot admin server is used to monitor and manage the program service. The default url is http://localhost:9081.

Testing

Unit Testing

./mvnw test

Integration Testing

./mvnw verify

Note, the above command run unit tests as well. It uses your local database and run tests against DEV/QA environment.

Test the running instances at DEV/QA

First, port forward the port which is serving grpc services (make sure you have access to the clusters).

kubectl port-forward --namespace qa svc/program-service-qa 50051

Then, use your favorite grpc debugging tool to test individual services, for example, using grpc_cli to list all services and create program

grpc_cli list localhost:50051
grpc_cli call localhost:50051 CreateProgram "program: {name: 'programName', short_name: 'shortName'}"

If you prefer GUI interfaces, try bloomrpc

Test email

All emails sent are captured by mailhog for both the developer's local machine or the Jenkins that run CI/CD.

Mocking service using WireMock

Why use WireMock?

When writing Ego integration tests, instead of calling actual Ego endpoints (which would pollute Ego's database with test data, or test failure if the Ego service is shutdown), we can mock Ego endpoints with WireMock. WireMock starts a mock server at a given or random port. For example, to define a WireMock rule at a random port:

    @Rule
    public WireMockRule wireMockRule = new WireMockRule(options().dynamicPort());

The response can also be mocked:

    stubFor(get(urlEqualTo(url))
        .willReturn(aResponse()
          .withStatus(OK.value())
          .withHeader("Content-Type", "application/json")
          .withBodyFile(filename)));

Note that the "filename" is the name of a self-defined json file which mocks response body. All mocked json response files are located in src/test/resources/__files.

Demo Mode

In this mode, the local program service code is built and run as a docker container along with all the dependent services. This allows the whole system to be demoed without having any external network dependency. Use the following commands to manage a demo:

make demo-start: Starts all the services
make demo-stop : Stops all the services and removes the instances
make demo-logs : Shows the logs from all the services
make demo-ps : Shows all the running services and their ports