txp-graphql-to-postman v1.0.7

Why

The primary purpose of this project is create a suite of functional tests to ensure that our service meets its external requirements by testing the entire system from end to end.

In order to achieve this, the system is treated as a black box and the tests exercise as much of the fully deployed system as possible, manipulating it through public interfaces such as GUIs and service APIs.

Given our new micro-services architecture includes more moving pieces these test-e2e tests will provide value by covering the possible gaps between our services.

There is a thorough design doc which was created by Annette which has a ton of great info on functional testing.

This project originally started as a clone of rigup/tp-test-collection but has diverged pretty drastically in the past week. One of my goals is to restructure parts of the repository to make them compatible with two test packages that were released by Annette: rigup/txp-test-variables rigup/txp-test-runner.

High Level Overview

The test suite generates a Postman collection that is then fed into Newman, which is a command line collection runner for Postman. Newman then processes the collection making each specified request against the current environment and then runs assertions against those requests. The output is similar to what you would see when running the specs in our service.

Usage

To add to the test suite you can either update an existing generate.ts file with a new query/mutation or you can create a new workflow directory with a new generate.ts file.

We can walk through an example of adding the supplierInvoices and supplierInvoice queries.

• We would first navigate to the queries directory and add supplierInvoices and supplierInvoice to the array of exports.

• Order is important here. Since we will need a valid supplierInvoiceUuid to query for a supplierInvoice we need to make sure that we add this to the array after the supplierInvoices query.

• This will allow you to use the response from the supplierInvoices query to set an environment variable storing the id to a valid supplierInvoice which can be used in the query.

• Update export default ['affiliations', 'projects', 'affiliation'];
• To export default ['affiliations', 'projects', 'affiliation', 'supplierInvoices', 'supplierInvoice']

• We will now run npm run test:build which will generate our test files for our supplierInvoices and supplierInvoice queries as well as update our postman-collection.json file.

• NOTE - npm run test:build will only generate new files, it will not update existing files. This is to prevent custom queries, variables, and tests from being overwritten.

• There should now be two additional subdirectories, supplierInvoices and supplierInvoice, in the queries directory.

• Take a look at the src/test/queries/supplier-invoices/supplier-invoices.ts file. You will see that some very basic tests and helper functions have been generated automatically. You will see nearly identical tests in src/test/queries/supplier-invoices/supplier-invoice.ts

• This is where you would add additional assertions to test the current request, we will leave these alone for now and go ahead and run the test suite.

• run npm run test:e2e:report

• You should see a html report pop open after the tests have finished running which you can use to inspect the results. It looks like we have one failing test attributed to the supplierInvoice query

• The error message reads "message": "invalid input syntax for type uuid: \"{{uuid}}\"", If you look at the src/test/queries/supplier-invoice/supplier-invoice.variables.ts file you will notice there is a variable that we never defined, this is why the test is failing.

• In order to fix this we will go to the supplierInvoices.ts file and set the required variable after that request has finished.

• Inside of the postmanTest function you will see a pre-populated variable responseData which returns a list of supplierInvoices

• We will grab the first supplierInvoice from the list and then set an environment variable uuid equal to the value of the id.

pm.environment.set('uuid', responseData.edges[0].node.uuid)

• Now run the test suite again npm run test:e2e:report and you should see that all of the tests are passing and the variable uuid is now set.

Script explanation:

1. Fetch the most recent version of our staging schema using the introspection query and then convert that into a valid GraphQl SDL schema.

   • src/generated/introspection-query.json

2. Using the scripts found in graphql-to-postman the entire schema is then converted into one large Postman collection which represents the entire federated graph.

   • src/generated/graphql-to-postman-collection.json

3. Begin to add queries, mutations, and workflows to the src/test directory.

4. For the query and mutations directories there is a root file called generate.ts which is where you can add any query or mutation that you would like to have included in the test collection.

5. Each workflow, for example onboard-client, follows a similar structure allowing you to define which requests you would like to include and also letting you specify the order in which they are processed.

If you were starting out with an empty project you would a directory structure similar to this:

└── test
    ├── mutations
    │   └── generate.ts
    ├── queries
    │   └── generate.ts
    └── workflows
        ├── onboard-client
        │   └── generate.ts
        └── onboard-worker
            └── generate.ts

6. generate.ts is just a simple array of GraphQl requests that looks like this

   • export default ['affiliations', 'projects', 'affiliation'];

7. After you create a new generate.ts file or add to an existing array you then run npm run test:build. This command will create a directory for each of the requests you specified and build three files in each of the directories. These generated files allow you to create and customize your collections and tests with little effort.

   Using Affiliation as an example you would see the following files:

   affiliation.graphql - This is the query that will be used when generating the final postman collection. You can edit this query at any time to remove or include arguments or fields. Once you have edited any of the generated files they will no longer be overwritten by the npm run test:build script.

   affiliaiton.variables.ts - This file contains an object which represents all of the required arguments for that particular request. These are formatted using handlebars by default which will allow you to programmatically set these values based on the results of previous requests.

   affiliaiton.ts - This file includes two functions; one is where you will write all of your assertions against the request and the other is a pre-request script that you can use to manipulate data and set environment variables that will be used during the request.

8. After you have all of your requests set you can run npm run test:build again which will iterate through each of the directories that were created and generate a new Postman collection based on a combination of the updated values and what ever customizations you have added.

9. The final step is to run npm run test:run; this will trigger Newman to process the newly created collection and will output the results of each of your assertions.