1.0.0 • Published 5 years ago

livecore-api v1.0.0

Weekly downloads
6
License
-
Repository
github
Last release
5 years ago

LiveCORE

A set of APIs to the CORE entities in LiveStories.

NOTE: Deployment is done through CircleCI

Getting up and running

npm install

Then start all required services, build the DB, schema and Indices, and pre-populate the DBs and finally create entitlements.

npm run services:up

# You want to wait for Elasticsearch to be up (it takes a few seconds)
# Run this CURL until you get a response that starts with "yellow". It will be red first...:
# yellow open .marvel-2018.02.01 1 1 44 0 1.1mb 691.7kb
curl http://localhost:9200/_cat/indices

# Then proceed...
npm run db:create
npm run db:migrate

# Same thing Elasticsearch, it needs to catch up!
# Run curl until all indices are marked "yellow". You should eventually get something like
# yellow open topics             5 1   0 0  970b    575b
# yellow open dashboards         5 1   0 0  812b    575b
# yellow open charts             5 1   0 0  970b    575b
# yellow open stories            5 1   0 0  776b    460b
# yellow open autocomplete       5 1   0 0  970b    575b
# yellow open datasets           5 1   0 0  812b    575b
# yellow open .marvel-2018.02.01 1 1 243 0 1.3mb 609.2kb
curl http://localhost:9200/_cat/indices

npm run db:seed:all

NOTE: you need to install docker and docker-compose FIRST!

Then start livecore:

npm start

Controlling services

change

npm services:up    // Starts all services
npm services:stop  // stops all services but doesn't destroy them (i.e. == suspend )
npm services:down  // permanently destroy all services and associated storage.

Migrations

Sequelize-cli is used for migrations.

npm run db:create             // create the PG DB
npm run db:migrate            // run all migrations to HEAD
npm run db:migrate:undo       // undo the last migration that was run.
npm run db:migrate:undo:all   // undo all migrations

To create a new blank migration:

./node_modules/.bin/sequelize migration:generate --name <a-meaningful-name!>

NOTE: The sequelize-cli-typescript package that we currently use to manage migrations has a bug preventing the ability to generate migrations. This can be fixed by making this change in the file ./node_modules/sequelize-cli-typescript/lib/commands/migration_generate.js, but that change will have to be undone to enable the ability to run the migrations.

Migrations are stored in /migrations. They are run at deployment time when master deploys.

DB seeders

Sequelize-cli is also used to seed the DBs.

npm run db:seed:all         // run all seeders
npm run db:seed:undo:all    // undo all seeders

PS: Entitlements aren't using this mechanism because our seeders haven't yet been ported over to Typescript.

Entitlements

Swayze needs the system roles to be persisted in the database. The entitlements seed can set this up for you and can be run via the setup-entitlements.js script like so:

npm run ts-node scripts/setup-entitlements.ts

Authorization

We perform authorization with JWTs passed in the 'Authorization' header. Example header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiNTg4N2FkYTE0MjFhYTkxMDMyMDVhMDNjIn0.SCidGbyk0iw16B0upnGBsDqqcnao8Go7Ygj33C4Zdoo

Testing

This section will explain how we are testing using Replay. Note that there are other testing approaches that are in development for LiveCore.

Here are the steps to run a Replay test:

Add prod creds to your services.manifest.json, and connect to the LiveStories VPN. Run npm run test:replay. The tests currently take ~3 mins to complete. Note that in the case that a user makes a change while the Replay tests are running, the results may return a false positive.

You can update the production creds with a replay key to map the replay file names to their auth tokens:

  "replay": {
    "no-auth-routes.csv": "",
    "user-1.csv": "<user-1 auth token>",
    "user-2.csv": "<user-2 auth token>"
  },

(Note that our routes files in user-1.csv and user-2.csv can be generated from our log files by running the script in scripts/log-to-replay-routes.py.)

Deployments

To deploy to staging, simply tag with a "staging-xxxxx" where xxxxx is a timestamps in seconds since epoch. The following shell command does the trick.

TAG="staging-$(date +%s)"; git tag $TAG; git push origin $TAG

This can be added as an alias in your .profile (or equivalent) with

alias tag-staging='TAG="staging-$(date +%s)"; git tag $TAG; git push origin $TAG'

From then on you can simply do tag-staging to push the current SHA to staging!

Endpoints

Session

  • Endpoint: /session

  • Methods:

    • POST
      • Description: Authenticate a user
      • Request: Email and password params
      • Response: JWT with user_id
        • Onboarded boolean

Team

  • Endpoint: /team/:teamId

  • Methods:

    • GET
      • Description: Fetch team by ID
      • Request:
      • Response: team document
        • Endpoint: /team/:teamId
        • Methods:
    • PATCH
      • Description: Update team details
      • Request:
      • Response: team document

  • Endpoint: /team/:teamId/membership

  • Methods:

    • GET
      • Description: Get list of team memberships
      • Request:
      • Response: Array of team memberships
    • POST
      • Description: Create team membership invite
      • Request:
      • Response: Membership

  • Endpoint: /team/:teamId/membership/:membershipId

  • Methods:

    • PATCH
      • Description: Update membership role
      • Request:
      • Response: Status code
    • DELETE
      • Description: Remove a membership
      • Request:
      • Response: Status code

  • Endpoint: /team/:teamId/dataset

  • Methods:

    • GET
      • Description: retrieve full dataset documents
      • Request:
      • Response: list of dataset documents

  • Endpoint: /team/:teamId/dataset/status

  • Methods:

    • GET
      • Description: retrieve minimal dataset description along with upload status
      • Request:
      • Response: minimal dataset object along with upload/index status

  • Endpoint: /team/:teamId/dataset/count

  • Methods:

    • GET
      • Description: retrieve the number of objects that would be returned by a call to /count
      • Request:
      • Response: numeric count of objects

  • Endpoint: /team/:teamId/story

  • Methods:

    • GET
      • Description: retrieve list of stories for given team
      • Request:
      • Response: list of story documents

  • Endpoint: /team/:teamId/story/count

  • Methods:

    • GET
      • Description: retrieve count of stories for a given team
      • Request:
      • Response: numeric count of story documents for given team

  • Endpoint: /team/:teamId/dashboard

  • Methods:

    • GET
      • Description: retrieve list of dashboards for a given team
      • Request:
      • Response: list of dashboard documents

  • Endpoint: /team/:teamId/dashboard/count

  • Methods:

    • GET
      • Description: retrieve count of dashboards for given team
      • Request:
      • Response: numeric count of dashboard documents

  • Endpoint: /team/:teamId/user

  • Methods:

    • GET
      • Description: retrieve list of users for a given team
      • Request:
      • Response: list of users

User

  • Endpoint: /user/:userId

  • Methods:

    • GET
      • Description: retrieve a full user document by ID
      • Request:
      • Response: numeric count of dashboard documents

  • Endpoint: /user/:userId/team

  • Methods:

    • GET
      • Description: retrieve the details about a users team memberships
      • Request:
      • Response: team objects and the users role within them

  • Endpoint: /user/:userId/dataset

  • Methods:

    • GET
      • Description: retrieve the datasets this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/dataset/count

  • Methods:

    • GET
      • Description: retrieve the number of datasets this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/story

  • Methods:

    • GET
      • Description: retrieve the stories this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/story/count

  • Methods:

    • GET
      • Description: retrieve the number of stories this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/dashboard

  • Methods:

    • GET
      • Description: retrieve the dashboards this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/dashboard/count

  • Methods:

    • GET
      • Description: retrieve the number of dashboards this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/iq_chart

  • Methods:

    • GET
      • Description: retrieve the IQCharts this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/iq_chart/count

  • Methods:

    • GET
      • Description: retrieve the number of IQCharts this user created or collaborated on.
      • Request:
      • Response: {entity,actions} array.

  • Endpoint: /user/:userId/notification

  • Methods:

    • GET
      • Description: retrieve a list of notifications for a given user
      • Request:
      • Response: a list of notification objects

  • Endpoint: /user/:userId/notification/count

  • Methods:

    • GET
      • Description: retrieve a full user document by ID
      • Request:
      • Response: numeric count of notifications

  • Endpoint: /user/:userId/team/:teamId/switch

  • Methods:

    • POST
      • Description: switch the 'active' team for a given user
      • Request:
      • Response:

Membership

  • Endpoint: /membership

  • Methods:

    • GET
      • Description: Use token in query param to find membership by token
      • Request:
      • Response: A membership

  • Endpoint: /membership/:membershipId

  • Methods:

    • PATCH
      • Description: Update the active attribute of a membership
      • Request:
      • Response: Status code

  • Endpoint: /membership/:membershipId/signup

  • Methods:

    • POST
      • Description: Update membership user params
      • Request:
      • Response: User

Dashboard

  • Endpoint: /dashboard/:dashboardId

  • Methods:

    • GET
      • Description: retrieve a dashboard document by ID
      • Request:
      • Response: a dashboard document

  • Endpoint: /dashboard/:dashboardId/trash

  • Methods:

    • PUT
      • Description: update the 'trashed' status of a dashboard document to trashed
      • Request:
      • Response:

  • Endpoint: /dashboard/:dashboardId/untrash

  • Methods:

    • PUT
      • Description: update the 'trashed' status to no longer be trashed
      • Request:
      • Response:

  • Endpoint: /dashboard/:dashboardId/comment

  • Methods:

    • GET
      • Description: retrieve comments on the dashboard
      • Request:
      • Response: an array of comments
    • POST
      • Description: create a comment on the dashboard
      • Request:
      • Response:

Dataset

  • Endpoint: /dataset/:datasetId

  • Methods:

    • GET
      • Description: retrieve a dataset document
      • Request:
      • Response: a dataset document

  • Endpoint: /dataset/:datasetId/trash

  • Methods:

    • PUT
      • Description: update the 'trashed' status of a dataset document to trashed
      • Request:
      • Response:

  • Endpoint: /dataset/:datasetId/untrash

  • Methods:

    • PUT
      • Description: update the 'trashed' status to no longer be trashed
      • Request:
      • Response:

  • Endpoint: /dataset/:datasetId/comment

  • Methods:

    • GET
      • Description: retrieve comments on the dataset
      • Request:
      • Response: an array of comments
    • POST
      • Description: create a comment on the dataset
      • Request:
      • Response:

IQChart

  • Endpoint: /iq_chart/:iqChartId/trash

  • Methods:

    • PUT
      • Description: update the 'trashed' status of an iqChart to trashed
      • Request:
      • Response:

  • Endpoint: /iq_chart/:iqChartId/untrash

  • Methods:

    • PUT
      • Description: update the 'trashed' status to no longer be trashed
      • Request:
      • Response:

  • Endpoint: /iq_chart/:iqChartId/comment

  • Methods:

    • GET
      • Description: retrieve comments on the iqChart
      • Request:
      • Response: an array of comments
    • POST
      • Description: create a comment on the iqChart
      • Request:
      • Response:

  • Endpoint: /iq_chart/:iqChartId/image?filename=:filename

  • Methods:

    • POST
      • Description: fetch urlbox.io image of desired chart
      • Request:
      • Response: S3-hosted image of desired chart

  • Endpoint: /iq_chart/:iqChartId/image

  • Methods:

    • GET
      • Description: fetch cached urlbox.io image of desired chart
      • Request:
      • Response: S3-hosted image of desired chart

Story

  • Endpoint: /story/:storyId

  • Methods:

    • GET
      • Description: retrieve a story document. This is a public endpoint.
      • Request:
      • Response: a story document, with private fields omitted.

  • Endpoint: /story/:storyId/trash

  • Methods:

    • PUT
      • Description: update the 'trashed' status of a story document to trashed
      • Request:
      • Response:

  • Endpoint: /story/:storyId/untrash

  • Methods:

    • PUT
      • Description: update the 'trashed' status to no longer be trashed
      • Request:
      • Response:

  • Endpoint: /story/:storyId/comment

  • Methods:

    • GET
      • Description: retrieve comments on the story
      • Request:
      • Response: an array of comments
    • POST
      • Description: create a comment on the story
      • Request:
      • Response:

Location

  • Endpoint: /location/user/:userId/team/:teamId

  • Methods:

    • GET
      • Description: Retrieve the locations for the provided user, within that user's team
      • Request:
      • Response: A collection of locations (if any) that were saved by the user

  • Endpoint: /location/user/:userId/team/:teamId

  • Methods:

    • PUT
      • Description: upsert a location for the provided user, within that user's team
      • Request:
      • Response:

Notification

  • Endpoint: /notification/:notificationId/read

  • Methods:

    • POST
      • Description: Mark notification as read
      • Request:
      • Response: Status code

  • Endpoint: /notification/:notificationId/read

  • Methods:

    • DELETE
      • Description: Mark notification as unread
      • Request:
      • Response: Status code

  • Endpoint: /notification/trash

  • Methods:

    • PUT
      • Description: Mark notification as unread
      • Request: notification_ids: List of ids to trash
      • Response: Status code

Chart

  • Endpoint: /charts/:chartId/image?filename=:filename

  • Methods:

    • POST
      • Description: fetch urlbox.io image of desired chart
      • Request:
      • Response: S3-hosted image of desired chart

  • Endpoint: /charts/:chartId/image

  • Methods:

    • GET
      • Description: fetch cached urlbox.io image of desired chart
      • Request:
      • Response: S3-hosted image of desired chart
@sendgrid/mail@sportsbet/bunyan-syslog@types/commander@types/node@types/pg-pool@types/qs@types/request-promise-native@types/restify@types/restify-errors@types/sequelize@types/superagentajvaws-sdkbluebirdbunyanclustercommandercsvelasticsearchfollow-redirectshttp-aws-esjsonwebtokenjuicemongojspgpugpusherqsredisrequestrequest-promise-nativerestifyrestify-cors-middlewarerestify-errorsrestify-jwtsequelizesequelize-cli-typescriptsuperagentswayzets-nodetypescripturijsurlboxuuidvalidator
1.0.0

5 years ago