@aptoma/amp-cli NPM

AMP CLI

Command line interface for AMP AWS tasks.

Install

npm install -g @aptoma/amp-cli

You also need to have the AWS-CLI set up and configured. Follow the instructions here.

Configuration

The configuration is read from amp.json in your working directory.

Subcommands

Lambda

The lambda commands for updating code only support S3 as a source not using local files. It's currently intended to be used with projects that are built and uploaded to S3 from travis.

There's also a run command to invoke the function locally.

Usage

Usage: amp lambda [options] [command]


Commands:

  create <commit>              create a new lambda function
  update <commit>              Update existing lambda function
  list-aliases [version]       List aliases for lambda function
  list-versions                List versions for lambda function
  alias <aliasName> <version>  Create / Update alias for lambda function
  run [options]                Run the Lambda function locally

Options:

  -h, --help  output usage information

Configuration

{
  "lambda": {
    "name": "deployment-test",
    "description": "test",
    "region": "eu-west-1",
    "role": "arn:aws:iam::xxxxxx:role/lambda_basic_execution",
    "s3": {
      "bucket": "my-release-bucket",
      "path": "myproject"
    }
  }
}

Cloudformation Stacks

Usage

Usage: amp stack [options] [command]


Commands:

create [options] <configName> <stackName> <environment>                    create a new cloudformation stack
create-change-set <configName> <stackName> <environment> <changeSetName>   create a change set for an existing stack
execute-change-set <configName> <stackName> <environment> <changeSetName>  create a change set for an existing stack
update <configName> <stackName> <environment>                              update existing cloudformation stack
replace [options] <configName> <environment>                               Replace cloudformation stack and associated resources
describe <configName> <stackName> <environment>                            describe existing cloudformation stack
attach-to-elb [options] <stackName> <elbName>                              attach ASG in a stack to ELB
detach-from-elb [options] <stackName> <elbName>                            detach ASG in a stack from ELB

Options:

  -h, --help  output usage information

Configuration

{
  "stack": {
    "ec2": {
      "template": "https://s3.eu-central-1.amazonaws.com/cloudformation-template.json",
      "department": "myDepartment",
      "service": "myServiceName",
      "development": { /* environment name */
        "region": "eu-central-1",
        "parameters": { /* params that exist in the template with same names */
          "Environment": "development",
          "IAMServiceRole": "myRole",
          "InstanceType": "t2.micro",
          "MaxInstances": "8",
          "MinInstances": "2"
        },
        "tags": {
          "TagName": "TagValue" /* custom tags */
        }
      }
    }
  }
}

Example command line for creating stack with above config

amp stack create ec2 my-stack-name development

Loading template from file

"template": "file://./mytemplates/my-template.json"

Template that requires capability IAM

{
  "stack": {
    "ec2": {
      "capabilities": ["CAPABILITY_IAM"]
    }
  }
}

Code Deploy

Usage

Usage: amp deploy [options] [command]


Commands:

create-deployment <deploymentGroup> [tag|commit|branch]         deploy a tag, commit or branch to a deployment group
show-deployment <deploymentId>                                  show the result of a deployment

Options:

  -h, --help  output usage information

Configuration

{
  "deploy": {
    "name": "myApplicationName",
    "region": "eu-central-1",
    "s3": {
      "bucket": "my-release-bucket",
      "path": "myproject"
    }
  }
}

Using multiple configuration for e.g different regions use the -c <configuration name> option.

{
  "deploy": {
    "myConfiguration1": {
      "name": "myApplicationName",
      "region": "eu-central-1",
      "s3": {
        "bucket": "my-release-bucket",
        "path": "myproject"
      }
    },
    "myConfiguration2": {
      "name": "myApplicationName",
      "region": "eu-west-1",
      "s3": {
        "bucket": "my-other-bucket",
        "path": "myproject"
      }
    }
  }
}

Logs

Usage: amp log [options] [command]


Commands:

  get [options] [configName]  Get files from S3

Options:

  -h, --help  output usage information

Get Examples

Config

{
  "log": {
    "cloudfront": "s3://bucket/cloudfront-logs/E234234234."
  }
}

Stream to stdout

amp log get cloudfront -s

Piping will autmatically stream to stdout

amp log get clodfront | more

Stream to stdout without config

amp log get -s -u s3://bucket/test/path

Download all logs to current working dir

amp log get cloudfront

Download new logfiles for today (this appends the current date to your path)

amp log get cloudfront -d

Download logs for a specific month

amp log get cloudfront -d 2016-04

Piping cloudfront logs to goaccess

amp log get cloudfront | goaccess --time-format='%H:%M:%S' --date-format='%Y-%m-%d' --log-format='%d\t%t\t%^\t%b\t%h\t%m\t%^\t%r\t%s\t%R\t%u\t%^'

AMI

Usage: amp ami [options] [command]


Commands:

list [options] <playbook>  List all AMIs from a given playbook in a given region
delete [options] <AMI-ID>  Remove an AMI and its snapshot

Options:

-h, --help  output usage information

List Examples

List defaults to returning images from eu-central-1. This can be overriden with the -r|--region parameter. A list of all outdated images can be generated with the -o|--outdated option.

List all images based on the ansible-nodejs-playbook.

amp ami list nodejs

List all Node.js images in Dublin.

amp ami list nodejs -r eu-west-1

List all outdated core images.

amp ami list core --outdated

Delete Examples

Delete an image.

amp ami delete ami-1783de64

Delete an image in Dublin.

amp ami delete ami-1783de64 -r eu-west-1

SSH

  Usage: amp ssh [options] [command]


  Options:

    -h, --help  output usage information


  Commands:

    deployment [options] <deployment_group>  SSH into an instance in the given deployment group

Deployment Examples

SSH into one of the machines in a given deployment group.

$ amp ssh deployment production

Add -p to print the DNS name of one of the machines instead of logging in.

amp ssh deployment production -p

Docker

  Usage: amp docker [options] [command]

  Options:

    -h, --help                                output usage information

  Commands:

    bootstrap                                 Bootstrap a new application or deployment
    list                                      List all deployments along with their environments and versions
    build [options]                           Build, tag and upload a docker image
    deploy <stack|alias> [tag|branch|commit]  Deploy a new image to a running service
    create <deployment>                       Create resources for a new deployment
    update <deployment>                       Deploy configuration changes to a deployment
    teardown <deployment>                     Tear down a deployment, removing all resources
    alias <deployment>                        Create an alias record for a deployment
    swap <deployment> <deployment>            Swap alias records between two deployments
    secrets <deployment>                      Edit secrets for a deployment

Bootstrap

If you have an application with a Dockerfile ready to go, you can bootstrap a deployment via amp docker bootstrap. Follow the instructions as given.

Configuration

{
  "docker": {
    "app": "lp-publish-proxy",
    "department": "drmobile",
    "region": "eu-central-1",
    "environments": {
      "dev": {
        "cluster": "dev",
        "resources": {
          "template": "file://resources.yml",
          "parameters": {
            "Environment": "development"
          }
        }
      }
    },
    "deployments": {
      "dev": {
        "environment": "dev",
        "http": {
          "port": 8080,
          "healthCheckPath": "/status",
        },
        "minCount": 2,
        "maxCount": 8,
        "cpu": 128,
        "memory": 128,
        "variables": {
          "ENVVAR": "foobar"
        }
      }
    }
  }
}

The minimum configuration for an app is the cluster name. The other values you see here are the defaults, and can be skipped unless they need to be overridden. The http-key may be set to null for worker nodes that don't need a load balancer.

Environments vs deployments

Simply put, a deployment is what you could consider to be "a service", i.e. something that runs on one or more hosts with a hostname, SSL and a load balancer in front. An environment defines shared resources that can be reused by one or more deployments.

CPU/Memory reservations

Your service runs with a certain resource reservation, which defaults to 128/128. For memory, this is simply the number of MB reserved for each container. For CPU, each core on the host machine is equivalent to 1024 CPU "units". Note that these are reservations, not hard caps. Your containers may be allowed to use more if the container instance has resources to spare. App developers are encouraged to be good neighbours and not reserve excessive capacity.

Autoscaling

Additional containers will be added to your service if its CPU usage exceeds 80% of its reservation. The host fleet itself will also autoscale to ensure that additional capacity is available when needed.

Environment variables

Each environment in the configuration can optionally define its own environment variables. These will be passed to the app on startup.

{
  "docker": {
    "deployments": {
      "dev": {
        "variables": {
          "NODE_ENV": "development"
        }
      }
    }
  }
}

Secrets

To edit the secrets for the dev deployment:

amp docker secrets dev

The AMP-CLI will open your default editor with an empty JSON object. Keys and values on this object will be encrypted and saved in AWS, and exposed as envvars on your containers.

Custom resources

Each environment can optionally define custom CloudFormation resources. These will be merged with the generic resource template when the environment is created.

{
  "docker": {
    "environments": {
      "dev": {
        "cluster": "dev",
        "resources": {
          "template": "file://resources.yml",
          "parameters": {
            "EncryptionKey": "foobar"
          }
        }
      }
    }
  }
}

If you create resources which the service needs special permission to access, you'll need to append policies to the task role. You can do this by defining a partial IAM Role named TaskRole with your additional policies.

Resources:

  TaskRole:
    Properties:
      Policies:
        - PolicyName: DynamoDBReader
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:GetItem
                  - dynamodb:PutItem
                Resource:
                  - arn:aws:dynamodb:eu-central-1:772125630481:table/mytable

For nearly all other cases, you can and should treat your own resource.yml as a standalone template.

Logs

You should configure your application to write logs to stdout and stderr. These end up in the syslog on the worker machines and get exported to Papertrail. There should be a group for each cluster where you can set up saved searches for your application, such as program:messages <your-app-name>.

Disk usage

There is an NFS mounted on /data in all containers. All containers for a given deployment share the same underlying storage. If you need a cache dir or just some scratch space, you should use this directory as the underlying storage is limitless. Cleanup-routines will periodically delete files to free up space, so it should not be used for permanent storage. The docker hosts are configured to use async writes when dealing with the NFS mounts, so writes may be dropped in the case of errors.