@eahefnawy/lego NPM

A core-agnostic implementation of Serverless Components (codenamed LEGO). The "core" in this repo is just a cli that runs the method/command you specified, from a serverless.js and pass the options as inputs.

The goal of this repo is to quickly innovate on the concept of components and abstractions, and design the core along the way as the needs arise: fat components first, thin core second.

Installation
Usage
- Run a serverless.js in cwd
- Run a registry component
Components
Learnings

Installation

npm i -g @eahefnawy/lego

Usage

Run a serverless.js in cwd

const deploy = async (inputs) => {
  console.log('deploy running...')
  return {}
}

const remove = async (inputs) => {
  console.log('remove running...')
  return {}
}

const hello = async (inputs) => {
  console.log(`hello ${inputs.name}...`)
  return {}
}

module.exports = { deploy, remove, hello }

lego deploy
lego remove
lego hello --name world

Run a registry component:

lego <OPERATION> <COMPONENT> {...inputs}

# example
lego deploy lambda --name hello --memory 512

Note: Check out the components & examples folder for real world working examples, as well as the poc folder for some abstraction ideas.

Components

Role

Inputs

name (string) - default: lego
service (string) - default: lambda.amazonaws.com

Outputs

name (string)
service (string)
arn (string)

example (master)$ lego deploy role

   Status:  Role Deployed
   
   Name:    lego
   Service: lambda.amazonaws.com
   ARN:     arn:aws:iam::552750238291:role/lego

example (master)$

Lambda

Inputs

name (string) - default: lego
description (string) - default: lego Lambda Component
handler (string) - default: handler.hello
runtime (string) - default: nodejs8.10
shim (string) - default: null
code (string) - default: cwd
memory (number) - default: 128
timeout (number) - default: 10
env (object) - default: {}

Outputs

name (string)
description (string)
handler (string)
runtime (string)
shim (string)
code (string)
memory (number)
timeout (number)
env (object)
arn (string)
role (object) - role outputs

example (master)$ lego deploy lambda

   Status:  Lambda Deployed
   
   Name:     lego
   Memory:   128
   Timeout:  10
   Runtime:  nodejs8.10
   Handler:  handler.hello
   ARN:      arn:aws:lambda:us-east-1:552750238299:function:lego

example (master)$

Website

Inputs

name (string) - default: lego
code (string) - default: cwd
assets (string) - default: .
envFileLocation (string) - default: ./src/env.js
env (object) - default: {}
buildCmd (string) - default: null

Outputs

name (string)
url (string)

example (master)$ lego deploy website

   Status:  Website Deployed
   URL:     lego-ebd3.s3-website-us-east-1.amazonaws.com

example (master)$

Table

Inputs

name (string) - default: lego
key (string) - default: id

Outputs

name (string)
key (string)

example (master)$ lego deploy table

   Status:  Table Deployed
   
   Name:    lego
   Key:     id

example (master)$

WebSockets

Inputs

name (string) - default: lego
description (string) - default: lego websockets
stage (string) - default: dev
routeSelectionExpression (string) - default: $request.body.action
routes (object) - key:value map of route:lambdaArn

Outputs

name (string)
description (string)
stage (string)
routeSelectionExpression (string)
routes (object)
id (string)
url (string)

example (master)$ lego deploy websockets

   Status:  WebSockets Deployed
   
   Name:        lego
   ID:          axvfuc9ql0
   Stage:       dev
   Expression:  $request.body.action
   URL:         wss://axvfuc9ql0.execute-api.us-east-1.amazonaws.com/dev/
   Routes: 
     - $connect
     - $disconnect
     - $default

example (master)$

Socket

Inputs

name (string) - default: lego
description (string) - default: lego websockets
stage (string) - default: dev
code (string) - default: cwd
memory (number) - default: 128
timeout (number) - default: 10
env (object) - default: {}

Outputs

lambda (object) - lambda outputs
websockets (object) - websockets outputs

example (master)$ lego deploy socket

   Status:  Socket Deployed
   URL:     wss://axvfuc9ql0.execute-api.us-east-1.amazonaws.com/dev/

example (master)$

Connect Command

The Socket component has a custom connect command that connects you to a local or deployed socket.

Examples

// socket.js in cwd
// if you deployed you'll be connected
// to the remote websockets server
// if not, you'll be connected to a local server
lego connect socket

// or you can force a local connection
lego connect socket --local

// By default, you're connected to the default route
// but you can specify a specific route to connect to
lego connect --route message

// if your socket.js file in another directory
lego connect socket --code ./backend

Realtime App

Inputs

name (string) - default: realtimeApp
description (string) - default: Realtime App
stage (string) - default: dev
frontend (object) - website inputs
backend (object) - socket inputs

Outputs

website (object) - website outputs
socket (object) - socket outputs

realtimeApp (master)$ node ../../bin/lego deploy
   
   Status:  Realtime App Deployed
   
   Socket URL:  wss://39jpalv9u5.execute-api.us-east-1.amazonaws.com/dev/
   Website URL: realtimeapp-dev-1hmjmr.s3-website-us-east-1.amazonaws.com
   
realtimeApp (master)$

Connect Command

The Realtime app component also exposes the Socket component connect command. So you can connect exactly the same way.

Learnings

serverless.yml is the bottleneck

Declaring composable components in a serverless.yml file is the source of all the complexity we have in v2. To accomplish this we face two very complicated and challenging issues:

Varaible Support: supporting variables feels like writing a programming language from scratch. We invested a lot in it in v1, and it still has hard to solve bugs. Variable support in v2 is even more challenging since it's used for component composition.
Components Dependency: to compose components together, some components would depend on others. So we need to somehow figure out the correct order of deployment, and a graph or a tree is usually required, causing a lot of complexity.

Programatic support first

As soon as we ignore the serverless.yml requirement and start to think about a serverless.js file, the majority of the core is shrinked. Giving power to the developer to do anything, without any new concepts or learnings! So instead of perfecting a serverless.yml implemnetation, we should instead provide them the tools/functions to easily and simply write deployment scripts. You can think of it as gulp for serverless infrastructure.

A component is just an npm package that contains deployment logic. Instead of enforcing a certain component core api, we should encourage best practices. For example, a component typically (but not required) has a deploy and remove functions. When we think of components as just npm packages, we can make full use of node.js and npm, just like plugins do in v1, helping us to iterate quickly and build a bigger community faster.

The real value is in the registry

The more components we have in the registry, the more valuable the entire project is, and the easier it becomes to write more components. With enough low level components, eventually we could reach a point where you don't need to provide any inputs for high level components. For example lego deploy users would deploy the users component by deploying the apig, lambda code/config and dynamoDb schema required, and logs the http endpoints you need.

With that in mind, it becomes clear that the registry and the community is the framework!

Component consumer = component producer

Supporting serverless.js instead of serverless.yml breaks the barrier between a component consumer and a component producer, and instead we have a single component user. The experience for using a component, and developing a component would be exactly the same. We just need to simplify it as much as possible. Publishing a component is as simple as publishing an npm package.

Each user defines his own cli experience

Since each component is just an npm package, they should not log anything when used by another component. Instead, each component should return its outputs along with its children's outputs up to the top level component. The top level component then specifies which of all those outputs to log.

In other words, our great uniform cli experience is that you can easily define your own customized, clean and use case focused cli experience.

Beyond JS

Ignoring serverless.yml limits us to only the JS commnuity. However, if components really simplify deploying serverless infrastructure, and since components are just npm packages, then we should be able to easily use them to deploy a deployment engine that is http exposed. We can then write simple http client in each language we wanna support. So we'd have a lambda that deploys a lambda :), that is if supporting multiple languages is a string requirement.