claycli v4.0.1

claycli

A CLI For Clay!

Installation

npm install -g claycli

Usage

clay <command> [options]

If installed globally, call clay from the command line. Much like git, claycli is configured using a dotfile (.clayconfig) in your home folder. In it you may specify references to api keys and urls / site prefixes that you use frequently. For urls and site prefixes, it will assume http:// and port 80 unless you specify otherwise.

Note that a site prefix is everything before the api route, e.g. http://domain.com/site1 in http://domain.com/site1/_components/article.

[keys]
  local = ha8yds9a8shdf98asdf
  qa = 8quwqwer09ewr0w9uer
  prod = bj34b6345k634jnk63n4
[urls]
  local-site1 = https://localhost.site1.com:3001
  local-site2 = site2.com/site-2 # http and port 80

For smaller Clay installations (or, ironically, for very large teams where devs spend most of their time on individual sites), you may specify a default api key and url / site prefix by using the CLAYCLI_DEFAULT_KEY and CLAYCLI_DEFAULT_URL environment variables.

Commands

• config
• lint
• import
• export

Common Arguments

claycli uses some common arguments across many commands.

• -v, --version will print the claycli version and exit
• -h, --help will print helpful info about claycli and exit
• -c, --concurrency allows setting the concurrency of api calls (defaults to 10)
• -k, --key allows specifying an api key or alias

Handling Files

Dispatch

Many claycli arguments allow you to pipe in the contents of files to stdin or pipe data out from stdout. The format that claycli uses to represent data (similar to a database dump) is called a dispatch, and it consists of newline-separated JSON without site prefixes.

{"/_components/article/instances/123":{"title":"My Article","content":[{"_ref":"/_components/paragraph/instances/234","text":"Four score and seven years ago..."}]}}
{"/_components/meta-title/instances/345":{"title":"My Article","ogTitle":"My Longer Titled Article","twitterTitle":"Article"}}

Each line of a dispatch contains composed data for a component (or page, user, list, etc), including any data for its child components. This means that each line is able to be sent as a cascading PUT to the Clay server, which is a highly efficient way of importing large amounts of data. Note that a dispatch is not meant to be human-readable, and manually editing it is a very easy way to introduce data errors.

A dispatch may be piped into or out of commands such as clay import and clay export. Because dispatches are a special format (rather than regular JSON files), the convention is to use the .clay extension, but this isn't required.

clay export domain.com > article_dump.clay
clay import domain.com < article_dump.clay
clay export domain.com | clay import localhost

Bootstrap

For working with human-readable data, we use a format called a bootstrap. These are human-readable yaml files that divide components (and pages, users, lists, etc) by type. This is the same format that is used by the bootstrap.yml files in your Clay install.

_components:
  article:
    instances:
      123:
        title: My Article
        content:
          - _ref: /_components/paragraph/instances/234
  paragraph:
    instances:
      234:
        text: Four score and seven years ago...
  meta-title:
    instances:
      345:
        title: My Article
        ogTitle: My Longer Titled Article
        twitterTitle: Article

A bootstrap may be piped into and out of any claycli commands that accept dispatches. To tell claycli that you're dealing with bootstraps, please use the --yaml argument.

clay export --yaml domain.com > article_dump.yml
clay import --yaml domain.com < article_dump.yml

If you're a backend developer or database architect, it may be helpful to think of dispatches and bootstraps as denormalized and normalized data. You'll notice that the two examples above contain the same data. The denormalized dispatches allow a single API call per line and use less memory because they're streamable, while the normalized bootstraps are better for hand-coding data because components are not duplicated if referenced multiple times. Generally speaking, use dispatches for transporting and storing data and bootstraps for hand-coding.

Config

clay config --key <alias> [value]
clay config --url <alias> [value]

Show or set configuration options. These are saved to ~/.clayconfig. As specified above, sites will assume http and port 80 if you do not write the protocol and port.

Arguments

• -k, --key allows viewing or saving an api key
• -u, --url allows viewing or saving a url / site prefix

Examples

clay config # view all configuration options
clay config --key local # view 'local' api key
clay config --key local ab27s9d # set 'local' api key
clay config --url qa # view 'qa' site prefix
clay config --url qa https://qa.domain.com:3001 # set 'qa' site prefix
clay config --url my-cool-article domain.com/_components/article/instances/123 # set a specific url

Lint

clay lint [--concurrency <number>] [url]

Verify Clay data against standardized conventions and make sure all child components exist.

Linting a page, component, or user url will verify that the data for that url exists, and (for pages and components) will (recursively) verify that all references to child components exist. The url may be a raw url, an alias specified via clay config, or may be omitted in favor of CLAYCLI_DEFAULT_URL.

Instead of specifying a url, you may pipe in a component's schema.yml to lint. It will go through the schema and verify that it conforms to Kiln's schema rules.

Arguments

• -c, --concurrency allows setting the concurrency of api calls

Examples

clay lint domain.com/_pages/123 # lint all components on a page
clay lint domain.com/2018/02/some-slug.html # lint a page via public url
clay lint my-cool-article # lint a component specified via config alias
clay lint < components/article/schema.yml # lint single schema

Import

clay import [--key <api key>] [--concurrency <number>] [--publish] [--yaml] [site prefix]

Imports data into Clay from stdin. Data may be in dispatch or bootstrap format. Site prefix may be a raw url, an alias specified via clay config, or may be omitted in favor of CLAYCLI_DEFAULT_URL. Key may be an alias specified via clay config, or may be omitted in favor of CLAYCLI_DEFAULT_KEY.

The publish argument will trigger a publish of the pages and / or components you're importing. Note that the generated url of an imported page might be different than its original url, depending on your Clay url generation / publishing logic.

Arguments

• -k, --key allows specifying an api key or alias
• -c, --concurrency allows setting the concurrency of api calls
• -p, --publish triggers publishing of imported pages
• -y, --yaml specifies that input is bootstrap format

Examples

clay import --key local localhost:3001 < db_dump.clay # import a dispatch
clay import --key qa --publish --yaml < bootstrap.yml # import and publish pages in a bootstrap
wordpress-export domain.com/blog | clay import --key local localhost.domain.com # pipe from 3rd party exporter
clay export --key prod domain.com/_components/article/instances/123 | clay import --key local localhost.domain.com # pipe from clay exporter
cat *.clay | clay import --key local localhost:3001 # import multiple dispatches
tail -n +1 *.yml | clay import --key local --yaml localhost:3001 # import multiple bootstraps (tail adds a delimiter)

Export

clay export [--key <api key>] [--concurrency <number>] [--size <number>] [--layout] [--yaml] [url]

Exports data from Clay to stdout. Data may be in dispatch or bootstrap format. The url may be a raw url, an alias specified via clay config, or may be omitted in favor of CLAYCLI_DEFAULT_URL.

If the url does not point to a specific type of data (a page, public url, component, user, list, etc), claycli will query the built-in pages index to pull the latest 10 pages from the site. When querying the pages index, you must either specify a key or have the CLAYCLI_DEFAULT_KEY set. The api key is only required when exporting multiple pages (by querying the pages index).

Instead of fetching the latest pages, you may pipe in a yaml-formatted elasticsearch query. Use this to set custom offsets (for batching and chunking exports), export non-page content from other indices, or filter exported data via certain properties. Note that if you pipe in a query that includes size, it will take precedence over the CLI size argument.

index: published-products
size: 5
from: 10
sort:
  - price
body:
  query:
    match_all: {}

By default, layouts are not exported when exporting pages. This allows you to easily copy individual pages between sites and environments. To trigger layout exporting, please use the layout argument.

Arguments

• -k, --key allows specifying an api key or alias
• -c, --concurrency allows setting the concurrency of api calls
• -s, --size specifies the number of pages to export (defaults to 10)
• -l, --layout triggers exporting of layouts
• -y, --yaml specifies that output is bootstrap format

Examples

clay export domain.com/_components/article/instances/123 > article_dump.clay # export individual component
clay export --yaml domain.com/_pages/123 > page_bootstrap.yml # export individual page
clay export --layout --yaml domain.com/_pages/123 > page_bootstrap.yml # export page with layout
clay export domain.com/_pages/123 | clay import --key local local.domain.com # copy page to local environment
clay export --key prod --size 1 domain.com > recent_page.clay # export latest updated page
cat query.yml | clay export > db_dump.clay # export custom query to dispatch
clay export --yaml < query.yml > pages.yml # export custom query to bootstrap

# other things you may export

clay export domain.com/_users/abs8a7s8d --yaml > my_user.yml # export single user
clay export domain.com/_users --yaml > users.yml # export all users
clay export domain.com/_lists/tags > tags.clay # export single list
clay export domain.com/_lists > lists.clay # export all lists
clay export domain.com/2017/02/some-slug.html # export published page via public url
clay export domnain.com/_lists/new-pages # export built-in 'New Page Templates' list (page uris will be unprefixed)

Contributing

Pull requests and stars are always welcome. For bugs and feature requests, please create an issue.

This project is released under the MIT license.