@xlnt/gnarly NPM

🤙 Gnarly

Gnarly’s reduces blockchain events into a steady state with confidence.
And that’s fuckin’ gnarly.

-----> Read the Medium post for more details <-----

Developer Install / Usage

clone this repo

npm install
npm run test
npm run watch-ts

To use it in a project, implement the following components and then put them all together:

stateReference — the state that you want to manage, as a mobx-state-tree
storeInterface — implement the interface to store gnarly's internal state
ITypeStore — implement the interface to store the actual info you want (add, update, delete)
onBlock — implement the state reduction function that gnarly uses to process events

Note that if you npm link gnarly, because mobx requires a single mobx instance to function, you need to symlink your project's mobx and mobx-state-tree to gnarly's with something like

npm link gnarly
ln -sF ./gnarly/node_modules/mobx `pwd`/node_modules/mobx
ln -sF ./gnarly/node_modules/mobx-state-tree `pwd`/node_modules/mobx-state-tree

which enforces that they reference the same module.

Now let's write some typescript. See a full example of a kitty tracker in ./examples/kitty/index.ts

// first, let's register the cryptokitties transfer abi with gnarly
const CRYPTO_KITTIES = '0x06012c8cf97BEaD5deAe237070F9587f8E7A266d'
addABI(CRYPTO_KITTIES, [{
    anonymous: false,
    inputs: [
      { indexed: false, name: 'from', type: 'address' },
      { indexed: false, name: 'to', type: 'address' },
      { indexed: false, name: 'tokenId', type: 'uint256' },
    ],
    name: 'Transfer',
    type: 'event',
  }],
)

// next let's configure the kitty table in our store
// this is just normal sequelize stuff
const Kitty = sequelize.define('kitty', {
  id: { type: Sequelize.INTEGER, primaryKey: true, autoIncrement: true },
  txId: { type: Sequelize.STRING },
  patchId: { type: Sequelize.STRING },

  kittyId: { type: Sequelize.STRING },
  owner: { type: Sequelize.STRING },
}, {
    indexes: [
      { fields: ['kittyId'] },
      { fields: ['owner'] },
      { fields: ['txId'] },
    ],
  })

// the TypeStore tells gnarly how to update the store for each type
// must be (relatively) atomic and not swallow errors
const MyTypeStore = {
  kittyTracker: {
    ownerOf: async (txId: string, patch: any) => {
      switch (patch.op) {
        case 'add': {
          await Kitty.create({
            txId,
            patchId: patch.id,
            kittyId: patch.key,
            owner: patch.value,
          })
          break
        }
        case 'replace': {
          await Kitty.update({
            txId,
            patchId: patch.id,
            kittyId: patch.key,
            owner: patch.value,
          }, {
              where: { kittyId: patch.key },
            },
          )
          break
        }
        case 'remove': {
          await Kitty.destroy({
            where: { kittyId: patch.key },
          })
          break
        }
        default: {
          throw new Error('wut')
        }
      }
    },
  },
}

// build the mobx-state-tree store
// this describes a state that contains a map of owner address to token id
// along with a single action, `setOwner` that sets a key/value
const KittyTracker = types
  .model('KittyTracker', {
    ownerOf: types.optional(types.map(types.string), {}),
  })
  .actions((self) => ({
    setOwner (tokenId, to) {
      self.ownerOf.set(tokenId, to)
    },
  }))

const Store = types.model('Store', {
  // ... any other stores you want in here ...
  kittyTracker: types.optional(KittyTracker, {}),
})

// now create a reference to an instance of this state
const stateReference = Store.create({
  kittyTracker: KittyTracker.create(),
})

// implement the state reduction function
// simply modify your state in reaction to a new block
// gnarly handles everything else behind the scenes!
const onBlock = async (block) => {
  // 1. for ever transaction in this block (async)...
  forEach(block.transactions, async (tx) => {
    // 2. if it's a direct transaction to the cryptokitties contract...
    if (addressesEqual(tx.to, CRYPTO_KITTIES)) {
      // 3. Load the rest of the transaction info (logs, internal transactions)
      // NOTE: to get the full tx, you need a parity archive+tracing node
      await tx.getReceipt()

      // 4. for each log (sync)
      tx.logs.forEach((log) => {
        // 5. if the event is Transfer
        if (log.event === 'Transfer') {
          const { to, tokenId } = log.args

          // 6. Give gnarly some context for this change using `because()`
          // (this context is (will be) provided to the ui as part of the event log)
          because('KITTY_TRANSFER', {}, () => {
            // 7. finally, update the state using your action
            stateReference.kittyTracker.setOwner(tokenId, to)
          })
        }
      })
    }
  })
}

// finally, put it all together
const gnarly = new Gnarly(
  stateReference,
  storeInterface,
  nodeEndpoint,
  MyTypeStore,
  onBlock,
)

const main = async () => {
  // reset gnarly's internal state
  await storeInterface.setup()
  // reset our app's derived state
  await Counter.sync({ force: true })
  await Kitty.sync({ force: true })
  // start gnarly
  await gnarly.shaka()
}

See a full example of a kitty tracker in ./examples/kitty/index.ts

TODO

We'd love your help with any of this stuff

literally just testing the code we've written at all, manually
- does it work
automated testing with mocha/chai/etc
- ourbit unit tests, with a stubbed store
- ourbit integration tests against sqlite (optional)
- blockstream with stubbed getters calls ourbit correctly
- test that persistStateWithStore works corectly
- test utils file
- gnarly itself works (integration test)
update README with example code
any sort of overall architecture improvements
replace block reconciliation polling with a web3 filter
replace getTransactions with a generator that can page through results
what should the developer-friendly cli/binary look like? config ala redis? opinions wanted!

Features

"Instant" updates with confidence intervals
- optimistic UI pattern; apply expected changes immediately but revert to source of truth as soon as it's known
reduces blockchain events into a steady state
- optimize client queries and data architecture
- compatible with the rest of the world of technology
graceful reorg and incorrect optimistic state handling
friendly error management
- developers get reasonable error contexts
- consumers get explanations about errors
- allows anyone to know (i) that something occurred and (ii) why it occurred
supports replay from arbitrary blocks to (i) bootstrap the steady state and (ii) resume after failures
default output is catered towards a graphql consuming client

Solving Severe Asynchronicity

“Severe asynchronicity” is the UX experience of using a first-layer blockchain today:

transactions publish within a reasonable timeframe (ms) but at very low confidence—it’s hard to know if and when they will succeed
transactions are finalized within an unreasonable timeframe (minutes/hours) but with very high confidence
Off-chain state is uncertain due to 1, 2, block re-orgs, short-lived forks, uncles, etc,
Off-chain software isn’t perfect; it can lag behind the blockchain (if waiting for confirmation blocks), fail to replay state updates in the event of reorgs/forks, improperly handle unconfirmed transactions, and much, much more.

Key Ideas

Gnarly uses the ideas behind redux and MobX to convert imperative blockchain events to declarative, reactive state.

Technology

Any/all of this can change, but here are the technologies currently used. Note that gnarly should be able to be used in both a browser and server-side environment.

Typescript
MobX
ethereumjs-blockstream

how did we get an 'add' patch for a key that hadn't been set yet?
- yeah, we're definitely getting 'add' patches for things when they don't actually exist yet
- yeah seriously, what's up with that? I doubt we're dropping connections, the uncaught exception handler would have exited
- is this annoying and weird and means we can't replay from failure yet
why doesn't an optional array inside of an optional map work?
- if setting default, do we get patches for that?
- if not, why the hell not?