@toa.io/core

Toa

Runtime for low code eventually consistent distributed systems.

Status

The project is under heavy development, while being used in production by its authors. Non-scattered documentation is coming some time later.

Features

Runtime

• Interservice communications
  • Reliable RPC and Events (ComQ)
  • Flow control
  • Eventual consistency guarantee (not yet)
  • Uniform interface
    • Input/query segregation
    • Successful rejections
    • Distributed exception handling
  • Transparent service discovery
  • Multi-protocol transmission
    • built-in support for AMQP and HTTP
  • In-memory communications
  • Messages validation
• Persistent state management
  • Concurrency control
  • Data validation
  • Multi-document transactions
  • Transactional outbox: state and its events commit together
  • Invariants (guards)
  • Built-in implementations
    • MongoDB
    • SQL (Knex)
• Cadence: calls that happen on their own time
  • a pulse calls a component's own operation on a cadence, with no schedule stored anywhere
  • a delay hands one call over to be made later, and answers the id that cancels it
• Configuration with secrets
• API Gateway
  • Resource discovery
  • Identity
    • Basis authentication
    • Identity federation (OIDC)
      • id_token authentication
      • Authorization code flow
    • Passkeys (WebAuthn)
    • OTP authentication
  • Access control
  • Cache control
  • File uploads, downloads, and processing.
  • Throttling: a distributed GCRA, with no per-request I/O
  • OAuth 2.1 authorization server: discovery, dynamic client registration, PKCE
  • JSON-RPC: every method the tree exposes, called as a procedure named after its route
  • Model Context Protocol: a method is a tool where it says so, and a model calls it as any other caller does
• Atomicity: what the replicas of a component decide together
  • Partitioning: an exclusive claim on one of a fixed number of slots
  • Distributed lock manager
  • Shared rate metering
• Telemetry
  • Structured logs
  • Distributed tracing
• Introspection: product topology collection and visualization
• Realtime events
• BLOB storage
  • MIME type detection and validation
  • Providers: file system, Amazon S3, Cloudinary
• Transient state
• Built-in Fetch with retries and telemetry

Development

• Language interoperability
  • Node.js
  • Bash :)
• Service prototyping (inheritance)
  • Generic prototype
• Data and operations schemas
• Declarative API Gateway endpoints with authorization policies
• Integration tests SDK
• Runtime bootloader API
• Development environment configuration
• CLI
  • toa mono — composition and extension services in one process
• Extensibility:
  • Persistent storages
  • Communication protocols
  • Runtime core abstractions
  • Arbitrary services

Operations

• Standardized containerization
• Compositions: the components, and the extension services, deployed as one pod
• Resource management
• Environment variables
• Standalone infrastructure configuration
• One-command fully automated deployment
  • Preset multi-arch docker images
  • Helm deployment to kubernetes
  • Single-image (toa deploy --mono) deployment
• CLI utilities
  • Secrets management
  • Remote container shell