firebolt-sdk v1.3.2

Firebolt Node.js SDK

Installation

This library is published in the NPM registry and can be installed using any compatible package manager.

npm install firebolt-sdk --save

# For Yarn, use the command below.
yarn add firebolt-sdk

Using the library

import { Firebolt } from 'firebolt-sdk'

const firebolt = Firebolt();

const connection = await firebolt.connect({
  auth: {
    client_id: process.env.FIREBOLT_CLIENT_ID,
    client_secret: process.env.FIREBOLT_CLIENT_SECRET,
  },
  account: process.env.FIREBOLT_ACCOUNT,
  database: process.env.FIREBOLT_DATABASE,
  engineName: process.env.FIREBOLT_ENGINE_NAME
});

const statement = await connection.execute("SELECT 1");

// fetch statement result
const { data, meta } = await statement.fetchResult();

// or stream result
const { data } = await statement.streamResult();

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

const rows = []

for await (const row of data) {
  rows.push(row);
}

console.log(rows)

Contents

• About
• Documentation
• Usage
  • Create connection
    • ConnectionOptions
    • AccessToken
    • Client credentials
    • engineName
  • Test connection
  • Engine URL
  • Execute query
    • ExecuteQueryOptions
    • parameters
    • Named parameters
    • QuerySettings
    • ResponseSettings
  • Fetch result
  • Stream result
  • Result hydration
  • Engine management
    • getByName
    • Engine
      • start
      • stop
  • Database management
    • getByName
    • Database
• Recipes
  • Streaming results
  • Custom stream transformers

About

The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.

firebolt-sdk supports Node.js > v14.

Documentation

Usage

Create connection

const connection = await firebolt.connect(connectionOptions);

ConnectionOptions

type AccessTokenAuth = {
  accessToken: string;
};

type ClientCredentialsAuth = {
  client_id: string;
  client_secret: string;
};

type ConnectionOptions = {
  auth: AccessTokenAuth | ServiceAccountAuth;
  database: string;
  engineName?: string;
  engineEndpoint?: string;
  account?: string;
};

engineName

You can omit engineName and execute AQL queries on such connection.

AccessToken

Instead of passing client id/secret directly, you can also manage authentication outside of node sdk and pass accessToken when creating the connection

const connection = await firebolt.connect({
  auth: {
    accessToken: "access_token",
  },
  engineName: 'engine_name',
  account: 'account_name',
  database: 'database',
});

Client credentials

Default way of authenticating is with the client credentials

const connection = await firebolt.connect({
  auth: {
    client_id: 'b1c4918c-e07e-4ab2-868b-9ae84f208d26';
    client_secret: 'secret';
  },
  engineName: 'engine_name',
  account: 'account_name',
  database: 'database',
});

Test connection

TODO: write motivation connection can be tested using:

const firebolt = Firebolt();
await firebolt.testConnection(connectionOptions)

which will perform authentication and simple select 1 query

Engine URL

Firebolt engine URLs use the following format:

<engine-name>.<account-name>.<region>.app.firebolt.io

For example: your-engine.your-account.us-east-1.app.firebolt.io. You can find and copy your engine endpoint name in the Firebolt web UI.

Execute Query

const statement = await connection.execute(query, executeQueryOptions);

Execute Query with set flags

const statement = await connection.execute(query, {
  settings: { query_id: 'hello' }
});

ExecuteQueryOptions

export type ExecuteQueryOptions = {
  parameters:? unknown[];
  settings?: QuerySettings;
  response?: ResponseSettings;
};

parameters

parameters field is used to specify replacements for ? symbol in the query.

For example:

const statement = await connection.execute("select ?, ?", {
  parameters: ["foo", 1]
});

will produce select 'foo', 1 query

Format Tuple:

import { Tuple } from 'firebolt-sdk'

const statement = await connection.execute("select ? where bar in ?", {
  parameters: [
    1,
    new Tuple(['foo'])
  ]
});

Named parameters

namedParameters field is used to specify replacements for :name tokens in the query.

For example:

const statement = await connection.execute("select :foo, :bar", {
  namedParameters: { foo: "foo", bar: 123 }
});

will produce select 'foo', 123 query

QuerySettings

You can also use QuerySettings to specify set flags. For example: { query_id: 'hello' }

ResponseSettings

Fetch result

const { data, meta, statistics } = await statement.fetchResult();

The Promise API is not recommended for SELECT queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.

It is recommended to use LIMIT in your queries when using the Promise API.

Stream result

const { data } = await statement.streamResult();
const rows: unknown[] = [];

data.on("metadata", metadata => {
  console.log(metadata);
});

data.on("error", error => {
  console.log(error);
});

for await (const row of data) {
  rows.push(row);
}

Result hydration

firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:

Engine management

Engines can be managed by using the resourceManager object.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const enginesService = firebolt.resourceManager.engine

getByName

Returns engine using engine name.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")

Engine

Start

Starts an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.start()

Stop

Stops an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.stop()

Engine create

Creates an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.create("engine_name");

Attach to database

Attaches an engine to a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.attachToDatabase("engine_name", "database_name");

Engine delete

Deletes an engine.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name");
await engine.delete();

Database management

Databases can be managed by using the resourceManager object.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const databaseService = firebolt.resourceManager.database

Database getByName

Returns database using database name.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name")

Database

Database create

Creates a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.create("database_name");

Get attached engines

Get engines attached to a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
const engines = database.getAttachedEngines();

Database delete

Deletes a database.

import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
await database.delete();

Recipes

Streaming results

The recommended way to consume query results is by using streams.

For convenience, statement.streamResult also returns meta: Promise<Meta[]> and statistics: Promise<Statistics>, which are wrappers over data.on('metadata') and data.on('statistics').

const firebolt = Firebolt();

const connection = await firebolt.connect(connectionParams);

const statement = await connection.execute("SELECT 1");

const {
  data,
  meta: metaPromise,
  statistics: statisticsPromise
} = await statement.streamResult();

const rows: unknown[] = [];

const meta = await metaPromise;

for await (const row of data) {
  rows.push(row);
}

const statistics = await statisticsPromise

console.log(meta);
console.log(statistics);
console.log(rows)

Custom stream transformers

To achieve seamless stream pipes to fs or stdout, you can use the Transform stream.

import stream,  { TransformCallback } from 'stream';

class SerializeRowStream extends stream.Transform {
  public constructor() {
    super({
      objectMode: true,
      transform(
        row: any,
        encoding: BufferEncoding,
        callback: TransformCallback
      ) {
        const transformed = JSON.stringify(row);
        this.push(transformed);
        this.push('\n')
        callback();
      }
    });
  }
}

const serializedStream = new SerializeRowStream()

const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("select 1 union all select 2");

const { data } = await statement.streamResult();


data.pipe(serializedStream).pipe(process.stdout);

Or use rowParser that returns strings or Buffer:

const { data } = await statement.streamResult({
  rowParser: (row: string) => `${row}\n`
});

data.pipe(process.stdout);

Development process

Actions before

Setup env variables

cp .env.example .env

Execute tests

  npm test

License

Released under Apache License.