@azure/storage-blob-changefeed v12.0.0-preview.4
Azure Storage Blob Change Feed client library for JavaScript
Server Version: 2019-12-12
The change feed provides an ordered, guaranteed, durable, immutable, read-only transaction log of all the changes that occur to blobs and blob metadata in your storage account. Client applications can read these logs at any time. The change feed enables you to build efficient and scalable solutions that process change events that occur in your Blob Storage account at a low cost.
This project provides a client library in JavaScript that makes it easy to consume the change feed.
Use the client libraries in this package to:
- Reading change feed events, all or within a time range
- Resuming reading events from a saved position
Source code | Package (npm) | API Reference Documentation | Product documentation | Samples |
Getting started
Prerequisites: You must have an Azure subscription and a Storage Account to use this package. If you are using this package in a Node.js application, then Node.js version 8.0.0 or higher is required.
Install the package
The preferred way to install the Azure Storage Blob Change Feed client library for JavaScript is to use the npm package manager. Type the following into a terminal window:
npm install @azure/storage-blob-changefeed
Authenticate the client
This library uses an authenticated BlobServiceClient
to initialize. Refer to storage-blob for how to authenticate a BlobServiceClient
.
Compatibility
For this preview, this library is only compatible with Node.js.
Key concepts
The change feed is stored as blobs in a special container in your storage account at standard blob pricing cost. You can control the retention period of these files based on your requirements. Change events are appended to the change feed as records in the Apache Avro format specification: a compact, fast, binary format that provides rich data structures with inline schema. This format is widely used in the Hadoop ecosystem, Stream Analytics, and Azure Data Factory.
This library offers a client you can use to fetch the change events.
Examples
Initialize the change feed client
The BlobChangeFeedClient
requires a BlobServiceClient
to initialize. Refer to storage-blob for how to create the blob service client. Here is an example using StorageSharedKeyCredential
.
const { BlobServiceClient, StorageSharedKeyCredential } = require("@azure/storage-blob");
const { BlobChangeFeedClient } = require("@azure/storage-blob-changefeed");
// Enter your storage account name and shared key
const account = "<account>";
const accountKey = "<accountkey>";
// Use StorageSharedKeyCredential with storage account and account key
// StorageSharedKeyCredential is only available in Node.js runtime, not in browsers
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const blobServiceClient = new BlobServiceClient(
`https://${account}.blob.core.windows.net`,
sharedKeyCredential
);
const changeFeedClient = new BlobChangeFeedClient(blobServiceClient);
Reading all events in the Change Feed
Use BlobChangeFeedClient.listChanges()
to get iterators to iterate through the change events.
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
let changeFeedEvents : BlobChangeFeedEvent[] = [];
for await (const event of changeFeedClient.listChanges()) {
changeFeedEvents.push(event);
}
By page.
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
let changeFeedEvents : BlobChangeFeedEvent[] = [];
for await (const eventPage of changeFeedClient.listChanges().byPage()) {
for (const event of eventPage) {
changeFeedEvents.push(event);
}
}
Resuming reading events with a cursor
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
let changeFeedEvents : BlobChangeFeedEvent[] = [];
const firstPage = await changeFeedClient.listChanges().byPage({maxPageSize: 10}).next();
for (const event of firstPage) {
changeFeedEvents.push(event);
}
// Resume iterating from the previous position with the continuationToken.
for await (const eventPage of changeFeedClient.listChanges().byPage({continuationToken: firstPage.continuationToken})) {
for (const event of eventPage) {
changeFeedEvents.push(event);
}
}
Reading events within a time range
Pass start time and end time to BlobChangeFeedClient.listChanges()
to fetch events within a time range.
Note that for this preview release, the change feed client will round start time down to the nearest hour, and round end time up to the next hour.
const { BlobChangeFeedEvent } = require("@azure/storage-blob-changefeed");
const start = new Date(Date.UTC(2020, 1, 21, 22, 30, 0)); // will be rounded down to 22:00
const end = new Date(Date.UTC(2020, 4, 8, 21, 10, 0)); // will be rounded up to 22:00
let changeFeedEvents : BlobChangeFeedEvent[] = [];
// You can also provide just a start or end time.
for await (const event of changeFeedClient.listChanges({ start, end })) {
changeFeedEvents.push(event);
}
Troubleshooting
Enabling logging may help uncover useful information about failures. In order to see a log of HTTP requests and responses, set the AZURE_LOG_LEVEL
environment variable to info
. Alternatively, logging can be enabled at runtime by calling setLogLevel
in the @azure/logger
:
import { setLogLevel } from "@azure/logger";
setLogLevel("info");
Next steps
More code samples:
- Blob Storage Change Feed Samples (JavaScript)
- Blob Storage Change Feed Samples (TypeScript)
- Blob Storage Change Feed Test Cases
Contributing
If you'd like to contribute to this library, please read the contributing guide to learn more about how to build and test the code.
Also refer to Storage specific guide for additional information on setting up the test environment for storage libraries.
2 days ago
6 days ago
8 days ago
9 days ago
12 days ago
13 days ago
14 days ago
15 days ago
16 days ago
19 days ago
20 days ago
21 days ago
22 days ago
28 days ago
29 days ago
1 month ago
1 month ago
2 months ago
2 months ago
2 months ago
2 months ago
2 months ago
2 months ago
2 months ago
3 months ago
3 months ago
3 months ago
3 months ago
3 months ago
3 months ago
4 months ago
4 months ago
4 months ago
5 months ago
5 months ago
6 months ago
6 months ago
6 months ago
5 months ago
6 months ago
6 months ago
6 months ago
6 months ago
10 months ago
7 months ago
9 months ago
7 months ago
7 months ago
7 months ago
9 months ago
7 months ago
9 months ago
10 months ago
11 months ago
11 months ago
11 months ago
11 months ago
10 months ago
12 months ago
12 months ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
1 year ago
2 years ago
2 years ago
2 years ago
1 year ago
1 year ago
1 year ago
2 years ago
1 year ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
2 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago