# native-bucket

> native-bucket — a bridge between Cloudflare R2/Workers and browser IndexedDB for heavy binary datasets (GIS archives, ZIP, large assets): proxied Fetch with progress, R2 bucket listing/upload, persistent Cache. Browser-only. MIT.

Latest version **1.0.0** (published 2026-09-23) · MIT license · 0 weekly downloads

## Install

```sh
npm install native-bucket
pnpm add native-bucket
yarn add native-bucket
bun add native-bucket
```

## Health

**Score 60/100 (C)** — status: active.

Positive: esm support; no vulnerabilities; recently updated; high maintenance score.

Warnings: low downloads; no types.

## Facts

| | |
|---|---|
| Version | 1.0.0 |
| Published | 2026-09-23 |
| First published | 2026-09-23 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | ESM + CommonJS |
| Dependencies | 1 |
| Unpacked size | 29.5 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | Kenji Yoshida |
| Maintainers | kenji.yoshida.home.2026 |
| Keywords | cloudflare, r2, workers, indexeddb, cache, binary, browser |

## Links

- npm: https://www.npmjs.com/package/native-bucket
- Repository: https://github.com/kenjiyoshidahome2026-bit/ortho-earth
- Homepage: https://www.ortho-earth.com/
- Issues: https://github.com/kenjiyoshidahome2026-bit/ortho-earth/issues
- npm.io page: https://npm.io/package/native-bucket

## Dependencies (1)

- [geopbf](https://npm.io/package/geopbf.md) ^1.13.1

## Alternatives

- [memory-cache](https://npm.io/package/memory-cache.md) — 795.0K weekly downloads
- [@httptoolkit/proxy-agent](https://npm.io/package/@httptoolkit/proxy-agent.md) — 11.2K weekly downloads
- [express-cache-controller](https://npm.io/package/express-cache-controller.md) — 5.3K weekly downloads
- [http-cache-middleware](https://npm.io/package/http-cache-middleware.md) — 4.5K weekly downloads
- [cache2](https://npm.io/package/cache2.md) — 1.5K weekly downloads

## Recent versions

- 1.0.0 (latest) — 2026-09-23

## README

# native-bucket.js (v1.0.0)

A high-performance bridge between **Cloudflare Edge (R2/Workers)** and **Browser Storage (IndexedDB)**. Optimized for handling heavy binary datasets (GIS, archives, large assets) with zero-latency interaction.

[![Cloudflare Workers](https://img.shields.io/badge/Powered_by-Cloudflare_Workers-F38020?logo=cloudflare-workers&logoColor=white)](https://dash.cloudflare.com/)
[![Vite](https://img.shields.io/badge/Build_with-Vite-646CFF?logo=vite&logoColor=white)](<https://vitejs.dev/>)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
![Size](https://img.shields.io/badge/Size-6.6KB-brightgreen.svg)

> **Repo layout note** — development happens in the [ortho-earth monorepo](https://github.com/kenjiyoshidahome2026-bit/ortho-earth) (`packages/native-bucket`); this standalone repo is a read-only mirror synced on each release. Issues are welcome here; patches land in the monorepo. / 開発はモノレポ側で行い、ここはリリースごとに同期される公開ミラーです（Issue歓迎・変更はモノレポへ）。

---

## 🏗 System Architecture

![Architecture](etc/architecture.png)
*Orchestration of data flow across Remote Servers, Edge Proxies, R2 Buckets, and Local Persistent Cache.*

---

## 🎮 **Demo** [👉 View Live Demo](https://kenjiyoshidahome2026-bit.github.io/native-bucket/demo/)

Experience the zero-latency data flow and surgical ZIP extraction in action.

---

## 🚀 Server-Side Setup ([Cloudflare Workers](https://dash.cloudflare.com))

### 1. Configuration (`wrangler.toml`)

(Sign-up and) deploy the backend to handle R2 operations and Proxy requests. The `index.js` automatically manages CORS for you.
Please edit the file: "wrangler.toml" under "worker" directory.

```toml
name = "native-bucket-api"
main = "index.js"
compatibility_date = "2026-04-01"

[[r2_buckets]]
# [DO NOT CHANGE] Internal binding for the library
binding = "MY_BUCKET"
# [REQUIRED] Your actual R2 bucket name
bucket_name = "my-r2-storage" # <=== change here

[vars]
# [WHITELIST] Comma-separated domains (Suffix matching supported)
# Example: "ortho-earth.com,localhost:5173" allows all subdomains of ortho-earth.
ALLOWED_DOMAINS = "ortho-earth.com,localhost:5173" # <=== change here
```

### 2. Deployment with bash in console

```bash
bash
cd workers
npx wrangler deploy
```

---

## 🛠 Client-Side Setup

### Option A: ESM (Modern Bundlers)

```javascript
import nativeBucket from './src/index.js';
```

### Option B: CDN / Global Script (The Easiest Way)

The library automatically attaches to `window.nativeBucket` (or `self.nativeBucket`) for non-ESM or direct HTML environments.

```html
<script type="module" src="https://cdn.jsdelivr.net/gh/kenjiyoshidahome2026-bit/native-bucket@main/dist/native-bucket.iife.js"></script>
<script>
  window.addEventListener('load', () => { // Access via global nativeBucket after page load
    const { Fetch, Bucket, Cache } = nativeBucket("https://your-worker.dev/");
    ...
   });
</script>
```

---

## 📖 Detailed API Reference

### Initialization

Register your Worker endpoint to unlock the three core modules.

```javascript
const { Fetch, Bucket, Cache } = nativeBucket("https://your-worker.workers.dev/");
```

### 🔒 Proxy access control

`/proxy` is a public endpoint, so forwarding is gated. A request passes if **either** gate opens:

1. **Target host is on the list** — `PROXY_ALLOWED_HOSTS` in `wrangler.toml` (dot-boundary suffix match: `gsi.go.jp` matches `maps.gsi.go.jp` but not `evilgsi.go.jp`). Open to anyone, `GET`/`HEAD` only.
2. **Caller is trusted** — request `Origin` is in `ALLOWED_DOMAINS`, or `X-API-Key` matches `API_KEY`. Any target host, any method.

Otherwise `403`. **If `PROXY_ALLOWED_HOSTS` is unset, only gate 2 opens** — a deployment with no configuration forwards nothing to anonymous callers.

```toml
[vars]
PROXY_ALLOWED_HOSTS = "e-stat.go.jp,nlftp.mlit.go.jp,naturalearth.s3.amazonaws.com"
```

Always enforced, even for trusted callers:

* `http:` / `https:` only — no `file:`, `data:`, etc.
* Loopback, private, link-local and cloud-metadata addresses are refused (SSRF).
* Self-reference is refused (amplification loop).
* Redirects are followed manually, **re-checked at every hop** (max 5), so an allow-listed host cannot bounce you to an arbitrary one.

Do not list user-content hosts (`raw.githubusercontent.com`, generic S3 domains) — that turns the proxy into an arbitrary-file laundering path. Reach those through gate 2 instead.

Run `npm run test:proxy` to verify the gate (33 cases, no deploy needed).

---

### 🛰 `/tellus` — Tellus Traveler API relay

[Tellus](https://www.tellusxdp.com/) (JAXA satellite data: PALSAR-2, AVNIR-2, …) needs a per-user Bearer token, sends no
CORS headers, and pins the CORS origin of its signed download URLs to `tellusxdp.com` — so a browser cannot talk to it
directly. `/tellus/*` relays a **read-only allow-list** of the Traveler API with the token from the `TELLUS_TOKEN` secret;
the signed URL it returns (S3-style, 1 hour) is then read through `/proxy?url=` with HTTP Range. Trusted callers only
(same rule as gate 2 above). Purchases (`order`) and anything outside the list are refused.

| | |
| :-- | :-- |
| `GET /tellus/datasets/` | dataset list |
| `POST /tellus/data-search/` · `POST /tellus/datasets/{id}/data-search/` | scene search (body relayed as-is, 64 KB max) |
| `GET /tellus/datasets/{id}/data/{id}/files/` · `POST …/files/{n}/download-url/` | file list / signed URL |
| `GET /tellus/webcog?dataset={id}&data={id}` | one call: picks the scene's `*_webcog.tif` (Tellus display COG, EPSG:4326) and returns `{download_url, name, size_bytes, expires_in}` |

```bash
npx wrangler secret put TELLUS_TOKEN     # issue the token at Tellus: account menu → API token
npm run test:tellus                      # 15 cases, no deploy needed
```

---

### 🌐 `Fetch(url, options)`

A smart proxy that bypasses CORS and can surgically extract specific files from remote ZIP archives.

| Parameter | Type | Description |
| :--- | :---: | :--- |
| `type` | String | Output format: `"file"` (Default), `"blob"`, `"json"`, `"text"`. |
| `cors` | Boolean | true/false: pre-flight check without this parameter |
| `target` | String | Path inside the ZIP to extract a specific file. |
| `encoding` | String | encoding (default:`"utf8"`) |
| `silent` | Boolean | if true then no progress log |
| `eventTarget` | dom | target of event (default: window or self[webWorker]) |

```javascript
// get an entire remote zip file
const zip = await Fetch("https://server.com/data.zip");
console.log(`Received: ${zip.name} (${zip.size} bytes)`);

// Extract a file from remote ZIP as JSON without pre-flight.
const json = await Fetch("https://server.com/data.zip", { target: "layers/japan.geojson" ,cors:true, type:"json"});
console.log(`Received: `, json);
```

### 🪣 `Bucket(directory, options)`

High-level interface for Cloudflare R2. Features automatic Gzip detection and parallelized Multipart uploads for files >5MB.

| Parameter | Type | Description |
| :--- | :---: | :--- |
| `silent` | Boolean | if true then no progress log |
| `eventTarget` | dom | target of event (default: window or self[webWorker]) |

```javascript
const storage = await Bucket("v1/geodata");
const file = new File(["This is a file"], "test.txt", {type:"text/plain"});

// Upload a File object (Auto-handles multipart if large)
await storage.put(file);

// Download as a File object (Auto-decompressed if Gzipped)
const file = await storage.get("test.txt");

// get meta information from the File. (size, ETag etc.)
const meta = await storage.meta("test.txt");

// Rename file
await storage.move("test.txt", "text.old.txt");

// delete file
await storage.del("text.old.txt");

// List items in the directory
const list = await storage.list();

// read a zip file as file array
const files = await storage.gets("name");

// put a zip file from file array
await storage.puts(fileArray);
```

### ⚡ `Cache(dbName/tableName)`

A persistent Key-Value file store powered by IndexedDB. Perfect for instant subsequent loads with **ultra-low latency**. For categorization, several tableNames can be assigned to the one same dbName. This case, the version of indexedDB will be incremented automatically, and users don't need to take care of "onupgradeneeded".

```javascript
// open the database with "dbName/tableName"
const local = await Cache("assets/v1");

// List names in database
const list = await local();

// Load the File object instantly (Getter)
const file = await local("tile_01");

// Save a File locally (Setter)
await local(file); // or await local(file.name, file);

// Delete a File locally
await local("tile_01", null);
```

---

## 🔒 Security: Suffix-Matching Whitelist

Access is strictly enforced via the `ALLOWED_DOMAINS` whitelist in `wrangler.toml`.

- **`ortho-earth.com`** matches `ortho-earth.com`, `www.ortho-earth.com`, `dev.ortho-earth.com`, etc.
- **`localhost:5173`** allows access from your local dev-server.

---

## 📄 License

(c) 2026 Kenji Yoshida. Released under the **MIT License**.

---
_Source: https://npm.io/package/native-bucket · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._