Nearfs NPM | npm.io

NEARFS

NEARFS is a distributed file system compatible with IPFS that uses the NEAR blockchain as a backend. It is a work in progress.

TLDR

IPFS blocks are recorded in transaction history as fs_store function calls
custom indexer can be used to collect all blocks and store them in a local file system (see scripts/load-from-near-lake.js)
IPFS-compatible gateway can be used to access files (see app.js)
Public gateway is available at https://ipfs.web4.near.page. It provides access to data stored on NEAR mainnet.

Useful tools

web4-deploy Deploy your website to NEAR blockchain + IPFS/NEARFS.
fast-ipfs Low level utilities to work with IPFS blocks, etc.
https://near.page Web4 gateway which uses NEARFS to resolve ipfs:// links when possible.

How to run gateway

yarn install
yarn start

How to run indexer

yarn install
node scripts/load-from-near-lake.js --bucket-name [near-lake-bucket-name]

Environment variables

PORT - port to run gateway on
NEARFS_STORAGE_PATH - path to NEARFS block storage
NEARFS_LOAD_NEAR_LAKE - if set to true (or yes) gateway will also load blocks from NEAR Lake. Use this if you want to avoid running load-from-near-lake.js script separately.
NEARFS_LAKE_BUCKET_NAME - name of the NEAR Lake S3 bucket to load blocks from
NEARFS_LAKE_REGION_NAME - region of the NEAR Lake S3 bucket to load blocks from
NEARFS_LAKE_ENDPOINT - endpoint of the NEAR Lake S3 bucket to load blocks from
NEARFS_LAKE_BATCH_SIZE - number of blocks to load from NEAR Lake in one batch
NEARFS_LAKE_INCLUDE - comma-separated list of account glob patterns to include when loading blocks from NEAR Lake
NEARFS_LAKE_EXCLUDE - comma-separated list of account glob patterns to exclude when loading blocks from NEAR Lake
AWS_ACCESS_KEY_ID - AWS access key ID to use when loading blocks from NEAR Lake
AWS_SECRET_ACCESS_KEY - AWS secret access key to use when loading blocks from NEAR Lake

API

Following IPFS gateway APIs are currently implemented. Note that there might be some differences in details like error handling, etc.

Most notable difference with regular IPFS is that all calls would either return immediately or 404. There is no wait for content to be fetched from other nodes, etc.

Also only CIDv1 is currently supported (see examples).