npm.io
2.13.12 • Published 2 weeks ago

n3

Licence
MIT
Version
2.13.12
Deps
2
Size
930 kB
Vulns
0
Weekly
0
Stars
797

Lightning fast, asynchronous, streaming RDF for JavaScript

Build Status Coverage Status npm version DOI

The N3.js library is an implementation of the RDF.js low-level specification that lets you handle RDF 1.2 in JavaScript easily. It offers:

Parsing and writing is:

  • asynchronous – triples arrive as soon as possible
  • streaming – streams are parsed as data comes in, so you can parse files larger than memory
  • ️ fast – triples are flying out at high speeds

Installation

For Node.js, N3.js comes as an npm package.

$ npm install n3
const N3 = require('n3');

N3.js seamlessly works in browsers via webpack or browserify. If you're unfamiliar with these tools, you can read webpack: Creating a Bundle – getting started or Introduction to browserify. You will need to create a "UMD bundle" and supply a name (e.g. with the -s N3 option in browserify).

You can also load it via CDN, either as a classic script that exposes a global N3:

<script src="https://unpkg.com/n3/browser/n3.min.js"></script>

or as an ES module that you can import directly:

<script type="module">
  import { Store, Parser, Writer } from 'https://unpkg.com/n3/browser/n3.esm.min.js';
</script>

Creating triples/quads

N3.js follows the RDF.js low-level specification.

N3.DataFactory will give you the factory functions to create triples and quads:

const { DataFactory } = N3;
const { namedNode, literal, defaultGraph, quad } = DataFactory;
const myQuad = quad(
  namedNode('https://ruben.verborgh.org/profile/#me'), // Subject
  namedNode('http://xmlns.com/foaf/0.1/givenName'),    // Predicate
  literal('Ruben', 'en'),                              // Object
  defaultGraph(),                                      // Graph
);
console.log(myQuad.termType);              // Quad
console.log(myQuad.value);                 // ''
console.log(myQuad.subject.value);         // https://ruben.verborgh.org/profile/#me
console.log(myQuad.object.value);          // Ruben
console.log(myQuad.object.datatype.value); // http://www.w3.org/1999/02/22-rdf-syntax-ns#langString
console.log(myQuad.object.language);       // en

When no language or datatype is supplied, literal automatically assigns XSD datatypes to JavaScript booleans, numbers, BigInts, and valid Date objects. Integer-valued numbers below 1e21 and all BigInts become xsd:integer, so use a BigInt when an integer must stay exact beyond Number.MAX_SAFE_INTEGER; other numbers become xsd:double. Dates are converted to UTC using Date.prototype.toISOString() and receive the xsd:dateTime datatype:

const created = literal(new Date('2017-04-27T14:39:48.901Z'));
console.log(created.value);          // 2017-04-27T14:39:48.901Z
console.log(created.datatype.value); // http://www.w3.org/2001/XMLSchema#dateTime

Always create terms through a data factory such as N3.DataFactory, and not by instantiating the term classes directly: direct construction is deprecated, because the factory functions are where term validation can be applied. In line with the RDF/JS specification, N3.js assumes that the value of any RDF/JS term it receives — whether from its own factory or from another implementation — was already validated when the term was created, and does not re-validate terms.

In the rest of this document, we will treat “triples” and “quads” equally: we assume that a quad is simply a triple in a named or default graph.

Parsing

From an RDF document to quads

N3.Parser transforms Turtle, TriG, N-Triples, or N-Quads document into quads through a callback:

const tomAndJerry = `PREFIX c: <http://example.org/cartoons#>
  # Tom is a cat
  c:Tom a c:Cat.
  c:Jerry a c:Mouse;
    c:smarterThan c:Tom.`

const parser = new N3.Parser();

parser.parse(tomAndJerry,
  (error, quad, prefixes) => {
    if (quad)
      console.log(quad);
    else
      console.log("# That's all, folks!", prefixes);
  });

The callback's first argument is an optional error value, the second is a quad. If there are no more quads, the callback is invoked one last time with null for quad and a hash of prefixes as third argument.

Alternatively, an object can be supplied with named callbacks for quads, prefixes, comments, and token processing:

const parser = new N3.Parser();

parser.parse(tomAndJerry, {
  // onQuad (optional) receives errors, quads, and completion
  onQuad: (err, quad) => { console.log(quad); },
  // onPrefix (optional) accepts a listener of type (prefix: string, iri: NamedNode) => void
  onPrefix: (prefix, iri) => { console.log(prefix, 'expands to', iri.value); },
  // onComment (optional) accepts a listener of type (comment: string) => void
  onComment: (comment) => { console.log('#', comment); },
});

onToken(token) runs immediately before a lexer token is processed, and onTokenEnd(token) runs immediately afterwards, including when processing throws.

const tokens = [];
const quads = parser.parse('<a> <b> "hello"@en.', {
  onToken: token => tokens.push(token),
  onTokenEnd: token => { /* Finish per-token bookkeeping here. */ },
});

Packages that extend a format with their own directives can register their names with the directives option. Between statements (outside graph blocks and formulas), a registered directive is accepted as @name . in Turtle, TriG and N3, and as NAME (case-insensitive) in all formats, including N-Triples and N-Quads. onDirective(name) receives each one in order with the quads, and N3.StreamParser emits them as directive events. Names consist of letters only, and cannot be words that already have a meaning in the grammar, such as prefix or a.

const parser = new N3.Parser({ format: 'N-Quads', directives: ['message'] });
parser.parse('<a> <b> <c> .\nMESSAGE\n<a> <b> <d> .\n', {
  onQuad: (error, quad) => { /* … */ },
  onDirective: name => { console.log('end of a message'); },
});

If no callbacks are provided, parsing happens synchronously returning an array of quads:

const parser = new N3.Parser();

// An array of resultant Quads
const quadArray = parser.parse(tomAndJerry);

By default, N3.Parser parses a permissive superset of Turtle, TriG, N-Triples, and N-Quads.
For strict compatibility with any of those languages, pass a format argument upon creation:

const parser1 = new N3.Parser({ format: 'N-Triples' });
const parser2 = new N3.Parser({ format: 'application/trig' });

Notation3 (N3) is supported only through the format argument:

const parser3 = new N3.Parser({ format: 'N3' });
const parser4 = new N3.Parser({ format: 'Notation3' });
const parser5 = new N3.Parser({ format: 'text/n3' });

It is possible to provide the base IRI of the document that you want to parse. This is done by passing a baseIRI argument upon creation:

const parser = new N3.Parser({ baseIRI: 'http://example.org/' });

In N3 mode, implicitEmptyPrefix can bind an undeclared empty prefix to the document IRI with a # fragment:

const parser = new N3.Parser({
  format: 'text/n3',
  baseIRI: 'http://example.org/document',
  implicitEmptyPrefix: true,
});

By default, N3.Parser will prefix blank node labels with a b{digit}_ prefix. This is done to prevent collisions of unrelated blank nodes having identical labels. The blankNodePrefix constructor argument can be used to modify the prefix or, if set to an empty string, completely disable prefixing:

const parser = new N3.Parser({ blankNodePrefix: '' });

The parser can output a backwards chaining rule such as _:q <= _:p. in two ways:

  • as _:p log:implies _:q. (default)
  • as _:q log:isImpliedBy _:p. (when the isImpliedBy flag is set to true)
const parser = new N3.Parser({ isImpliedBy: true });

By default, an empty formula {} is kept as a blank node graph term. The N3 spec tests (and the direction discussed in w3c-cg/N3#185) read it as the boolean literal "true"^^xsd:boolean instead; the emptyFormulaAsTrue flag enables that behavior:

const parser = new N3.Parser({ format: 'text/n3', emptyFormulaAsTrue: true });

In N3 documents, the parser by default rescopes blank node labels in lists and blank node property lists, in addition to formulas, so _:a inside a list does not co-reference _:a outside of it. The formulaScopedBlankNodes flag scopes blank node labels to formulas only, matching N3's formula-scoped blank node semantics (this will become the default in the next major version):

const parser = new N3.Parser({ format: 'N3', formulaScopedBlankNodes: true });

The parser rejects triple terms nested more than 1024 levels deep, so that the resulting quads stay within what the store and writer can traverse. The maxTripleTermDepth option changes this limit (Infinity removes it):

const parser = new N3.Parser({ maxTripleTermDepth: 64 });
From an RDF stream to quads

N3.Parser can parse Node.js streams as they grow, returning quads as soon as they're ready.

const parser = new N3.Parser(),
      rdfStream = fs.createReadStream('cartoons.ttl');
parser.parse(rdfStream, console.log);

N3.StreamParser is a Node.js stream and RDF.js Sink implementation. This solution is ideal if your consumer is slower, since source data is only read when the consumer is ready.

const streamParser = new N3.StreamParser(),
      rdfStream = fs.createReadStream('cartoons.ttl');
rdfStream.pipe(streamParser);
streamParser.pipe(new SlowConsumer());

function SlowConsumer() {
  const writer = new require('stream').Writable({ objectMode: true });
  writer._write = (quad, encoding, done) => {
    console.log(quad);
    setTimeout(done, 1000);
  };
  return writer;
}

A dedicated prefix event signals every prefix with prefix and term arguments. A dedicated comment event can be enabled by setting comments: true in the N3.StreamParser constructor.

Note that prefix and comment events are emitted as soon as they are parsed, whereas quads can remain buffered until the consumer is ready to read them. The order of these events relative to data events is therefore not guaranteed to match the position of prefixes and comments in the document. If their position matters, use N3.Parser with the onQuad, onPrefix and onComment callbacks instead, which are invoked in document order.

From a Web Stream to quads

N3.js consumes Node.js streams natively, but sources such as fetch produce Web Streams. On Node.js 17 or higher, convert such a stream into a Node.js stream:

const streamParser = new N3.StreamParser(),
      { Readable } = require('stream');
Readable.fromWeb(response.body).pipe(streamParser);

In browsers (or anywhere without Node.js streams), write the chunks to the parser directly, since N3.StreamParser exposes a standard writable stream interface:

const streamParser = new N3.StreamParser(),
      reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
(async () => {
  for (let result; !(result = await reader.read()).done;)
    if (!streamParser.write(result.value))
      await new Promise(resolve => streamParser.once('drain', resolve));
  streamParser.end();
})();

Writing

N3.Writer writes each quad as it arrives, so it cannot write N3 formulas (graphs used as terms). For full writing, including formulas, use n3-full-writer.

From quads to a string

N3.Writer serializes quads as an RDF document. Write quads through addQuad.

const writer = new N3.Writer({ prefixes: { c: 'http://example.org/cartoons#' } }); // Create a writer which uses `c` as a prefix for the namespace `http://example.org/cartoons#`
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Tom'),   // Subject
  namedNode('http://example.org/cartoons#name'),  // Predicate
  literal('Tom')                                  // Object
));
writer.end((error, result) => console.log(result));

By default, N3.Writer writes Turtle (or TriG if some quads are in a named graph).
To write N-Triples (or N-Quads) instead, pass a format argument upon creation:

const writer1 = new N3.Writer({ format: 'N-Triples' });
const writer2 = new N3.Writer({ format: 'application/trig' });

A baseIRI argument makes the writer abbreviate IRIs relative to that base in Turtle/TriG serializations. Pass writeBase: true to also write the base as an @base directive at the top of the document (N-Triples and N-Quads remain directive-free).

const writer = new N3.Writer({ baseIRI: 'http://example.org/', writeBase: true });

A version argument writes an RDF 1.2 version directive at the top of the document, as @version "1.2". in Turtle/TriG or VERSION "1.2" in N-Triples/N-Quads. The constructor throws for a label that contains quotes, backslashes, control characters, or unpaired surrogates.

const writer = new N3.Writer({ version: '1.2' });

Packages that add their own syntax can extend N3.Writer. this._endStatement() finishes the pending statement (and closes an open graph block), this._write(text, done) writes text to the output, and this._lineMode tells whether the output is N-Triples or N-Quads.

class GroupWriter extends N3.Writer {
  addGroup(quads, done) {
    for (const quad of quads)
      this.addQuad(quad);
    this._endStatement();
    this._write(this._lineMode ? 'GROUP\n' : '@group .\n', done);
  }
}
From quads to an RDF stream

N3.Writer can also write quads to a Node.js stream through addQuad.

const writer = new N3.Writer(process.stdout, { end: false, prefixes: { c: 'http://example.org/cartoons#' } });
writer.addQuad(
  namedNode('http://example.org/cartoons#Tom'),                   // Subject
  namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),   // Predicate
  namedNode('http://example.org/cartoons#Cat')                    // Object
);
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Tom'),     // Subject
  namedNode('http://example.org/cartoons#name'),  // Predicate
  literal('Tom')                                    // Object
));
writer.end();
From a quad stream to an RDF stream

N3.StreamWriter is a Node.js stream and RDF.js Sink implementation.

const streamParser = new N3.StreamParser(),
      inputStream = fs.createReadStream('cartoons.ttl'),
      streamWriter = new N3.StreamWriter({ prefixes: { c: 'http://example.org/cartoons#' } });
inputStream.pipe(streamParser);
streamParser.pipe(streamWriter);
streamWriter.pipe(process.stdout);
Blank nodes and lists

You might want to use the […] and list (…) notations of Turtle and TriG. However, a streaming writer cannot create these automatically: the shorthand notations are only possible if blank nodes or list heads are not used later on, which can only be determined conclusively at the end of the stream.

The blank and list functions allow you to create them manually instead:

const writer = new N3.Writer({ prefixes: { c: 'http://example.org/cartoons#',
                                       foaf: 'http://xmlns.com/foaf/0.1/' } });
writer.addQuad(
  writer.blank(
    namedNode('http://xmlns.com/foaf/0.1/givenName'),
    literal('Tom', 'en')),
  namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
  namedNode('http://example.org/cartoons#Cat')
);
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Jerry'),
  namedNode('http://xmlns.com/foaf/0.1/knows'),
  writer.blank([{
    predicate: namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
    object:    namedNode('http://example.org/cartoons#Cat'),
  },{
    predicate: namedNode('http://xmlns.com/foaf/0.1/givenName'),
    object:    literal('Tom', 'en'),
  }])
));
writer.addQuad(
  namedNode('http://example.org/cartoons#Mammy'),
  namedNode('http://example.org/cartoons#hasPets'),
  writer.list([
    namedNode('http://example.org/cartoons#Tom'),
    namedNode('http://example.org/cartoons#Jerry'),
  ])
);
writer.end((error, result) => console.log(result));

Storing

N3.Store allows you to store triples in memory and find them fast.

In this example, we create a new store and add the triples :Pluto a :Dog. and :Mickey a :Mouse.
Then, we find triples with :Mickey as subject.

const store = new N3.Store();
store.add(
  quad(
    namedNode('http://ex.org/Pluto'),
    namedNode('http://ex.org/type'),
    namedNode('http://ex.org/Dog')
  )
);
store.add(
  quad(
    namedNode('http://ex.org/Mickey'),
    namedNode('http://ex.org/type'),
    namedNode('http://ex.org/Mouse')
  )
);

// Retrieve all quads
for (const quad of store)
  console.log(quad);
// Retrieve Mickey's quads
for (const quad of store.match(namedNode('http://ex.org/Mickey'), null, null))
  console.log(quad);

If you are using multiple stores, you can reduce memory consumption by allowing them to share an entity index:

const entityIndex = new N3.EntityIndex();
const store1 = new N3.Store([], { entityIndex });
const store2 = new N3.Store([], { entityIndex });
Dataset Interface

This store adheres to the Dataset interface which exposes the following properties

Attributes:

  • size — A non-negative integer that specifies the number of quads in the set.

Methods:

  • add — Adds the specified quad to the dataset. Existing quads, as defined in Quad.equals, will be ignored.
  • delete — Removes the specified quad from the dataset.
  • has — Determines whether a dataset includes a certain quad.
  • match — Returns a new dataset that is comprised of all quads in the current instance matching the given arguments.
  • [Symbol.iterator] — Implements the iterator protocol to allow iteration over all quads in the dataset as in the example above.
  • toCanonical — Throws because it is not implemented. n3-canon provides a Store subclass that implements it with RDFC-1.0.
Addition and deletion of quads

The store implements the following manipulation methods in addition to the standard Dataset Interface (documentation):

  • addQuad to insert one quad
  • addQuads to insert an array of quads
  • removeQuad to remove one quad
  • removeQuads to remove an array of quads
  • remove to remove a stream of quads
  • removeMatches to remove all quads matching the given pattern
  • deleteGraph to remove all quads with the given graph
  • createBlankNode returns an unused blank node identifier
Searching quads or entities

The store provides the following search methods (documentation):

  • match returns a stream and generator of quads matching the given pattern
  • getQuads returns an array of quads matching the given pattern
  • countQuads counts the number of quads matching the given pattern
  • forEach executes a callback on all matching quads
  • every returns whether a callback on matching quads always returns true
  • some returns whether a callback on matching quads returns true at least once
  • getSubjects returns an array of unique subjects occurring in matching quads
  • forSubjects executes a callback on unique subjects occurring in matching quads
  • getPredicates returns an array of unique predicates occurring in matching quad
  • forPredicates executes a callback on unique predicates occurring in matching quads
  • getObjects returns an array of unique objects occurring in matching quad
  • forObjects executes a callback on unique objects occurring in matching quads
  • getGraphs returns an array of unique graphs occurring in matching quad
  • forGraphs executes a callback on unique graphs occurring in matching quads
Configuring match() semantics

The behaviour of match() can be configured as a store-wide default or per-call as follows:

import { Store, DataFactory } from 'n3';
const { namedNode } = DataFactory;

const store = new Store([], { matchSemantics: 'snapshot' });
const snapshot = store.match(namedNode('s'));
const forwarded = store.match(namedNode('s'), null, null, null, { matchSemantics: 'forwarded' });

Supported values:

  • 'lazy' (default) — the view (return value of Store#match) reflects the parent store until the first operation that materializes it (a mutation, or a materializing read such as size or has), after which it is frozen to a snapshot. Parent mutations made before that point remain visible in the view. In the next major version, only mutating operations will materialize a lazy view.
  • 'snapshot' — the view reflects the parent contents at the time of match(). Later parent mutations never affect it. This is the most spec-correct interpretation of an RDF/JS dataset and will become the default in the next major version.
  • 'forwarded' — the view always reflects the parent state, and mutations to the view are written through to the parent. Attempts to mutate on the view which do not match the views #match pattern result in an error.

A sub-view inherits its parent's matchSemantics.

For 'snapshot' and 'forwarded', an iteration (synchronous or via the stream) that is already in progress keeps a stable view of the quads as of when it started.

Reasoning

N3.js supports reasoning as follows:

import { Reasoner, Store, Parser } from 'n3';

const parser = new Parser({ format: 'text/n3' });
const rules = `
{
  ?s a ?o .
  ?o <http://www.w3.org/2000/01/rdf-schema#subClassOf> ?o2 .
} => {
  ?s a ?o2 .
} .
`

const rulesDataset = new Store(parser.parse(rules));
const dataset = new Store(/* Dataset */)

// Applies the rules to the store; mutating it
const reasoner = new Reasoner(store);
reasoner.reason(rulesDataset);

Note: N3.js currently only supports rules with Basic Graph Patterns in the premise and conclusion. Built-ins and backward-chaining are not supported. For an RDF/JS reasoner that supports all Notation3 reasoning features, see eye-js.

Limiting reasoning cost

When reasoning over rules or data that are not fully trusted, optional budgets bound the work reason() may perform:

const reasoner = new Reasoner(store, {
  maxDerivations: 100000, // maximum number of quads reason() may derive
  maxPremiseDepth: 10,    // maximum number of premise triples per rule
});
reasoner.reason(rulesDataset);

Both budgets are unbounded by default; reason() throws when one is exceeded, leaving any quads derived up to that point in the store.

Extensions

The following packages build on N3.js to support formats or features beyond the W3C specifications listed under Compatibility. They are maintained separately from N3.js; please report issues with them in their own repositories.

Package Description
n3.js-messages Parses and writes RDF Messages (message-delimited Turtle, TriG, N-Triples and N-Quads).
@jeswr/n3-provenance Experimental. Maps the terms of parsed quads back to their source positions in the input document. Terms that the parser generates, such as list nodes and reification scaffolding, have no position. It builds on the parser's onToken and onTokenEnd callbacks, but also subclasses private parser and index internals, so it is pinned to a specific N3.js revision.
n3-match-bgp Experimental. Matches basic graph patterns against a Store, or any RDF/JS dataset, and yields RDF/JS Bindings. It is being worked on as a consideration for the RDF/JS specifications, and will only move into N3.js if it lands there. On a Store, it joins on private index internals for speed.
n3-full-writer Experimental. Writes complete N3 documents, including formulas, by holding back the statements that use formulas until the end. It was proposed for N3.js in #822. It extends N3.Writer and overrides private writer methods.
n3-canon Experimental. Writes a Store, any RDF/JS dataset, or an iterable of quads as canonical N-Quads with blank nodes labeled by RDFC-1.0. It was proposed for N3.js in #856. It exports toCanonical() and a Store that extends N3.Store with that method, and reads the Store's private index internals.

Features intended for N3.js itself are considered once their specification is settled. If you would like to see one of these packages in N3.js, please open an issue or discussion.

To list a package here, open a pull request that adds a row to this table.

Compatibility

Format specifications

The N3.js parser and writer is fully compatible with the following W3C specifications:

In addition, the N3.js parser also supports Notation3 (N3) (no official specification yet).

The default mode is permissive and allows a mixture of different syntaxes. Pass a format option to the constructor with the name or MIME type of a format for strict, fault-intolerant behavior.

Validation

The parser validates the syntax of the selected format's grammar, with the following exceptions:

  • IRIs are not checked for full RFC 3987 well-formedness (<http://example.org/%ZZ> parses), and relative IRIs remain relative when no baseIRI option is given;
  • literal values are not checked against their datatype ("abc"^^xsd:integer parses);
  • language tags are checked against the grammar, not against BCP 47;

The writer trusts the terms it is given. Quads constructed with invalid term values are serialized as-is and can yield invalid documents.

Therefore, term validation should be done post-parsing to ensure that valid RDF terms should be produced.

One should also ensure that terms are valid prior to being passed into the writer; either by validation, or ensuring that valid RDF will always be produced by the application logic producing the terms.

The following code snipped shows how to validate that NamedNodes and Literals are validly formed. Depending on your application you may wish to apply further validation: such as ensuring that nested Quad terms are valid in RDF 1.2, and ensuring that termTypes are only occuring in the positions that is valid for RDF 1.1 and RDF 1.2.

const { Transform } = require('stream');
const { validateIri, IriValidationStrategy } = require('validate-iri');
const { validators } = require('rdf-validate-datatype');
const { parse: parseLanguageTag } = require('bcp-47');

function validateTerm(term) {
  switch (term.termType) {
  case 'NamedNode': // RDF requires absolute IRIs
    return validateIri(term.value, IriValidationStrategy.Strict) || null;
  case 'Literal':
    if (term.language) {
      let invalid = false;
      parseLanguageTag(term.language, { warning: () => { invalid = true; } });
      return invalid ? new Error(`Invalid language tag "${term.language}"`) : null;
    }
    const validate = validators.find(term.datatype);
    return validate && !validate(term.value)
      ? new Error(`Invalid value "${term.value}" for datatype ${term.datatype.value}`)
      : null; // unknown datatypes cannot be judged
  default:
    return null;
  }
}

const quadStream = fs.createReadStream('data.ttl')
  .pipe(new N3.StreamParser())
  .pipe(new Transform({
    objectMode: true,
    transform(quad, encoding, done) {
      const error = validateTerm(quad.subject) || validateTerm(quad.predicate) ||
                    validateTerm(quad.object) || validateTerm(quad.graph);
      done(error, error ? undefined : quad); // or: skip/collect instead of failing
    },
  }));

Parser-level opt-in validation modes covering the term and version dimensions are proposed in #634.

Interface specifications

The N3.js submodules are compatible with the following RDF.js interfaces:

License and contributions

N3.js is released under the MIT License.

Contributions are welcome, and bug reports or pull requests are always helpful. If you plan to implement a larger feature, it's best to contact us first.

Keywords