# @totemorg/core

> Provide core functions

Latest version **1.2.0** (published 2024-11-23) · ISC license · 0 weekly downloads

## Install

```sh
npm install @totemorg/core
pnpm add @totemorg/core
yarn add @totemorg/core
bun add @totemorg/core
```

## Health

**Score 30/100 (F)** — status: maintenance-mode.

Positive: esm support; no vulnerabilities.

Warnings: low downloads; no types.

Negative: stale; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.2.0 |
| Published | 2024-11-23 |
| First published | 2024-11-23 |
| Weekly downloads | 0 |
| License | ISC |
| TypeScript types | none |
| Module format | ESM + CommonJS |
| Dependencies | 5 |
| Unpacked size | 125.9 KB |
| Known vulnerabilities | 0 (+12 in 1 direct dependencies) |
| Install scripts | no |
| Author | ACME Software Disposal Service |
| Maintainers | totem4, totem3, totem2, totem1 |

## Links

- npm: https://www.npmjs.com/package/@totemorg/core
- Repository: https://github.com/totemorg/core
- Homepage: https://github.com/totemorg/enums#readme
- Issues: https://github.com/totemorg/enums/issues
- npm.io page: https://npm.io/package/@totemorg/core

## Dependencies (5)

- [imap](https://npm.io/package/imap.md) ^0.8.19
- [mysql2](https://npm.io/package/mysql2.md) ^3.9.7
- [nodemailer](https://npm.io/package/nodemailer.md) ^6.9.13
- [neo4j-driver](https://npm.io/package/neo4j-driver.md) ^4.4.11
- [nodemailer-smtp-transport](https://npm.io/package/nodemailer-smtp-transport.md) ^2.7.4

## Recent versions

- 1.2.0 (latest) — 2024-11-23

## README

# [ENUMS](https://www.npmjs.com/package/@totemorg/enums)

Provides methods to fetch, enumerate, stream, and index data.  Also provides connections
to the mysql, neo4j, txmail, rxmail services per env variables.
 
## Install

	npm install @totemorg/enums	
	npm update

## Start

	npm run	startdbs				# Start database servers
	npm run start 					# Start/unit test

## Manage

	npm run verminor				# Roll minor version
	npm run vermajor				# Roll major version
	npm run redoc					# Regen documentation
	npm run pubminor				# republish as minor version
	npm run pubmajor				# republish as major version

## Usage

	import { Copy, Each, Extend, Stream, ... } from "@totemorg/enums";

See program reference for more information.


## Program Reference
<details>
<summary>
<i>Open / Close</i>
</summary>
## Modules

<dl>
<dt><a href="#module_CORE">CORE</a></dt>
<dd><p>Provides enumeration and indexing methods, stream and serialization methods, MySQL and Neo4J connectors,
attach agent endpoints, regulate job workflows, fetch data, file, graph, and json stores.</p>
<p>This module documented IAW <a href="https://jsdoc.app/">jsdoc</a>.</p>
</dd>
<dt><a href="#mathjx.module_array">array</a></dt>
<dd></dd>
<dt><a href="#CORE.module_Array">Array</a></dt>
<dd></dd>
<dt><a href="#CORE.module_String">String</a></dt>
<dd></dd>
</dl>

<a name="module_CORE"></a>

## CORE
Provides enumeration and indexing methods, stream and serialization methods, MySQL and Neo4J connectors,
attach agent endpoints, regulate job workflows, fetch data, file, graph, and json stores.
 
This module documented IAW [jsdoc](https://jsdoc.app/).

**Requires**: <code>module:[os](https://nodejs.org/docs/latest/api/)</code>, <code>module:[cluster](https://nodejs.org/docs/latest/api/)</code>, <code>module:[fs](https://nodejs.org/docs/latest/api/)</code>, <code>module:[http](https://nodejs.org/docs/latest/api/)</code>, <code>module:[https](https://nodejs.org/docs/latest/api/)</code>, <code>module:[vm](https://nodejs.org/docs/latest/api/)</code>, <code>module:[cp](https://nodejs.org/docs/latest/api/)</code>, <code>module:[crypto](https://nodejs.org/docs/latest/api/)</code>, <code>module:[stream](https://nodejs.org/docs/latest/api/)</code>, <code>module:[mysql](https://www.npmjs.com/package/mysql)</code>, <code>module:[neo4j-driver](https://www.npmjs.com/package/neo4j-driver)</code>, <code>module:[nodemailer](https://www.npmjs.com/package/nodemailer)</code>, <code>module:[nodemailer-smtp-transport](https://www.npmjs.com/package/nodemailer-smtp-transport)</code>  
**Author**: [ACMESDS](https://totemorg.github.io)  

* [CORE](#module_CORE)
    * _static_
        * [.Attach(opts, agents, cb)](#module_CORE.Attach) ⇒
            * [~getGraph(req, res)](#module_CORE.Attach..getGraph)
            * [~getStore(req, res)](#module_CORE.Attach..getStore)
            * [~getFile(req, res)](#module_CORE.Attach..getFile)
            * [~getAgent(spec, agents)](#module_CORE.Attach..getAgent)
        * [.Pipe()](#module_CORE.Pipe)
        * [.Fetch(ref, data, ack)](#module_CORE.Fetch)
    * _inner_
        * [~TRACE(msg, ...args)](#module_CORE..TRACE)
        * [~LexisNexis()](#module_CORE..LexisNexis)

<a name="module_CORE.Attach"></a>

### CORE.Attach(opts, agents, cb) ⇒
Attach `agents` to the *agent cloud*, where each `agent(req,res)` specifies
its endpoint as follows:

    NAME: agent                     // handle notebook/data requests
    "/AREA": agent                  // handle file requests
    ".TYPE": agent                  // handle conversion requests 
    "/": agent 	                    // default handler for file requests 
    ".": agent	                    // default handler for type conversions
    NAME_MACHINE: agent             // handle requests via a MACHINE = py|cv|m|jx|db| ... 
    _IGNORE: agent                  // ignored
    "#EVENT": agent                 // handle notebook EVENT = charge|run requests
    GET|PUT|POST|DELETE: agent      // handle CRUD data requests 

The supplied `agents` hash is automatically extended with "/json", "/graph", and "/file" reader agents.

When a request arrives to a *agent server* , `Attach` first 

+ attempts to resolve the request to the *master agent*, then (if SQL-equipped)
+ attempts to resolve the request to a *remote agent*, then 
+ attempts to resolve the request to a *notebook agent*, then 
+ defaults to a CRUD agent based on the request type.  

The following options `opts` can be provided when attaching `agents`:

+ `cores` number of servers (workers) attached to the agents.
+ `port` number that servers listen on.

+ `CORS` flag to support Cross-Origin-Scripting.  

+ `never = file cacheing exemption hash { AREA: 1, PATH: 1, ... }`.
+ `parse` hash of (post,body) callbacks to parse posts in request body.
+ `mimes` mime hash sets the mime header for requested TYPE.
+ `https` options hash when creating a HTTPS server.  
+ `directs` = file remapping hash {AREA: "./PathPrefix", ...}.
+ `filters` = data filtering hash {TYPE: (res,req,res) => res(...), ...}.

+ `access(req)` method to validate client request req.
+ `proxy(cert,headers)` method to adjust cert from http headers.
+ `init(server,agents)` method to initialize the *agent server* after listener attached.
+ `busy()` method to block DOS attacks.
Machines* require additional modules:

+ py,cv,m   requires [@totemorg/agent](https://github.com/totemorg/agent)
+ jx        requires [@totemorg/man](https://github.com/totemorg/man)
+ db        requires [@totemorg/enums](https://github.com/totemorg/enums)

to support installed [python](), [opencv](), [matlab](), [mathjx](), [mysql db](https://dev.mysql.com/downloads/mysql/), 
and [neo4j db](https://neo4j.com/download/).

Each `agent(req,res)` is provided a `res` response method accepting a:

    string
    object
    array
    error
    function

and a `req` request containing:

    cookie: "...."		// client cookie string
    agent: "..."		// client browser info
    ipAddress: "..."	// client ip address
    referer: "http://site"		// url during a cross-site request
    method: "GET|PUT|POST|DELETE" 		// http request method
    now: date			// date stamp when request started
    post: "..."			// raw body text
    url	: "/query"		// requested url path
    reqSocket: socket	// socket to retrieve client cert, post etc
    resSocket: socket	// method to create socket to accept response
    cert: {...} 		// full client cert

    path: "/[area/...]name.type"	// full node path
    area: "name"		// file area being requested
    table: "name"		// name of dataset/table being requested
    type: "type" 		// type descriptor 

    query: {...} 		// raw keys from url
    where: {...} 		// sql-ized query keys from url
    body: {...}			// body keys from request 
    flags: {...} 		// flag keys from url
    index: {...}		// sql-ized index keys from url


To attach your nodejs-ready machine to the *agent cloud*:

    curl "http://totem/attach" | node [- ?|option=value|list|file.js] 

where `option` overrides the default port,cores,agents,etc.

**Kind**: static method of [<code>CORE</code>](#module_CORE)  
**Returns**: MACHINES Context of machines required by the agents  

| Param | Type | Description |
| --- | --- | --- |
| opts | <code>Object</code> | Service options |
| agents | <code>function</code> \| <code>Object</code> | Hash of (req,res)-agents |
| cb | <code>function</code> | Optional callback(agent) to process resolved agent |


* [.Attach(opts, agents, cb)](#module_CORE.Attach) ⇒
    * [~getGraph(req, res)](#module_CORE.Attach..getGraph)
    * [~getStore(req, res)](#module_CORE.Attach..getStore)
    * [~getFile(req, res)](#module_CORE.Attach..getFile)
    * [~getAgent(spec, agents)](#module_CORE.Attach..getAgent)

<a name="module_CORE.Attach..getGraph"></a>

#### Attach~getGraph(req, res)
Graph reader agent returns neo4j graph at the requested `path` = /AGENT/GRAPH?QUERY

**Kind**: inner method of [<code>Attach</code>](#module_CORE.Attach)  

| Param | Type | Description |
| --- | --- | --- |
| req | <code>Object</code> | totem session request |
| res | <code>function</code> | totem session response |

<a name="module_CORE.Attach..getStore"></a>

#### Attach~getStore(req, res)
Json reader Agent returns json store at the requested `path` = /AGENT/TABLE/STORE$EXPRESSION.

**Kind**: inner method of [<code>Attach</code>](#module_CORE.Attach)  

| Param | Type | Description |
| --- | --- | --- |
| req | <code>Object</code> | totem session request |
| res | <code>function</code> | totem session response |

<a name="module_CORE.Attach..getFile"></a>

#### Attach~getFile(req, res)
File reader agent returns file at the requested `path` = /AREA/FILE

**Kind**: inner method of [<code>Attach</code>](#module_CORE.Attach)  

| Param | Type | Description |
| --- | --- | --- |
| req | <code>Object</code> | totem session request |
| res | <code>function</code> | totem session response |

<a name="module_CORE.Attach..getAgent"></a>

#### Attach~getAgent(spec, agents)
Return agent from the `agents` hash with the given {type,area,method,table} spec.

**Kind**: inner method of [<code>Attach</code>](#module_CORE.Attach)  

| Param | Type | Description |
| --- | --- | --- |
| spec | <code>Object</code> | {type,area,method,table} to derive agent |
| agents | <code>Object</code> | hash of agents |

<a name="module_CORE.Pipe"></a>

### CORE.Pipe()
**Kind**: static method of [<code>CORE</code>](#module_CORE)  
<a name="module_CORE.Fetch"></a>

### CORE.Fetch(ref, data, ack)
Fetch data from a specified `ref` url

    PROTOCOL://HOST/FILE ? QUERY 

where PROTOCOL is

+ http(s) 	= http (https) protocol
+ curl(s) 	= curl (curls uses certs/fetch.pfx to authenticate)
+ wget(s)	= wget (wgets uses certs/fetch.pfx to authenticate)
+ mask 		= http access via rotated proxies
+ lexnex 	= Lexis-Nexis oauth access to documents
+ mysql		= mysql database
+ neo4j		= neo4j database
+ csv       = csv file
+ stream    = stream file
+ json      = json file
+ jpg       = image file
+ list      = list file
+ null      = null file
+ file      = file/folder 
+ SITEREF   = shortcut name

File paths may also contain wild-* cards.  The callback name (GET || PUT || POST || DELETE || JSON) determines the 
request method.  Optional QUERY keys:

    batch 	= NUMBER of records to fetch 
    offset	= NUMBER of records to offset fetch
    select	= "FIELD, ..." to fetch
    keys	= [...] header keys in csv source
    comma	= "delim" for csv source
    newline = "delim" for csv source

**Kind**: static method of [<code>CORE</code>](#module_CORE)  

| Param | Type | Description |
| --- | --- | --- |
| ref | <code>String</code> | source URL |
| data | <code>function</code> \| <code>object</code> | or ack callback(results||null) |
| ack | <code>function</code> | callback(results||null) |

**Example**  
```js
Fetch( ref, text => { // get request
})
```
**Example**  
```js
Fetch( ref, function json(ctx) {    // get request with json ctx
})
```
**Example**  
```js
Fetch( ref, [ ... ], function post(stat) { 	// post request with data hash list
})
```
**Example**  
```js
Fetch( ref, { ... }, function put(stat) { 	// put request with data hash
})
```
**Example**  
```js
Fetch( ref, null, function deete(stat) {	// delete request 
})
```
<a name="module_CORE..TRACE"></a>

### CORE~TRACE(msg, ...args)
Trace log message and args.

**Kind**: inner method of [<code>CORE</code>](#module_CORE)  

| Param | Type | Description |
| --- | --- | --- |
| msg | <code>String</code> | Tracing message |
| ...args | <code>Object</code> | Tracing arguments |

<a name="module_CORE..LexisNexis"></a>

### CORE~LexisNexis()
Start `N` Lexis-Nexis scrapes residing at the endpoint `endpt` + random counter when `R` is non-zero.

**Kind**: inner method of [<code>CORE</code>](#module_CORE)  
<a name="mathjx.module_array"></a>

## array

* [array](#mathjx.module_array)
    * [.mysql](#mathjx.module_array.mysql)
    * [.neo4j](#mathjx.module_array.neo4j)
    * [.txmail](#mathjx.module_array.txmail)
    * [.rxmail](#mathjx.module_array.rxmail)
    * [.Log](#mathjx.module_array.Log)
    * [.neoThread(cb)](#mathjx.module_array.neoThread)
    * [.sqlThread(cb)](#mathjx.module_array.sqlThread)
    * [.Start(host, ctx)](#mathjx.module_array.Start)
    * [.isFunction()](#mathjx.module_array.isFunction)
    * [.isString()](#mathjx.module_array.isString)
    * [.isObject()](#mathjx.module_array.isObject)
    * [.isNumber()](#mathjx.module_array.isNumber)
    * [.Copy(src, tar, deep)](#mathjx.module_array.Copy) ⇒ <code>Object</code>
    * [.Each(A, cb)](#mathjx.module_array.Each)
    * [.Notify()](#mathjx.module_array.Notify)

<a name="mathjx.module_array.mysql"></a>

### array.mysql
Stash for MySQL configuation options.

**Kind**: static constant of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.neo4j"></a>

### array.neo4j
Stash for Neo4J configuation options.

**Kind**: static constant of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.txmail"></a>

### array.txmail
Stash for Send-Mail configuation options.

**Kind**: static constant of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.rxmail"></a>

### array.rxmail
Stash for Receive-Mail configuation options.

**Kind**: static constant of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.Log"></a>

### array.Log
Dump message to the console.

**Kind**: static constant of [<code>array</code>](#mathjx.module_array)  

| Param | Type | Description |
| --- | --- | --- |
| ...args | <code>Object</code> | Data to dump |

<a name="mathjx.module_array.neoThread"></a>

### array.neoThread(cb)
Provide a Neo4J connector to the callback.

**Kind**: static method of [<code>array</code>](#mathjx.module_array)  

| Param | Type | Description |
| --- | --- | --- |
| cb | <code>function</code> | Callback cb(connector) |

<a name="mathjx.module_array.sqlThread"></a>

### array.sqlThread(cb)
Provide a MySQL connector to the callback.

**Kind**: static method of [<code>array</code>](#mathjx.module_array)  

| Param | Type | Description |
| --- | --- | --- |
| cb | <code>function</code> | Callback cb(connector) |

<a name="mathjx.module_array.Start"></a>

### array.Start(host, ctx)
Start a unit test using an optional `$(cmd,ctx)` command processor or a `CB()` callback provided in the `ctx` context.  
Inspect a KEY variable, file, or notebook or function with `?KEY`.

**Kind**: static method of [<code>array</code>](#mathjx.module_array)  

| Param | Type | Description |
| --- | --- | --- |
| host | <code>String</code> | Name of hosting module |
| ctx | <code>Object</code> | comamnd context hash |

<a name="mathjx.module_array.isFunction"></a>

### array.isFunction()
**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.isString"></a>

### array.isString()
**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.isObject"></a>

### array.isObject()
**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.isNumber"></a>

### array.isNumber()
**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
<a name="mathjx.module_array.Copy"></a>

### array.Copy(src, tar, deep) ⇒ <code>Object</code>
Copy source hash src to target hash tar.  If the copy is shallow (deep = false), a 
Copy({...}, {}) is equivalent to new Object({...}).  In a deep copy,
(e.g. deep = "."), src keys are treated as keys into the target thusly:

	{	
		A: value,			// sets target[A] = value

		"A.B.C": value, 	// sets target[A][B][C] = value

		"A.B.C.": {			// appends X,Y to target[A][B][C]
			X:value, Y:value, ...
		},	
	}

**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
**Returns**: <code>Object</code> - target hash  

| Param | Type | Description |
| --- | --- | --- |
| src | <code>Object</code> | source hash |
| tar | <code>Object</code> | target hash |
| deep | <code>String</code> | copy key |

<a name="mathjx.module_array.Each"></a>

### array.Each(A, cb)
Enumerate Object A over its keys with callback cb(key,val).

**Kind**: static method of [<code>array</code>](#mathjx.module_array)  

| Param | Type | Description |
| --- | --- | --- |
| A | <code>Object</code> | source object |
| cb | <code>function</code> | callback (key,val) |

<a name="mathjx.module_array.Notify"></a>

### array.Notify()
**Kind**: static method of [<code>array</code>](#mathjx.module_array)  
<a name="CORE.module_Array"></a>

## Array
<a name="CORE.module_String"></a>

## String
<a name="CORE.module_String.Task"></a>

### String.Task(spec, ctx, cb)
Task callback `cb(t,ctx)` at triggered events defined by the event context `ctx`:

	.start = starting number in relative-step mode || start datetime in clock-time mode (now clock-time mode)
	.end = end time (start + 100 steps)
	.on = number triggers steps enabled (0 disable)
	.off = number of trigger steps disabled (0 disable)
	.step = number of ms between triggers || "NUMBER [s|m|h|d|y]"  (1000 ms)
    .queue = "task queue" to share job status ("" disable)
    .client = "client name" to charge this request ("" disable)
    .priority = number of seconds to monitor task signoff status (0 disable)
    .source = "record source url" to fetch and feed to callback ("" disable)
    .select = "field, ..." to select from source ("" none)
    .batch = number of source records feed to callback cb(t,recs)

**Kind**: static method of [<code>String</code>](#CORE.module_String)  

| Param | Type | Description |
| --- | --- | --- |
| spec | <code>String</code> | Job regulation spec "STEP / ONSTEPS / OFFSTEPS / START / END" || [spec, ...] |
| ctx | <code>Object</code> | Job context |
| cb | <code>function</code> | Job callback(t,ctx) |

</details>

## Contacting, Contributing, Following

Feel free to 
* submit and status [TOTEM issues](http://totem.hopto.org/issues.view) 
* contribute to [TOTEM notebooks](http://totem.hopto.org/shares/notebooks/) 
* revise [TOTEM requirements](http://totem.hopto.org/reqts.view) 
* browse [TOTEM holdings](http://totem.hopto.org/) 
* or follow [TOTEM milestones](http://totem.hopto.org/milestones.view) 

## License

[MIT](LICENSE)

* * *

&copy; 2012 ACMESDS

---
_Source: https://npm.io/package/@totemorg/core · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._