# @datagica/datanote-service-file2doc

> Datanote extraction engine as micro service

Latest version **0.0.0** (published 2018-10-18) · GPLv3 license · 0 weekly downloads

## Install

```sh
npm install @datagica/datanote-service-file2doc
pnpm add @datagica/datanote-service-file2doc
yarn add @datagica/datanote-service-file2doc
bun add @datagica/datanote-service-file2doc
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.0.0 |
| Published | 2018-10-18 |
| First published | 2018-10-18 |
| Weekly downloads | 0 |
| License | GPLv3 |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 52 |
| Unpacked size | 43.9 KB |
| Known vulnerabilities | 0 (+2 in 1 direct dependencies) |
| Install scripts | no |
| Author | Julian Bilcke |
| Maintainers | datagica |

## Links

- npm: https://www.npmjs.com/package/@datagica/datanote-service-file2doc
- npm.io page: https://npm.io/package/@datagica/datanote-service-file2doc

## Dependencies (52)

- [tmp](https://npm.io/package/tmp.md) 0.0.33
- [flat](https://npm.io/package/flat.md) ^4.1.0
- [micro](https://npm.io/package/micro.md) ^9.3.3
- [file-type](https://npm.io/package/file-type.md) ^10.0.0
- [micro-post](https://npm.io/package/micro-post.md) ^2.0.0
- [node-fetch](https://npm.io/package/node-fetch.md) ^2.2.0
- [query-string](https://npm.io/package/query-string.md) ^6.2.0
- [micro-compress](https://npm.io/package/micro-compress.md) ^1.0.0
- [micro-jwt-auth](https://npm.io/package/micro-jwt-auth.md) ^1.10.0
- [micro-ratelimit](https://npm.io/package/micro-ratelimit.md) ^0.4.0
- [@datagica/parse-cells](https://npm.io/package/@datagica/parse-cells.md) 0.0.1
- [@datagica/parse-drugs](https://npm.io/package/@datagica/parse-drugs.md) 0.0.0
- [@datagica/parse-ibans](https://npm.io/package/@datagica/parse-ibans.md) 0.0.1
- [@datagica/parse-names](https://npm.io/package/@datagica/parse-names.md) 0.0.8
- [@datagica/parse-alloys](https://npm.io/package/@datagica/parse-alloys.md) 0.0.0
- [@datagica/parse-emails](https://npm.io/package/@datagica/parse-emails.md) 0.0.4
- [@datagica/parse-events](https://npm.io/package/@datagica/parse-events.md) 0.0.0
- [@datagica/parse-family](https://npm.io/package/@datagica/parse-family.md) 0.0.5
- [@datagica/parse-nerves](https://npm.io/package/@datagica/parse-nerves.md) 0.0.1
- [@datagica/parse-phones](https://npm.io/package/@datagica/parse-phones.md) 0.0.6
- [@datagica/parse-skills](https://npm.io/package/@datagica/parse-skills.md) 0.0.2
- [@datagica/parse-artists](https://npm.io/package/@datagica/parse-artists.md) 0.0.0
- [@datagica/parse-muscles](https://npm.io/package/@datagica/parse-muscles.md) 0.0.1
- [@datagica/parse-viruses](https://npm.io/package/@datagica/parse-viruses.md) 0.0.2
- [@datagica/parse-weapons](https://npm.io/package/@datagica/parse-weapons.md) 0.0.1
- [@datagica/read-document](https://npm.io/package/@datagica/read-document.md) ^0.1.2
- [@datagica/parse-arteries](https://npm.io/package/@datagica/parse-arteries.md) 0.0.1
- [@datagica/parse-bacteria](https://npm.io/package/@datagica/parse-bacteria.md) 0.0.2
- [@datagica/parse-diplomas](https://npm.io/package/@datagica/parse-diplomas.md) 0.0.0
- [@datagica/parse-diseases](https://npm.io/package/@datagica/parse-diseases.md) 0.0.1
- [@datagica/parse-document](https://npm.io/package/@datagica/parse-document.md) 0.0.16
- [@datagica/parse-products](https://npm.io/package/@datagica/parse-products.md) 0.0.3
- [@datagica/parse-proteins](https://npm.io/package/@datagica/parse-proteins.md) 0.0.1
- [@datagica/parse-symptoms](https://npm.io/package/@datagica/parse-symptoms.md) 0.0.0
- [@datagica/parse-addresses](https://npm.io/package/@datagica/parse-addresses.md) 0.0.7
- [@datagica/parse-companies](https://npm.io/package/@datagica/parse-companies.md) 0.0.7
- [@datagica/parse-countries](https://npm.io/package/@datagica/parse-countries.md) 0.0.0
- [@datagica/parse-interests](https://npm.io/package/@datagica/parse-interests.md) 0.0.11
- [@datagica/parse-languages](https://npm.io/package/@datagica/parse-languages.md) 0.0.4
- [@datagica/parse-locations](https://npm.io/package/@datagica/parse-locations.md) 0.0.8
- [@datagica/parse-positions](https://npm.io/package/@datagica/parse-positions.md) 0.0.0
- [@datagica/classify-document](https://npm.io/package/@datagica/classify-document.md) 0.0.5
- [@datagica/parse-compensation](https://npm.io/package/@datagica/parse-compensation.md) 0.0.2
- [@datagica/parse-health-risks](https://npm.io/package/@datagica/parse-health-risks.md) 0.0.0
- [@datagica/parse-institutions](https://npm.io/package/@datagica/parse-institutions.md) 0.0.9
- [@datagica/parse-protagonists](https://npm.io/package/@datagica/parse-protagonists.md) 0.0.5
- [@datagica/parse-social-handles](https://npm.io/package/@datagica/parse-social-handles.md) 0.0.6
- [@datagica/parse-business-events](https://npm.io/package/@datagica/parse-business-events.md) 0.0.0
- [@datagica/parse-generic-animals](https://npm.io/package/@datagica/parse-generic-animals.md) 0.0.2
- [@datagica/parse-criminal-evidences](https://npm.io/package/@datagica/parse-criminal-evidences.md) 0.0.0
- [@datagica/parse-engineering-events](https://npm.io/package/@datagica/parse-engineering-events.md) 0.0.0
- [@datagica/parse-engineering-elements](https://npm.io/package/@datagica/parse-engineering-elements.md) 0.0.0

## Recent versions

- 0.0.0 (latest) — 2018-10-18

## README

# text2doc

*The Datanote feature extraction engine, as micro service*

## TODO

- support multiple formats:
  - datanote: a custom, low-level format supported by Datanote
  - json: basic list of entities
  - gexf: GEXF graph
  - csv: CSV graph (for Neo4J)  https://neo4j.com/developer/guide-import-csv/

## List of features

### Domains

The following domains are recognized by the API, but you can also define your own
(see the `Custom fields` paragraph).

detective:
- address
- email
- event
- evidence
- family
- phone
- position
- protagonist
- weapon

lifesciences:
- muscle
- animal
- artery
- bacteria
- cell
- disease
- drug
- healthRisk
- institution
- nerve
- protagonist
- location
- protein
- symptom
- virus

marketing:
- businessEvent
- institution
- position
- protagonist
- social
- product

engineer:
- engineeringElement
- engineeringEvent
- healthRisk
- institution
- protagonist

workplace:
- address
- diploma
- email
- family
- institution
- interest
- language
- phone
- position
- protagonist
- skill
- social
-
fiction:
- character
- location
- weapon
- event
- evidence

arts:
- email
- phone
- address
- compensation
- protagonist
- artist
- interest

generic:
- businessEvent
- engineeringEvent
- intelligenceEvent
- protagonist
- location

### Custom fields

Optional url parameters:

- locale: `en`, `fr` (example: `?locale=en`, `&locale=fr`..)
- fields: values to keep (example: `fields=id,label`, `&fields=label,links,target`..)
- domain: `PoliceReport`, see source for more (example: `?domain=PoliceReport`..)
- types: `bacteria`, `address`, `event`, see source for more
- - format: `graphson`, `gdf`, `gexf` (example: `?format=gdf`..)

Note: since `domain` cannot be used at the same time as `types`, `types` will
have priority and `domain` will have no effect.

### Domains and entity types

Current extraction model (you can change this, if your edit `engine.js`):

```javascript
{
  PoliceReport: [
    'email',
    'phone',
    'location',
    'evidence',
    'event',
    'protagonist',
    'position',
    'weapon',
  ],
  generic: [
    'protagonist',
  ]
}
```

## Usage

Examples use [httpie](https://github.com/jakubroztocil/httpie) with [jq](https://github.com/stedolan/jq), but you can also use curl or something else.

The content-type is optional, it can help the app if there is an encoding
issue with magic number.

### Example with curl

```bash
curl -X POST "http://localhost:3000?locale=en&types=animal&format=gdf" -d "THE HIPPO KILLS THE DOLPHIN"
curl -X POST "http://localhost:3000?locale=en&types=protagonist,weapon&format=gdf" -d "James bond buys an ak-47"
curl -X POST "http://localhost:3000" --data-binary "@tests/fixtures/police_en.txt"
curl -X POST "http://localhost:3000?locale=en&types=protagonist,virus" -d "James Bond has caught the terrorist carrying H5N1"
```

### Example with httpie and jq

```bash
https POST "http://localhost:3000?locale=en&types=virus" body="the monkey died of ebola" | jq
https POST "http://localhost:3000" body="James Bond" | jq
https POST "http://localhost:3000" body="James Bond" | jq
https POST "http://localhost:3000?&fields=label,links,link,target&locale=en" body="James Bond"  | jq
https POST "http://localhost:3000?locale=en" body="James Bond"  | jq
https POST "http://localhost:3000?&fields=label,links,link,target" body="James Bond"  | jq
https POST "http://localhost:3000?locale=en&types=protagonist,virus" body="James Bond has caught the terrorist carrying H5N1" | jq
```

### Longer example

```bash
https POST "http://localhost:3000?fields=link,links,target,properties,ngram,begin,end,label,gender,number,firstname,lastname&locale=en" body="James Bond buys an AK-47"
```

output:
```json
{
  "type": "record",
  "label": {},
  "properties": {},
  "links": [
    {
      "link": {
        "type": "link",
        "label": "Mentions"
      },
      "properties": {
        "ngram": "James Bond",
        "begin": 0,
        "end": 10
      },
      "target": {
        "properties": {
          "firstname": "james",
          "lastname": "bond",
          "gender": [
            "m"
          ]
        },
        "links": [
          {
            "link": {
              "type": "link",
              "label": "Type"
            },
            "properties": {},
            "target": {
              "type": "entity",
              "label": "Protagonist"
            }
          },
          {
            "link": {
              "type": "purchase",
              "label": "Purchase"
            },
            "properties": {},
            "target": {
              "properties": {
                "number": "singular",
                "gender": "neutral"
              },
              "links": [
                {
                  "link": {
                    "type": "link",
                    "label": "Type"
                  },
                  "properties": {},
                  "target": {
                    "type": "entity",
                    "label": "Generic"
                  }
                }
              ],
              "label": "AK-47",
              "type": "entity"
            }
          }
        ],
        "label": "James BOND",
        "type": "entity"
      }
    },
    {
      "link": {
        "type": "link",
        "label": "Mentions"
      },
      "properties": {
        "begin": 19,
        "end": 24,
        "ngram": "AK-47"
      },
      "target": {
        "properties": {
          "number": "singular",
          "gender": "neutral"
        },
        "links": [
          {
            "link": {
              "type": "link",
              "label": "Type"
            },
            "properties": {},
            "target": {
              "type": "entity",
              "label": "Generic"
            }
          }
        ],
        "label": "AK-47",
        "type": "entity"
      }
    }
  ]
}```

### Medical example

```bash
https POST "http://localhost:3000?locale=en&types=virus" body="H5N1" | jq
```

```json
{
  "type": "record",
  "id": "record:undefined__undefined",
  "date": "2017-07-11T22:27:51.438Z",
  "label": {},
  "indexed": "H5N1",
  "properties": {},
  "links": [
    {
      "link": {
        "type": "link",
        "id": "link:mention",
        "label": "Mentions",
        "description": "Mention in a document",
        "aliases": [
          "mentioned in",
          "has a mention",
          "is mentioned",
          "are mentioned"
        ]
      },
      "properties": {
        "ngram": "H5N1",
        "score": 1,
        "sentence": 1,
        "word": 0,
        "begin": 0,
        "end": 4
      },
      "target": {
        "properties": {
          "category": "species"
        },
        "links": [
          {
            "link": {
              "type": "link",
              "id": "link:instanceof",
              "label": "Type",
              "plural": "Types",
              "description": "Of type",
              "aliases": [
                "of type"
              ]
            },
            "properties": {},
            "target": {
              "type": "entity",
              "id": "entity:virus",
              "label": "Virus",
              "plural": "Viruses",
              "description": "Virus",
              "aliases": [
                "virus",
                "viruses"
              ]
            }
          }
        ],
        "id": "entity:virus__influenza-a-virus-h5n1",
        "label": "Influenza A (H5N1)",
        "description": "Influenza A virus (subtype H5N1)",
        "aliases": [
          "H5N1",
          "H5N1 flu",
          "Influenza A H5N1",
          "Influenza A (H5N1)",
          "Influenza A subtype H5N1",
          "Influenza A (subtype H5N1)",
          "Influenza A (H5N1 subtype)"
        ],
        "type": "entity"
      }
    }
  ]
}
```

### GDF

```bash
curl -X POST "http://localhost:3000?locale=en&types=animal,virus&format=gdf" -d "the monkey has ebola"
```

```csv
nodedef>id VARCHAR,label VARCHAR
entity:animal__monkey,Monkey
entity:virus__ebolavirus,Ebolavirus
edgedef>id VARCHAR,source VARCHAR,target VARCHAR
```

### Graphson

```bash
curl -X POST "http://localhost:3000?locale=en&types=animal,virus&format=graphson" -d "the monkey has ebola"
```

```json
{
  "graph": {
    "mode": "NORMAL",
    "vertices": [
      {
        "_id": "entity:animal__monkey",
        "name": "Monkey",
        "_type": "vertex"
      },
      {
        "_id": "entity:virus__ebolavirus",
        "name": "Ebola",
        "_type": "vertex"
      }
    ],
    "edges": []
  }
}
```

## Deployment

To start the service locally: `npm run start`.
To deploy on Now: `npm run deploy`.

---
_Source: https://npm.io/package/@datagica/datanote-service-file2doc · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._