# mirror_farmos

> Tools aimed at creating and keeping a farmOS mirror up to date, currently focused on mongoDB hosting. There are also extensive tools for data dereferencing, consolidation and simplification.

Latest version **0.6.2** (published 2023-06-02) · GPL-3.0-or-later license · 0 weekly downloads

## Install

```sh
npm install mirror_farmos
pnpm add mirror_farmos
yarn add mirror_farmos
bun add mirror_farmos
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.6.2 |
| Published | 2023-06-02 |
| First published | 2023-06-01 |
| Weekly downloads | 0 |
| License | GPL-3.0-or-later |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 7 |
| Unpacked size | 206.1 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | Octavio M. Duarte |
| Maintainers | octavio_duarte |
| Keywords | farmOS, mongodb, drupal |

## Links

- npm: https://www.npmjs.com/package/mirror_farmos
- npm.io page: https://npm.io/package/mirror_farmos

## Dependencies (7)

- [jsdoc](https://npm.io/package/jsdoc.md) ^4.0.0
- [dotenv](https://npm.io/package/dotenv.md) ^16.0.0
- [mongodb](https://npm.io/package/mongodb.md) ^4.4.1
- [cryptojs](https://npm.io/package/cryptojs.md) ^2.5.3
- [papaparse](https://npm.io/package/papaparse.md) ^5.3.2
- [node-fetch](https://npm.io/package/node-fetch.md) ^2.6.7
- [drupal-jsonapi-params](https://npm.io/package/drupal-jsonapi-params.md) ^2.0.0

## Alternatives

- [angular-pipes](https://npm.io/package/angular-pipes.md) — 5.6K weekly downloads
- [@ng-web-apis/midi](https://npm.io/package/@ng-web-apis/midi.md) — 2.6K weekly downloads
- [happn-3](https://npm.io/package/happn-3.md) — 1.6K weekly downloads
- [@opensip-cli/lang-go](https://npm.io/package/@opensip-cli/lang-go.md) — 1.2K weekly downloads
- [mongoose-typescript](https://npm.io/package/mongoose-typescript.md) — 85 weekly downloads

## Recent versions

- 0.6.2 (latest) — 2023-06-02
- 0.6.0 — 2023-06-01

## README

# Introduction

  The objective of the `mongo_mirror` is to centralize, transform and serve data from a huge number of discrete and private farmOS instances into a unified, fast, efficient, privacy respectful and versatile database.
  A mirror allows data operations that would be overly costly, time consuming and privacy harming if done instancywise.
  This library provides a very simple interface that will allow a mirror to be built from scratch by only providing the adequate security keys and some variables describing the expected behaviour. As the farmOS data structure is fiarly complex to be directly consumed by some applications, this library is also able to consolidate it and de reference it into structures that are as flat as possible without loosing data content or precission. This also makes data analysis easier.
  The `mongo_mirror` can optionally supervise the application of as many data pipelines as needed to be applied over this consolidated and simplified dataset, in order to deliver the exact data that each applicatino needs. For example, a first dataset might only have anonimized GPS positions to show in a map the farms

# How this library works

## Transformations Applied to Entities

* Entities are mirrored inside a tree structure before dereferencing.

* Each entity gets a field called `related`. It contains the entities mentioned in `relations`. Each one of this entities contains its own `related` field, the entities look the same when searched directly or as relationships of other related entities.

* Entities inside `related` are tagged with a `relationship_class` attribute. It informs under which `relations` attribute it was originally stored and allows to distinguis what kind of information should be obtained from it. 

* I traversed the FarmOs UI and identified the source (either an attribute on the main entity or inside a relationship) for each bit of data that is shown there for each type of entity. These are copied into the main entity with appropiate data names as first level object properties, obtained either from `attributes` on the original main entity or inside any of its related sub entities.


## Modules

### Nightly Build

Meant to grab information from FarmOS instances and centralize it into a MongoDB unique database.

#### Main Functions

* `copyFarmToMongo`: Provided a farm and a list of types, all entitities for each type are retrieved, tagged with the adequate `farmDomain` attribute and stored inside `raw_entities` collection.
* `copyManyFarmsToMongo`: Helps iterate through the whole list of farms.

### Dereference and Simplify

Meant to reformat the data into a consolidated, as flat as possible without loosing details planting centric database.
It assembles whole objects based on FarmOS references attributes so that each object contains directly all of its related information. 
It employs a set of parsers called `Instructions` to grab only the data attributes that are relevant from each entity and arrange them in the most possible meaningful way without making wrong structure assumptions. 

* `processOneFarm` retrieves all entities from `raw_entities` for a given farm, dereferences them, simplifies them and copies them into `simplified_entities`.


### Coffeeshop Aggregations

Not currently a library.

An aggregation grabs plantings from `simplified_entities`, attaches events (as they are reverse dependencies) and consolidates planting rows into `coffeeshop_full`. This is further simplified and stored as `coffeeshop_simplified`, which is the exact payload the coffeeshop frontend expects.

### Declarative Configuration


# Where to find the Summaries

## Main Branch 

* [Latest version of QA artifacts](https://gitlab.com/our-sci/software/mirror_farmos/-/jobs/artifacts/main/browse/summaries?job=general_summaries)

---
_Source: https://npm.io/package/mirror_farmos · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._