# latona-datacatalog > Latona addon for generating model-based human-readable data catalog pages (md, html, Confluence Cloud) Latest version **0.1.5** (published 2021-06-05) · Apache 2.0 license · 0 weekly downloads ## Install ```sh npm install latona-datacatalog pnpm add latona-datacatalog yarn add latona-datacatalog bun add latona-datacatalog ``` Provides the command `dc2confcloud`. ## Health **Score 15/100 (F)** — status: abandoned. Positive: no vulnerabilities. Warnings: low downloads; no types; no esm support; pre 1.0. Negative: abandoned; low maintenance score. ## Facts | | | |---|---| | Version | 0.1.5 | | Published | 2021-06-05 | | First published | 2021-04-04 | | Weekly downloads | 0 | | License | Apache 2.0 | | TypeScript types | none | | Module format | CommonJS | | Node | >=10.0.0 | | Dependencies | 10 | | Unpacked size | 99.3 KB | | Known vulnerabilities | 0 (+25 in 2 direct dependencies) | | Install scripts | no | | GitHub stars | 0 | | Maintainers | latona_team | | Keywords | DataArt, Latona, Data Catalog | ## Links - npm: https://www.npmjs.com/package/latona-datacatalog - Repository: https://github.com/DataArt-Latona/latona-datacatalog - Homepage: https://github.com/DataArt-Latona/latona-datacatalog#readme - Issues: https://github.com/DataArt-Latona/latona-datacatalog/issues - npm.io page: https://npm.io/package/latona-datacatalog ## Dependencies (10) - [qs](https://npm.io/package/qs.md) ^6.9.4 - [ajv](https://npm.io/package/ajv.md) ^6.12.2 - [axios](https://npm.io/package/axios.md) ^0.21.1 - [chalk](https://npm.io/package/chalk.md) ^4.0.0 - [lodash](https://npm.io/package/lodash.md) ^4.17.21 - [p-limit](https://npm.io/package/p-limit.md) ^3.0.2 - [fs-extra](https://npm.io/package/fs-extra.md) ^7.0.1 - [mustache](https://npm.io/package/mustache.md) ^3.2.1 - [commander](https://npm.io/package/commander.md) ^5.1.0 - [markdown-it](https://npm.io/package/markdown-it.md) ^10.0.0 ## Alternatives - [cli-color](https://npm.io/package/cli-color.md) — 3.4M weekly downloads - [log](https://npm.io/package/log.md) — 1.3M weekly downloads - [logstash-client](https://npm.io/package/logstash-client.md) — 4.5K weekly downloads - [@nocobase/plugin-logger](https://npm.io/package/@nocobase/plugin-logger.md) — 2.0K weekly downloads - [child-process-debug](https://npm.io/package/child-process-debug.md) — 695 weekly downloads ## Recent versions - 0.1.5 (latest) — 2021-06-05 - 0.1.4 — 2021-06-05 - 0.1.3 — 2021-04-04 ## README # Latona - Data Catalog Latona addon for generating model-based human-readable data catalog pages (md, html, Confluence Cloud) - [Disclaimer](#disclaimer) - [Usage](#usage) - [Quick start](#quick-start) - [Addon options](#addon-options) - [Upload pages to Confluence Cloud](#upload-pages-to-confluence-cloud) - [Contributing](#contributing) - [License](#license) ## Disclaimer This is _NOT_ a fully-fledged data catalog implementation like [Apache Atlas](https://atlas.apache.org/). This is just an addon to [Latona](https://github.com/DataArt-Latona/latona) code generator. This addon renders a set of pages to simplify digesting metadata in a user-friendly way. Nothing more than that. We used this tool on a number occasions, usually when it is too early to start thinking about heavy data cataloging tools, but presenting some collected metadata is beneficial. ## Usage ### Quick start 1. Install Latona - see [latona documentation](https://github.com/DataArt-Latona/latona#install) 2. Scaffold model and project - see [latona documentation](https://github.com/DataArt-Latona/latona#use) 3. Add `latona-datacatalog` addon to `package.json`: ```sh npm i latona-datacatalog ``` 4. Add `latona-datacatalog` to your latona project. To use the default configuration add this snippet to the `addons` array: ```json { "moduleName": "latona-datacatalog", "options": { "renderFormat": "md" } } ``` 5. Add tables to your model. See our [test model](./test/mocks/addon.spec/model.json) for examples. Data catalog expects `tables` array to be at the root of the model. 6. Add extra collections for other catalogs. Default configuration will look for `collections.businessProcesses` for business process catalog, and for `collections.applications` for applications catalog. Nothing bad will happen if these are missing - respective catalogs will be empty in this case. 7. Add relationship types to your model. They should land in `collections.relationships` by default. Each relationship type should be structured as follows (titles are used to group relationship links at the catalog item's page): ```json { "id": "string", "title_forward": "string", "title_backward": "string" } ``` 8. Add relationship records. The default place for them is at `collections.relationshipTypes`. `typeId` should be one of defined at the previous steps, `source` is the source (upstream) object's slug, `target` is the target (downstream) object's slug. Typical relationship looks like this: ```json { "typeId": "fk", "source": "[orders].[dbo].[FactOrderItem]", "target": "[orders].[dbo].[DimClient]" } ``` 9. Add custom markdown content to the `./content` folder (use item's slug as a file name). 10. Render as described in the [Latona's guide](https://github.com/DataArt-Latona/latona#use). ### Addon options Please refer for more information to these source files: - [Default configuration](./src/addonInternals/latona-datacatalog-options-defaults.json) - [Options schema](./src/addonInternals/latona-datacatalog-options-schema.json) - [Test model](./test/mocks/addon.spec/model.json) | Option | Type | Required | Description | | ------------------------------------------------------------------------------- | ----------------------------------------- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | paths | object | yes | | | paths.outPath | string | yes | output path (default: `./output`) | | paths.customContentPath | string | yes | path to the folder with custom markdown files (use slug for names and custom markdown will be injected) | | renderFormat | enum ["md", "html", "conf"] | yes | | | pageTreeFileName | string | yes | filename for the JSON file with the list of all generated pages | | conf | object | | Confluence-specific configuration | | conf.createNotesPage | boolean | | `true` by default | | conf.notesSlugSuffix | string | yes (only if parent object is present) | Slug suffix for "notes" pages - empty pages that may be editted directly at Confluence. Notes pages are injected into catalog pages via macro. | | conf.notesTitleSuffix | string | yes (only if parent object is present) | Suffix for "notes" page titles. | | conf.modelConfig | object | yes (only if parent object is present) | | | conf.modelConfig.spaceKey | string | yes (only if parent object is present) | Confluence space key | | catalogs | object | yes (only if parent object is present) | Catalogs configuration (there may be many) | | catalogs.<catalogName> | object | yes | Catalog configuration object. Only latin letters. | | catalogs.<catalogName>.collection | string | yes | path to array of catalog items (from the model root) | | catalogs.<catalogName>.title | string | yes | catalog title | | catalogs.<catalogName>.slug | string | yes | catalog slug (url-friendly string) | | catalogs.<catalogName>.sectionBy | object | | sectioning configuration (at least `first` level should be described, only two levels are supported) | | catalogs.<catalogName>.sectionBy.first | object | yes (only if parent object is present) | first level of grouping | | catalogs.<catalogName>.sectionBy.first.properties[] | array | yes (only if parent object is present) | properties to group by | | catalogs.<catalogName>.sectionBy.first.properties[].name | string | yes | property name (for internal use only) | | catalogs.<catalogName>.sectionBy.first.properties[].path | string | yes | property path (relative to catalog item) | | catalogs.<catalogName>.sectionBy.first.properties[].title | string | yes | property title | | catalogs.<catalogName>.sectionBy.first.properties[].show | boolean | | show property | | catalogs.<catalogName>.sectionBy.first.properties[].showInLinkTable | boolean | | show property on the parent page if `"inParentAs": "linkTable"` | | catalogs.<catalogName>.sectionBy.first.inParentAs | enum ["linkTable", "linkList", "section"] | yes (only if parent object is present) | sets how to show sections on the parent page (catalog root page for the first level of grouping) | | catalogs.<catalogName>.sectionBy.first.tableLinkProperty | string | | sets which property should be used for linking to the section details page | | catalogs.<catalogName>.sectionBy.first.titleTemplate | string | yes (only if parent object is present) | section title template (variable expansion is supported) | | catalogs.<catalogName>.sectionBy.first.slugTemplate | string | yes (only if parent object is present) | section slug template (variable expansion is supported) | | catalogs.<catalogName>.sectionBy.second | object | | see `catalogs.<catalogName>.sectionBy.first` | | catalogs.<catalogName>.details | object | yes | see `catalogs.<catalogName>.sectionBy.first` | | catalogs.<catalogName>.detailsCollections | object | | describes inner collections for the catalog's item (e.g.: if database table is a catalog item, then table fields is the inner collection) | | catalogs.<catalogName>.detailsCollections.<name> | object | yes (only if parent object is present) | Collection configuration object. Only latin letters. | | catalogs.<catalogName>.detailsCollections.<name>.collection | string | yes (only if parent object is present) | path to array of collection items (relative to item) | | catalogs.<catalogName>.detailsCollections.<name>.properties[] | array | yes (only if parent object is present) | collection properties | | catalogs.<catalogName>.detailsCollections.<name>.properties[].name | string | yes (only if parent object is present) | property name (for internal use only) | | catalogs.<catalogName>.detailsCollections.<name>.properties[].path | string | yes (only if parent object is present) | property path (relative to collection item) | | catalogs.<catalogName>.detailsCollections.<name>.properties[].title | string | yes (only if parent object is present) | property title | | catalogs.<catalogName>.detailsCollections.<name>.heading | string | yes (only if parent object is present) | section's title | | relationshipsCollection | string | yes | path to array of relationships (relative to `collections` object within the latona model) | | relationshipTypesCollection | string | yes | path to array of relationship types (relative to `collections` object within the latona model) | | dropDefaultCatalogs | boolean | | `false` by default, removes default catalogs configuration if `true` | ### Upload pages to Confluence Cloud The `latona-datacatalog` package also includes `dc2confcloud` utility which enables automated upload of pages to your Confluence Cloud space. 1. Install the package globally to use this tool: ``` npm i -g latona-datacatalog ``` 2. Create the configuration json: ```json { "confluence": { "baseUrl": ".atlassian.net", "spaceKey": "" }, "contentFolder": "./path/to/generated/conf/output", "notesSuffix": " - Notes", // see addon options "pageTree": "___page_tree___.json", // see addon options // specify parent pages for each "root" catalog page "parents": [ { "slug": ".datacatalog", "confPageId": 123456789 // you need to create this page manually }, { "slug": ".datacatalog-notes", "confPageId": 123456789 // you need to create this page manually }, { "slug": ".bpcatalog", "confPageId": 123456789 // you need to create this page manually }, { "slug": ".bpcatalog-notes", "confPageId": 123456789 // you need to create this page manually }, { "slug": ".appcatalog", "confPageId": 123456789 // you need to create this page manually }, { "slug": ".appcatalog-notes", "confPageId": 123456789 // you need to create this page manually } ] } ``` 3. Run the tool: ``` dc2confcloud -u -t -c <./path/to/dc2confcloud.json> ``` Usage details: ``` Usage: dc2confcloud [options] Options: -u, --user Confluence user name (email) -t, --token Confluence access token -c, --config path to dc2confcloud's configuration json (default: "./dc2confcloud.json") -h, --help display help for command ``` ## Contributing Please read our [Latona's contribution guidelines](https://github.com/DataArt-Latona/latona/blob/master/docs/dev/CONTRIBUTING.md) for details on our development approach, and the process for submitting pull requests to us. All contributors should comply with our [Latona's Code of Conduct](https://github.com/DataArt-Latona/latona/blob/master/CODE_OF_CONDUCT.md) ## License **Latona** is copyright (c) 2019-present DataArt (www.dataart.com) and all contributors and licensed under the Apache License, Version 2.0. See the [LICENSE](./LICENSE) file for more details. --- _Source: https://npm.io/package/latona-datacatalog · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._