1.0.4 • Published 6 years ago

tracemaster-builder v1.0.4

Weekly downloads
-
License
-
Repository
-
Last release
6 years ago

Prerequisites

Building the docker image

  • docker-compose -f ./docker/docker-compose.yml build

Run conversion:

Using the local file system

  • Create a source directory, and add some files: mkdir ./source && curl http://download.lodlaundromat.org/b7f7f37bc2573a2e70901b63940fa7ab > ./source/somedata.nt.gz && curl http://download.lodlaundromat.org/9ebcd9dca02d157749a3a756c006637f > ./source/someMoreData.nt.gz
  • Create the hdt directory where the HDT is stored: mkdir ./hdt
  • Run the conversion script: docker-compose -f ./docker/docker-compose.yml up
  • On success, the created HDT file is accessible in the ./hdt directory

Using AWS S3

  • Add this clause to your docker-compose file:
environment:
  - AWS_ACCESS_KEY_ID=<key>
  - AWS_SECRET_ACCESS_KEY=<access-key>
  - S3_BUCKETNAME_RDF=s3://....
  - S3_BUCKETNAME_HDT=s3://....
  • Run the conversion script: docker-compose -f <dockerfile> up

Local development:

Run the building process using docker-compose -f ./docker/docker-compose-dev.yml up

Update procedure:

  • npm run <patch|minor|major> && npm publish
  • Wait for docker container to be build
  • Ssh to tracemaster ETL container
  • docker pull triply/tracemaster-builder