1.0.20240823 • Published 11 months ago

@ideditor/nsi-collector v1.0.20240823

Weekly downloads
-
License
ISC
Repository
github
Last release
11 months ago

npm version

nsi-collector

Scripts to collect names for the name-suggestion-index project.

Collecting names from the OSM planet

This takes a long time (~1-2h) and a lot of disk space (~75GB). It can be done occasionally by project maintainers.

Get the planet file

  • Download the planet
    • Mirrors are likely faster than the main repo
    • curl -L -o planet-latest.osm.pbf https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf

Filter and collect names

2 choices:

Use docker and run.py

  • Make sure your dockermachine has at least 2GB of RAM
  • Place the pbf of the planet osm file you wish to process in the same directory as input-planet.osm.pbf
  • Remove the node_modules directory if it exists from a former run (the script will remind you)
  • Run ./run.py
  • This will also run md5 of the input planet file and write it to last_run.md5 on success.
  • That way you can see if it's even worth a bunch of resources to run this script
  • Md5 hashes of pbf files are available: https://planet.openstreetmap.org/pbf/

Manually

  • Install osmium command-line tool (may only be available on some environments)
    • apt-get install osmium-tool or brew install osmium-tool or similar
  • Prefilter the planet file to only include named items with keys we are looking for:
    • osmium tags-filter planet-latest.osm.pbf -R name,brand,operator,network --overwrite -o filtered.osm.pbf
  • Run the collection script
    • This is complicated because node-osmium is available prebuilt only for older environments. It seems to work ok on Node 10.
    • node collect_osm.js /path/to/filtered.osm.pbf

Check in the collected names

  • git add . && git commit -m 'Collected common names from latest planet'

License

This project is available under the ISC License. See the LICENSE.md file for more details.

1.0.20240823

11 months ago

1.0.20230528

2 years ago

1.0.20230522

2 years ago

1.0.20230513

2 years ago

1.0.20230510

2 years ago

1.0.20230619

2 years ago

1.0.20230409

2 years ago

1.0.20230317

2 years ago

1.0.20230424

2 years ago

1.0.20230304

2 years ago

1.0.20230428

2 years ago

1.0.20230429

2 years ago

1.0.20230225

2 years ago

1.0.20230205

2 years ago

1.0.20230129

2 years ago

1.0.20230217

2 years ago

1.0.20221229

3 years ago

1.0.20221112

3 years ago

1.0.20221119

3 years ago

1.0.20230116

2 years ago

1.0.20221204

3 years ago

1.0.20221127

3 years ago

1.0.20230115

2 years ago

1.0.20230108

3 years ago

1.0.20221219

3 years ago

1.0.20221217

3 years ago

1.0.20221106

3 years ago

1.0.20221001

3 years ago

1.0.20221008

3 years ago

1.0.20221016

3 years ago

1.0.20221025

3 years ago

1.0.20220910

3 years ago

1.0.20220924

3 years ago

1.0.20220903

3 years ago

1.0.20220823

3 years ago

1.0.20220829

3 years ago

1.0.20220619

3 years ago

1.0.20220716

3 years ago

1.0.20220614

3 years ago

1.0.20220813

3 years ago

1.0.20220612

3 years ago

1.0.20220704

3 years ago

1.0.20220801

3 years ago

1.0.20220625

3 years ago

1.0.20220709

3 years ago

1.0.20220522

3 years ago

1.0.20220508

3 years ago

1.0.20220528

3 years ago

1.0.20220416

3 years ago

1.0.20220423

3 years ago

1.0.20220409

3 years ago

1.0.20220403

3 years ago

1.0.20220430

3 years ago

1.0.20220306

3 years ago

1.0.20220313

3 years ago

1.0.20220219

3 years ago

1.0.20220109

4 years ago

1.0.20220205

3 years ago

1.0.20220129

3 years ago

1.0.20211226

4 years ago

1.0.20220115

3 years ago

1.0.20220213

3 years ago

1.0.20220122

3 years ago

1.0.20220101

4 years ago

1.0.20211120

4 years ago

1.0.20211204

4 years ago

1.0.20211127

4 years ago

1.0.20211106

4 years ago

1.0.20211211

4 years ago

1.0.20211113

4 years ago

1.0.20211218

4 years ago

1.0.20211024

4 years ago

1.0.20211016

4 years ago

1.0.20211003

4 years ago

1.0.20210919

4 years ago