@swiftstar/gtfs NPM

Node-GTFS

node-GTFS loads transit data in GTFS format, unzips it and stores it to a MongoDB database. In addition, this library provides some methods to query for agencies, routes, stops, times, fares and calendars. It also offers spatial queries to find nearby stops, routes and agencies. In addition, it can convert stops and shapes to geoJSON.

Breaking changes in version 1.0.0

As of version 1.0.0, all node-gtfs methods have changed to accept a query object instead of individual arguments. This allows for all fields of all GTFS files to be queried using this library. Most method names have been changed to be more general and more specific methods have been removed. For example, getRoutes now replaces getRoutesByAgency, getRoutesById, getRoutesByDistance and getRoutesByStop.

// Old method with individual arguments, no longer supported in `node-gtfs` 1.0.0
gtfs.getRoutesByStop(agency_key, stop_id)
.then(routes => {
  // do something with the array of `routes`
})

// Query in `node-gtfs` version 1.0.0
gtfs.getRoutes({
  agency_key: 'caltrain',
  stop_id: '123'
})
.then(routes => {
  // do something with the array of `routes`
})

Additionally, as of version 0.11.0, node-gtfs methods don't support callbacks. Use promises instead:

gtfs.getAgencies()
.then(agencies => {
  // do something with the array of `agencies`
})
.catch(err => {
  // handle errors here
});

Or, you use async/await:

const myAwesomeFunction = async () => {
  try {
    const agencies = await gtfs.getAgencies();
  } catch (error) {
    // handle errors here
  }
}

This library has two parts: the GTFS import script and the query methods.

Example Application

The GTFS-to-HTML app uses node-gtfs for downloading, importing and querying GTFS data. It provides a good example of how to use this library.

The GTFS-to-geojson app creates geoJSON files for transit routes for use in mapping. It uses node-gtfs for downloading, importing and querying GTFS data. It provides a good example of how to use this library.

Installation

Install node-gtfs directly from npm:

npm install gtfs -g

Note: If you are writing a project that uses node-gtfs then you'll want to include mongoose as a dependency with the same version that is specified in node-gtfs package.json. Otherwise, two versions will get installed and you'll won't be able to query data that was imported.

Command-line example

gtfs-import [--configPath /path/to/your/custom-config.json] [--skipDelete]

Code example

const gtfs = require('gtfs');
const config = require('./config.json');

gtfs.import(config)
.then(() => {
  console.log('Import Successful');
})
.catch(err => {
  console.error(err);
});

Configuration

Copy config-sample.json to config.json and then add your projects configuration to config.json.

cp config-sample.json config.json

option	type	description
`agencies`	array	An array of GTFS files to be imported.
`mongoUrl`	string	The URL of the MongoDB database to import to.
`verbose`	boolean	Whether or not to print output to the console.
`skipDelete`	boolean	Whether or not to skip deleting existing data from the database.

Agencies

Specify the GTFS files to be imported in an agencies array. GTFS files can be imported via a url or a local path.

Each file needs an agency_key, a short name you create that is specific to that GTFS file. For GTFS files that contain more than one agency, you only need to list each GTFS file once in the agencies array, not once per agency that it contains.

To find an agency's GTFS file, visit transitfeeds.com. You can use the URL from the agency's website or you can use a URL generated from the transitfeeds.com API along with your API token.

Specify a download URL:

{
  "agencies": [
    {
      "agency_key": "county-connection",
      "url": "http://countyconnection.com/GTFS/google_transit.zip"
    }
  ]
}

Specify a path to a zipped GTFS file:

{
  "agencies": [
    {
      "agency_key": "myAgency",
      "path": "/path/to/the/gtfs.zip"
    }
  ]
}

Specify a path to an unzipped GTFS file:

{
  "agencies": [
    {
      "agency_key": "myAgency",
      "path": "/path/to/the/unzipped/gtfs/"
    }
  ]
}

Exclude files - if you don't want all GTFS files to be imported, you can specify an array of files to exclude.

{
  "agencies": [
    {
      "agency_key": "myAgency",
      "path": "/path/to/the/unzipped/gtfs/",
      "exclude": [
        "shapes",
        "stops"
      ]
    }
  ]
}

Optionally specify a proj4 projection string to correct poorly formed coordinates in the GTFS file

{
  "agencies": [
    {
      "agency_key": "myAgency",
      "path": "/path/to/the/unzipped/gtfs/",
      "proj": "+proj=lcc +lat_1=46.8 +lat_0=46.8 +lon_0=0 +k_0=0.99987742 +x_0=600000 +y_0=2200000 +a=6378249.2 +b=6356515 +towgs84=-168,-60,320,0,0,0,0 +pm=paris +units=m +no_defs"
    }
  ]
}

MongoDB URI

Add the MongoDB URI to config.json with the key mongoUrl. Running locally, you could use something like mongodb://localhost:27017/gtfs.

{
  "mongoUrl": "mongodb://localhost:27017/gtfs",
  "agencies": [
    {
      "agency_key": "myAgency",
      "path": "/path/to/the/unzipped/gtfs/"
    }
  ]
}

Logging

If you don't want the import script to print any output to the console, you can set verbose to false. Defaults to true.

{
  "mongoUrl": "mongodb://localhost:27017/gtfs",
  "agencies": [
    {
      "agency_key": "localAgency",
      "path": ""/path/to/the/unzipped/gtfs/"
    }
  ],
  "verbose": false
}

Deleting existing data

If you don't want the import script to delete all existing data from the database with the same agency_key, you can set skipDelete to true. Defaults to false.

{
  "mongoUrl": "mongodb://localhost:27017/gtfs",
  "agencies": [
    {
      "agency_key": "localAgency",
      "path": ""/path/to/the/unzipped/gtfs/"
    }
  ],
  "skipDelete": true
}

`gtfs-import` Script

The gtfs-import script reads from a JSON configuration file and imports the GTFS files specified to a MongoDB database. Read more on setting up your configuration file.

Make sure MongoDB is running

If you want to run this locally, make sure MongoDB in installed and running.

mongod

Run the `gtfs-import` script from Command-line

gtfs-import

By default, it will look for a config.json file in the project root. To specify a different path for the configuration file:

gtfs-import --configPath /path/to/your/custom-config.json

Command Line options

Skip Delete

By default, the import script will delete any existing data with the same agency_key from your database. If you don't want this to happen, pass the --skipDelete flag

gtfs-import --skipDelete

Specify path to config JSON file

You can specify the path to a config file to be used by the import script.

gtfs-import --configPath /path/to/your/custom-config.json

Show help

Show all command line options

gtfs-import --help

Use GTFS import script in code

Use gtfs.import() in your code to run an import of a GTFS file specified in a config.json file.

const gtfs = require('gtfs');
const mongoose = require('mongoose');
const config = require('config.json');

mongoose.Promise = global.Promise;
mongoose.connect(config.mongoUrl, {useMongoClient: true});

gtfs.import(config)
.then(() => {
  console.log('Import Successful');
})
.catch(err => {
  console.error(err);
});

Configuration can be a JSON object in your code

const gtfs = require('gtfs');
const mongoose = require('mongoose');
const config = {
  mongoUrl: 'mongodb://localhost:27017/gtfs',
  agencies: [
    {
      agency_key: 'county-connection',
      url: 'http://countyconnection.com/GTFS/google_transit.zip',
      exclude: [
        'shapes'
      ]
    }
  ]
};

mongoose.Promise = global.Promise;
mongoose.connect(config.mongoUrl, {useMongoClient: true});

gtfs.import(config)
.then(() => {
  console.log('Import Successful');
})
.catch(err => {
  console.error(err);
});

Query Methods

This library includes many methods you can use in your project to query GTFS data. These methods return promises.

For example, to get a list of all agencies within 5 miles of a specific point:

gtfs.getAgencies({
  within: {
    lat: 37.7749,
    lon: -122.4194,
    radius: 5
  }
})
.then(agencies => {
  // Do something with the array of `agencies`
})
.catch(err => {
  // Be sure to handle errors here
});

Setup

Include this library.

var gtfs = require('gtfs');

Connect to mongo via mongoose.

const mongoose = require('mongoose');
mongoose.Promise = global.Promise;
mongoose.connect('YOUR-MONGODB-URI', {useMongoClient: true});

If you are running locally, your MongoDB uri might be something like:

mongodb://localhost:27017/gtfs

You probably want to use the same value used in your configuration JSON file for importing GTFS.

Query Methods

Once you have included the library and connected to your MongoDB database you can use the following methods.

gtfs.getAgencies()

Queries agencies and returns a promise. The result of the promise is an array of agencies.

// Get all agencies
gtfs.getAgencies()
.then(agencies => {

});

// Get all agencies within a a `radius` of the `lat`, `lon` specified.
// `radius` is optional and in miles. Default: 25 miles.
gtfs.getAgencies({
  within: {
    lat: 37.7749,
    lon: -122.4194,
    radius: 5
  }
})
.then(agencies => {

});

// Get a specific agency
gtfs.getAgencies({
  agency_key: 'caltrain'
})
.then(agencies => {

});

// Get a specific agency by `agency_name`
gtfs.getAgencies({
  agency_name: 'Caltrain'
})
.then(agencies => {

});

// Get all agencies in a specific timezone
gtfs.getAgencies({
  agency_timezone: 'America/Los_Angeles'
})
.then(agencies => {

});

gtfs.getRoutes()

Queries routes and returns a promise. The result of the promise is an array of routes.

// Get all routes for an agency
gtfs.getRoutes({
  agency_key: 'caltrain'
})
.then(routes => {

});

// Get a specific route
gtfs.getRoutes({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR'
})
.then(routes => {

});

// Get a few routes
const routeIds = [
  'Bu-16APR',
  'Lo-16APR'
]
gtfs.getRoutes({
  agency_key: 'caltrain',
  route_id: {
    $in: routeIds
  }
})
.then(routes => {

});

// Get all routes within a radius of the `lat`, `lon` specified.
// `radius` is optional and in miles. Default: 1 mile.
gtfs.getRoutes({
  within: {
    lat: 37.7749,
    lon: -122.4194,
    radius: 5
  }
})
.then(routes => {

});


// Get routes that serve a specific stop.
gtfs.getRoutes({
  agency_key: 'caltrain',
  stop_id: '70011'
})
.then(routes => {

});

gtfs.getStops()

Queries stops and returns a promise. The result of the promise is an array of stops.

// Get all stops for an agency
gtfs.getStops({
  agency_key: 'caltrain'
})
.then(stops => {

});

// Get a specific stop by stop_id
gtfs.getStops({
  agency_key: 'caltrain',
  stop_id: '70011'
})
.then(stops => {

});

// Get a few stops
const stopIds = [
  '70011',
  '70012'
];
gtfs.getStops({
  agency_key: 'caltrain',
  stop_id: {
    $in: stopIds
  }
})
.then(stops => {

});

// Get all stops for a specific route and direction
gtfs.getStops({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 1
})
.then(stops => {

});

// Get all stops within a `radius` of the `lat`, `lon` specified.
// `radius` is optional and in miles. Default: 1 mile.
gtfs.getStops({
  within: {
    lat: 37.7749,
    lon: -122.4194,
    radius: 5
  }
})
.then(stops => {

});

gtfs.getStopsAsGeoJSON()

Queries stops and returns a promise. The result of the promise is an geoJSON object of stops. All valid queries for gtfs.getStops() work for gtfs.getStopsAsGeoJSON().

// Get all stops for an agency as geoJSON
gtfs.getStopsAsGeoJSON({
  agency_key: 'caltrain'
})
.then(stops => {

});

// Get all stops for a specific route and direction as geoJSON
gtfs.getStopsAsGeoJSON({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 1
})
.then(geojson => {

});

stop_ids is optional and can be a single stop_id or an array of stop_ids.

gtfs.getStoptimes()

Queries stop_times and returns a promise. The result of the promise is an array of stop_times. agency_key and stop_id are required.

// Get all stoptimes for a specific stop
gtfs.getStoptimes({
  agency_key: 'caltrain',
  stop_id: '70011'
})
.then(stoptimes => {

});

// Get all stoptimes for a specific stop, route and direction
gtfs.getStoptimes({
  agency_key: 'caltrain',
  stop_id: '70011',
  route_id: 'Lo-16APR',
  direction_id: 0
})
.then(stoptimes => {

});

// Get all stoptimes for a specific stop and service_id
gtfs.getStoptimes({
  agency_key: 'caltrain',
  stop_id: '70011',
  service_id: 'CT-16APR-Caltrain-Weekday-01'
})
.then(stoptimes => {

});

gtfs.getTrips()

Queries trips and returns a promise. The result of the promise is an array of trips.

// Get trips for a specific route and direction
gtfs.getTrips({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 0
})
.then(trips => {

});

// Get trips for a specific route and direction limited by service_ids
const serviceIds = [
  'CT-16APR-Caltrain-Saturday-02',
  'CT-16APR-Caltrain-Sunday-02'
];
gtfs.getTrips({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 0,
  service_id: {
    $in: serviceIds
  }
})
.then(trips => {

});

gtfs.getDirectionsByRoute()

Queries trips and returns a promise. The result of the promise is an array of direction_ids. Useful to determine if a route has two directions or just one. agency_key and route_id are required.

// Find all directions for a specific route
gtfs.getDirectionsByRoute({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR'
})
.then(directions => {

});

// Find all directions for a specific route and service_id
gtfs.getDirectionsByRoute({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  service_id: 'CT-16APR-Caltrain-Sunday-02'
})
.then(directions => {

});

Example result:

[
  {
    route_id: 'Bu-16APR',
    trip_headsign: 'SAN FRANCISCO STATION',
    direction_id: 0
  },
  {
    route_id: 'Bu-16APR',
    trip_headsign: 'DIRIDON STATION',
    direction_id: 1
  },
  { route_id: 'Bu-16APR',
    trip_headsign: 'TAMIEN STATION',
    direction_id: 1
  }
];

gtfs.getShapes()

Queries shapes and returns a promise. The result of the promise is an array of shapes sorted by shape_pt_sequence.

// Get all shapes for an agency
gtfs.getShapes({
  agency_key: 'caltrain'
})
.then(shapes => {

});

// Get all shapes for a specific route and direction
gtfs.getShapes({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 0
})
.then(shapes => {

});

// Get all shapes for a specific trip_id
gtfs.getShapes({
  agency_key: 'caltrain',
  trip_id: '37a'
})
.then(shapes => {

});

// Get all shapes for a few trip_ids
const tripIds = [
  '37a',
  '39a'
];
gtfs.getShapes({
  agency_key: 'caltrain',
  trip_id: {
    $in: tripIds
  }
})
.then(shapes => {

});

gtfs.getShapesAsGeoJSON()

Queries shapes and returns a promise. The result of the promise is an geoJSON object of shapes. All valid queries for gtfs.getShapes() work for gtfs.getShapesAsGeoJSON().

Returns geoJSON of shapes for the agency_key specified.

// Get geoJSON of all routes in an agency
gtfs.getShapesAsGeoJSON({
  agency_key: 'caltrain'
})
.then(geojson => {

});

// Get geoJSON of a specific route in an agency
gtfs.getShapesAsGeoJSON({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR'
})
.then(geojson => {

});

// Get geoJSON of a specific route and direction in an agency
gtfs.getShapesAsGeoJSON({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR',
  direction_id: 0
})
.then(geojson => {

});

gtfs.getCalendars()

Queries calendars and returns a promise. The result of the promise is an array of calendars.

// Get all calendars for an agency
gtfs.getCalendars({
  agency_key: 'caltrain'
})
.then(calendars => {

});

// Get calendars for an agency between two dates that apply on a Tuesday
gtfs.getCalendars({
  agency_key: 'caltrain',
  start_date: {$lt: 20160405},
  end_date: {$gte: 20160404},
  tuesday: 1
})
.then(calendars => {

});

// Get calendars for an agency by service_ids
gtfs.getCalendars({
  agency_key: 'caltrain',
  service_id: 'CT-16APR-Caltrain-Sunday-02'
})
.then(calendars => {

});

// Get calendars for an specific route
gtfs.getCalendars({
  agency_key: 'caltrain',
  route_id: 'TaSj-16APR'
})
.then(calendars => {

});

// Get calendars for several routes
gtfs.getCalendars({
  agency_key: 'caltrain',
  route_id: {$in: ['TaSj-16APR', 'Lo-16APR']}
})
.then(calendars => {

});

gtfs.getFeedInfo()

Queries feed_info and returns a promise. The result of the promise is an array of feed_infos.

// Get feed_info for a specified agency
gtfs.getFeedInfo({
  agency_key: 'caltrain'
})
.then(feedInfo => {

});

gtfs.getFareRules()

Queries fare_rules and returns a promise. The result of the promise is an array of fare_rules.

// Get fare_rules for a route
gtfs.getFareRules({
  agency_key: 'caltrain',
  route_id: 'Lo-16APR'
})
.then(fareRules => {

});

gtfs.getFrequencies()

Queries frequencies and returns a promise. The result of the promise is an array of frequencies.

// Get frequencies for a trip
gtfs.getFrequencies({
  agency_key: 'caltrain',
  trip_id: '1234'
})
.then(frequencies => {

});

gtfs.getStopAttributes()

Queries stop_attributes and returns a promise. The result of the promise is an array of stop_attributes. These are from the non-standard stop_attributes.txt file.

// Get stop attributes for a few stops
const stopIds = [
  '70011',
  '70012'
];
gtfs.getStopAttributes({
  agency_key: 'caltrain',
  stop_id: {
    $in: stopIds
  }
})
.then(stopAttributes => {

});

gtfs.getTimetables()

Queries timetables and returns a promise. The result of the promise is an array of timetables. These are from the non-standard timetables.txt file.

// Get all timetables for an agency
gtfs.getTimetables({
  agency_key: 'caltrain'
})
.then(timetables => {

});

// Get a specific timetable
gtfs.getTimetables({
  agency_key: 'caltrain',
  timetable_id: '1'
})
.then(timetables => {

});

gtfs.getTimetableStopOrders()

Queries timetable_stop_orders and returns a promise. The result of the promise is an array of timetable_stop_orders. These are from the non-standard timetable_stop_order.txt file.

// Get timetable_stop_orders for a specific timetable
gtfs.getTimetableStopOrders({
  agency_key: 'caltrain',
  timetable_id: '1'
})
.then(TimetableStopOrders => {

});

Timetable Pages

Queries timetable_pages and returns a promise. The result of the promise is an array of timetable_pages. These are from the non-standard timetable_pages.txt file.

// Get all timetable_pages for an agency
gtfs.getTimetablePages({
  agency_key: 'caltrain'
})
.then(timetablePages => {

});

// Get a specific timetable_page
gtfs.getTimetablePages({
  agency_key: 'caltrain',
  timetable_page_id: '2'
})
.then(timetablePages => {

});

Contributing

Pull requests are welcome, as is feedback and reporting issues.

Tests

To run tests:

npm test

Linting

npm run lint

transit gtfs transportation geojson

csv extract-zip fs-extra lodash moment mongodb mongoose node-fetch proj4 untildify yargs

@infinitebrahmanuniverse/nolb-_swi @everything-registry/sub-chunk-887 @zalastax/nolb-_swi

7 years ago

7 years ago

7 years ago

7 years ago

7 years ago

Node-GTFS

Breaking changes in version 1.0.0

Example Application

Installation

Command-line example

Code example

Configuration

Agencies

MongoDB URI

Logging

Deleting existing data

gtfs-import Script

Make sure MongoDB is running

Run the gtfs-import script from Command-line

Command Line options

Skip Delete

Specify path to config JSON file

Show help

Use GTFS import script in code

Query Methods

Setup

Query Methods

gtfs.getAgencies()

gtfs.getRoutes()

gtfs.getStops()

gtfs.getStopsAsGeoJSON()

gtfs.getStoptimes()

gtfs.getTrips()

gtfs.getDirectionsByRoute()

gtfs.getShapes()

gtfs.getShapesAsGeoJSON()

gtfs.getCalendars()

gtfs.getFeedInfo()

gtfs.getFareRules()

gtfs.getFrequencies()

gtfs.getStopAttributes()

gtfs.getTimetables()

gtfs.getTimetableStopOrders()

Timetable Pages

Contributing

Tests

Linting

`gtfs-import` Script

Run the `gtfs-import` script from Command-line