Live instance: a public deployment merging the six La Réunion transit networks is running at https://gtfs.bus.re/.
A single-binary service that fetches GTFS Static and GTFS-Realtime feeds from
N transit networks, namespaces every identifier (carjaune:4211), merges
them into one consolidated feed, synthesises location_type=1 parent stations
linking equivalent stops across networks, and serves the merged feed, each
source feed (post-transform), and operational status endpoints.
Design document: gtfs-merge-design.md. This implementation covers milestones M0–M3 (fetch/cache, prefixed merge, realtime pipeline with snapshot pinning, clustering with reports and overrides).
make build
./gtfs-merge -config reunion.yaml # daemon on :8080
./gtfs-merge -config reunion.yaml -once # one-shot: writes ./out/ and exitsreunion.yaml is a ready-to-run config for the six La Réunion networks
(Car Jaune, Alternéo, Carsud, Kar'Ouest, Citalis, Estival).
Data endpoints are versioned under /api/v1; the pre-/api/v1 paths answer
with permanent redirects. Special pages (/status, /healthz, /readyz)
and Prometheus (/metrics) stay at the root.
| Path | Description |
|---|---|
GET /api/v1/gtfs.zip |
Merged static feed. ETag/If-None-Match, ranges. |
GET /api/v1/gtfs-rt/{trip-updates,vehicle-positions,alerts}.pb |
Merged realtime, per entity type. ?format=json for debugging. |
GET /api/v1/gtfs-rt/all.pb |
Merged realtime, every entity type in one FeedMessage. |
GET /api/v1/feeds |
Configured feeds, their endpoints and health. |
GET /api/v1/feeds/{id}/gtfs.zip |
One feed, prefixed, otherwise untouched. |
GET /api/v1/feeds/{id}/gtfs-rt/{kind}.pb |
One feed's realtime, prefixed + resolved. |
GET /api/v1/feeds/{id}/original/gtfs.zip |
Raw upstream bytes (publish.expose_originals). |
GET /api/v1/stations/clusters.json |
Every synthetic parent station with members and pair scores. |
GET /api/v1/stations/near-misses.json |
Pairs within ±0.1 of the threshold — the tuning surface. |
GET /api/v1/stations/clusters.geojson |
Clusters for a map / QGIS. |
GET /api/v1/id-map?id=carjaune:123 |
Resolve a prefixed ID → feed, original, cluster. |
GET /api/v1/status.json |
Machine-readable status. |
GET /api/v1/webui.json |
Web-UI settings (webui: config block, e.g. the PMTiles URL). |
GET /api/v1/version |
Build info, config hash, snapshot version. |
POST /api/v1/admin/refresh?feed=x |
Force refetch+rebuild (Authorization: Bearer <admin_token>). |
GET /, GET /status |
Dashboard: a static tabbed page (vanilla JS fetching /api/v1): status, merged-stops table, rejected-merges table, and two maplibre-gl maps rendering the protomaps PMTiles from webui.pmtiles_url. |
GET /metrics |
Prometheus. |
GET /healthz, /readyz |
Liveness; readiness (200 once a snapshot exists). |
Feed responses carry X-GTFS-Static-Version, X-Feed-Age-Seconds and
X-Feeds-Degraded headers. Realtime responses are pinned to exactly one
static snapshot version.
newID = prefix + ":" + originalID, applied to every ID column and every
foreign key in every file (see the table in the design doc §5), and to every
ID-bearing GTFS-RT field. Original IDs that already contain : (alterneo)
stay unescaped: split on the first : against the known prefix set to
reverse. Synthetic parent stations use the reserved P: prefix.
A feed whose agency.txt has no agency_id (legal upstream, illegal once
merged — carsud) gets one synthesized from its feed id, applied consistently
in agency.txt, routes.txt and fare_attributes.txt.
Stops are clustered at station level: an upstream station absorbs its
children and is never broken apart; two stations from the same feed never
merge (the operator said they're different). Candidate pairs come from a
spatial grid, are scored on name similarity (Jaro-Winkler / token-set /
trigram, after diacritic folding, abbreviation expansion and noise-token
removal), distance and mode, with hard gates on both name and distance. A
repair pass splits any component that exceeds max_cluster_diameter /
max_cluster_size, which is what prevents transitive drift ("Place de la
République" chains). Authoritative external identities (stops.identity_field)
must-link regardless of name, capped by a 2 km sanity check.
Stops with identical (case/accent-folded) names within
exact_name_distance (default 200 m) merge even when the weighted score
would reject them — feeds that model no parent stations name their
opposite-direction stops identically. Implemented as a score floor, so such
edges are the first sacrificed by the repair pass and every guardrail
(must_not_link, same-feed station separation, the diameter cap) still
applies. Set it to 0 to disable.
Materialisation writes the synthetic parents (medoid coordinates, highest-
priority feed's name), reparents members, demotes absorbed upstream stations,
and emits transfers.txt (transfer_type=2) between clustered stops.
Human overrides live in overrides.yaml (must_link / must_not_link) and
are re-read on every build; see overrides.example.yaml. Parent IDs are
stable for stable membership (P: + hash of member IDs); a must_link entry
pins a human-chosen ID (P:gare-st-pierre) that survives membership churn.
- Upstream fetch fails → serve last good copy (disk cache survives restarts),
feed marked
degraded, retries with jittered exponential backoff. - Corrupt/unscannable feed → excluded from the build, everything else ships.
- Realtime source stale (>
stale_after) → still served, synthetic service alert injected into the merged alerts feed (so RT-only consumers learn of the outage); pastdrop_afterits entities are dropped (an empty valid feed rather than lies). - Dangling RT references (trip/stop unknown in the pinned static index) are
dropped and counted per feed — watch
gtfs_rt_dangling_total.
make once # CI-style dry run into ./out
make docker # distroless image
docker compose up # with a persistent cache volumeA hardened systemd unit is in deploy/gtfs-merge.service; SIGHUP (or
systemctl reload) hot-reloads the config (changing server.listen still
needs a restart).
CORS is off by default. server.cors_origins accepts exact origins,
"*", and scheme-agnostic patterns: "localhost:*" (any port),
"*.bus.re" (subdomains only), "sysdevrun.github.io" (any scheme/port).
The dashboard's map tabs need webui.pmtiles_url pointing at a protomaps
basemap PMTiles archive; maplibre-gl / pmtiles / protomaps-themes-base are
loaded from unpkg by the browser. webui.terrain_pmtiles_url is optional and
adds a hillshade layer — it must point at a terrarium-encoded DEM archive.
From the design doc: validation gate, feed archive, cluster-ID persistence
journal (beyond must_link pinning), review UI, trip/vehicle de-duplication,
GTFS-RT TripModifications (absent from the Go protobuf bindings; none of
the configured feeds publish it), INCREMENTAL realtime feeds.
MIT — © 2026 Théophile Helleboid contact@sys-dev-run.fr.