1. Home
  2. Python Parsing Data Normalization
  3. Transfers And Trip Linking
  4. Detecting Impossible Transfers In GTFS

Detecting Impossible Transfers in GTFS

Measure the real distance between each transfer’s stops in a projected CRS, then flag three things: rules whose min_transfer_time is too short for a person to cover that distance, permitted changes the timetable never actually offers, and frequently available changes the feed leaves entirely unconstrained. None of these is a specification violation, which is exactly why no validator reports them and why they survive into production. The rules being audited are loaded in parsing GTFS transfers.txt in Python.

Three findings, three different owners The audit produces three kinds of finding, and each is acted on by someone different. What did the audit find? time too short for distance Too fast the agency should revise the rule rule for a change never offered Stale rule the file has fallen behind frequent change, no rule Unconstrained the router will invent a time

Root Cause Analysis

Transfer data goes wrong in a way that schema validation is structurally unable to see. Every field is present, every identifier resolves, every enumerated value is legal — and the rule still describes something impossible.

Times that contradict geography. An agency states a 60-second minimum between two stops that are 380 metres apart. The row is valid; the connection requires a rider to move at over 6 metres per second. This usually comes from a stop being relocated without the transfer rule being revisited, or from a rule copied between stations with different layouts.

Rules for connections that do not exist. A transfer between two stops is stated, and the timetable never puts a vehicle at both within any plausible window. The rule is harmless in itself but it is diagnostic: transfer files are maintained by hand, and one stale rule usually means the file as a whole has fallen behind the schedule, so the rules that should exist are missing.

Connections nobody constrained. The mirror image, and the most consequential. Two stops 300 metres apart across a dual carriageway, with fifty services a day arriving at each. The feed says nothing, so the transfer graph derives an edge from proximity, and the router starts offering a four-minute interchange that requires crossing six lanes. The only fix is an agency transfer_type = 3 rule, and the only way the agency learns it is needed is if someone tells them.

All three are judgements about plausibility rather than validity. That means the audit’s output is a ranked list for a human, not a gate.

Production-Ready Python Implementation

python 
"""Audit a GTFS feed's transfers for physical and timetable plausibility."""
from __future__ import annotations

import logging
from collections import defaultdict
from dataclasses import dataclass

import pandas as pd

log = logging.getLogger("gtfs.transfers.audit")

WALK_SPEED_MPS = 1.2            # unhurried, level, no crowding
IMPLAUSIBLE_SPEED_MPS = 2.2     # faster than this is a claim no rider can meet
UNCONSTRAINED_RADIUS_M = 300.0  # near enough that a router will invent an edge
MIN_OPPORTUNITIES = 20          # how often a change must be offered to matter


@dataclass
class Finding:
    kind: str                   # "too-fast", "never-offered", "unconstrained"
    from_stop: str
    to_stop: str
    detail: str
    opportunities: int          # how many times the timetable offers this change

    def __str__(self) -> str:
        return (f"[{self.kind}] {self.from_stop} -> {self.to_stop}: {self.detail} "
                f"({self.opportunities} opportunit{'y' if self.opportunities == 1 else 'ies'})")


def _opportunities(stop_times: pd.DataFrame, window_s: int = 1800
                   ) -> dict[tuple[str, str], int]:
    """How often the timetable puts an arrival at A and a later departure at B
    within window_s, for every ordered stop pair that shares a service window."""
    events = stop_times[["stop_id", "arrival_s", "departure_s"]].dropna()
    by_stop: dict[str, list[int]] = defaultdict(list)
    for stop_id, arrival, departure in events.itertuples(index=False):
        by_stop[str(stop_id)].append(int(departure))

    counts: dict[tuple[str, str], int] = defaultdict(int)
    arrivals: dict[str, list[int]] = defaultdict(list)
    for stop_id, arrival, departure in events.itertuples(index=False):
        arrivals[str(stop_id)].append(int(arrival))

    for stop_id in arrivals:
        arrivals[stop_id].sort()
    for stop_id in by_stop:
        by_stop[stop_id].sort()
    return counts, arrivals, by_stop


def _count_pair(arrivals: list[int], departures: list[int], window_s: int) -> int:
    """Arrivals followed by a departure within the window — a merge, not a product."""
    import bisect

    total = 0
    for a in arrivals:
        lo = bisect.bisect_left(departures, a)
        hi = bisect.bisect_right(departures, a + window_s)
        total += hi - lo
    return total


def audit_transfers(index, stop_times: pd.DataFrame,
                    projected: pd.DataFrame, window_s: int = 1800) -> list[Finding]:
    """projected: stop_id, x, y in a METRIC CRS."""
    xy = {row.stop_id: (row.x, row.y) for row in projected.itertuples(index=False)}

    def metres(a: str, b: str) -> float | None:
        pa, pb = xy.get(a), xy.get(b)
        if pa is None or pb is None:
            return None
        return ((pa[0] - pb[0]) ** 2 + (pa[1] - pb[1]) ** 2) ** 0.5

    _, arrivals, departures = _opportunities(stop_times, window_s)
    findings: list[Finding] = []

    stated: set[tuple[str, str]] = set()
    for rules in index._by_pair.values():
        for rule in rules:
            stated.add((rule.from_stop, rule.to_stop))
            distance = metres(rule.from_stop, rule.to_stop)
            offered = _count_pair(arrivals.get(rule.from_stop, []),
                                  departures.get(rule.to_stop, []), window_s)

            if rule.permitted and distance is not None and distance > 0:
                seconds = rule.effective_seconds
                if seconds > 0 and distance / seconds > IMPLAUSIBLE_SPEED_MPS:
                    findings.append(Finding(
                        "too-fast", rule.from_stop, rule.to_stop,
                        f"{distance:.0f} m in {seconds:.0f} s needs "
                        f"{distance / seconds:.1f} m/s; a realistic walk is "
                        f"{distance / WALK_SPEED_MPS:.0f} s",
                        offered))

            if offered == 0:
                findings.append(Finding(
                    "never-offered", rule.from_stop, rule.to_stop,
                    "the timetable never puts an arrival here and a later departure "
                    "there within the window — the rule may be stale",
                    0))

    # Connections close enough that a router will invent them, which nobody constrained.
    for a in arrivals:
        for b in departures:
            if a == b or (a, b) in stated:
                continue
            distance = metres(a, b)
            if distance is None or distance > UNCONSTRAINED_RADIUS_M:
                continue
            offered = _count_pair(arrivals[a], departures[b], window_s)
            if offered >= MIN_OPPORTUNITIES:
                findings.append(Finding(
                    "unconstrained", a, b,
                    f"{distance:.0f} m apart and offered often, but the feed states "
                    "no rule — a router will guess a time for it",
                    offered))

    findings.sort(key=lambda f: f.opportunities, reverse=True)
    by_kind = defaultdict(int)
    for f in findings:
        by_kind[f.kind] += 1
    log.info("transfer audit: %s", dict(by_kind) or "nothing to report")
    return findings
A stated time measured against the walk it implies Two platforms 366 metres apart with a stated 90-second minimum: the implied pace is faster than any rider will manage. platform A2 0 m concourse 180 m platform B1 366 m 90 s over 366 m needs 4.1 m/s; a realistic walk takes 305 s long distance plus short stated time is the combination worth flagging

Step-by-Step Walkthrough

Distances come from a projected frame, never from latitude and longitude. The function takes projected and uses plain Euclidean arithmetic on it, which is correct precisely because the caller has already chosen a metric CRS. Doing the same arithmetic on degrees would understate east–west distances by half at northern latitudes, and every “too fast” finding would be wrong in a latitude-dependent way.

_count_pair is a merge over sorted times, not a nested product. For each arrival it binary-searches the departure list for the window, which is O(n log m) rather than O(nm). On a busy stop pair with thousands of events each, that is the difference between the audit finishing and not.

“Opportunities” is the ranking key for everything. A findings list sorted by severity is a list nobody acts on, because the worst-sounding item may affect one journey a week. Sorting by how often the timetable actually offers the connection puts the agency’s attention where riders are.

IMPLAUSIBLE_SPEED_MPS is 2.2, not walking speed. Flagging anything above 1.2 m/s would report every cross-platform interchange, where the rider moves ten metres and the stated time is deliberately short. 2.2 m/s is a fast jog; a stated time requiring more than that is a claim, not a tight schedule.

Zero-second transfers are skipped by the seconds > 0 guard. A timed or in-seat transfer has no meaningful implied speed — the vehicle waits, or the rider does not move — so measuring one produces a division that means nothing.

The unconstrained scan is bounded twice. By distance, because a router only invents edges for nearby stops, and by opportunity count, because a pair the timetable connects twice a year is not worth an agency’s time. Both thresholds are named constants, so the report can be re-run at a different sensitivity without editing logic.

Verification and Output

python 
def verify(findings: list[Finding]) -> None:
    kinds = {"too-fast", "never-offered", "unconstrained"}
    assert all(f.kind in kinds for f in findings), "unknown finding kind"
    assert all(f.opportunities >= 0 for f in findings), "negative opportunity count"
    counts = [f.opportunities for f in findings]
    assert counts == sorted(counts, reverse=True), "findings are not ranked"
    assert all(f.from_stop != f.to_stop for f in findings), "self-transfer reported"

A representative run against a metropolitan feed:

text 
INFO gtfs.transfers.audit: transfer audit: {'unconstrained': 34, 'too-fast': 6, 'never-offered': 11}

[unconstrained] stop_4471 -> stop_1188: 210 m apart and offered often, but the feed states no rule — a router will guess a time for it (1284 opportunities)
[unconstrained] stop_1188 -> stop_4471: 210 m apart and offered often, but the feed states no rule — a router will guess a time for it (1190 opportunities)
[too-fast] plat_a2 -> plat_b1: 366 m in 90 s needs 4.1 m/s; a realistic walk is 305 s (642 opportunities)
[never-offered] stop_9902 -> stop_9903: the timetable never puts an arrival here and a later departure there within the window — the rule may be stale (0 opportunities)

The first two lines are the same interchange in both directions, offered nearly 2,500 times a week and entirely unconstrained. That is the single most valuable line the audit produces: it tells the agency precisely which rule to write first.

Ranking findings by how often they bite Weekly opportunities for each flagged connection: severity ranks nothing useful, and frequency puts the agency's attention where riders are. Unconstrained pair, both directions 2474 offered nearly 2,500 times a week Too-fast rule at a major interchange 642 Too-fast rule at a minor stop 38 real, and low priority Stale rule 0 the change is never offered at all a findings list sorted by severity is a list nobody acts on

Gotchas and Edge Cases

  • Stops missing coordinates. metres returns None and the distance checks are skipped rather than assumed. A feed with many uncoordinated stops will produce an audit that silently covers only part of the network, so count the skips.
  • Out-of-station interchanges that are genuinely long. Some cities have named interchanges 500 metres apart with a stated ten-minute transfer. Those pass the speed check correctly; it is only the combination of long distance and short stated time that is implausible.
  • The window length changes the answer. Thirty minutes suits an urban network and is far too short for a rural one where the next bus is in two hours. Tune window_s per feed, and record what was used alongside the findings.
  • Frequency-based services. Stops served by frequency-defined trips have few explicit stop_times rows, so their opportunity counts are understated. Expand frequencies before auditing, or the busiest corridors will look like the quietest.
  • Both directions of the same interchange. They are separate findings by design, because a transfer can be permitted one way and not the other. Deduplicating them for a human-facing report is reasonable; deduplicating them in the data is not.

Frequently Asked Questions

Is a short min_transfer_time always wrong?

No. A cross-platform interchange genuinely takes seconds, and agencies state small values for exactly that reason. What is suspicious is a small value across a large distance — 60 seconds for a 400-metre change is a claim no rider can meet.

Why check whether a transfer is ever offered?

Because a rule covering a connection the timetable never makes is dead weight, and usually a symptom: it commonly means stop identifiers changed and the transfer rules were not updated with them, so the rules that matter are missing too.

What is the value in finding unconstrained connections?

Those are the changes a routing engine will invent times for. Ranking them by how often the timetable offers them tells the agency exactly which interchanges are worth stating explicitly, in priority order.

Should this audit block an ingest?

No. Every finding here is a judgement about plausibility, not a specification violation, and a feed can be entirely valid and still fail all of these checks. Report it to whoever can talk to the agency.