1. Home
  2. Python Parsing Data Normalization
  3. Transfers And Trip Linking
  4. Parsing GTFS Transfers In Python

Parsing GTFS transfers.txt in Python

Read transfers.txt with explicit dtypes, tolerate its absence by returning an empty typed frame, and index every rule under its (from_stop_id, to_stop_id) pair with the candidates ranked by how many of the optional trip and route qualifiers they populate. Resolution is then a dictionary probe followed by picking the top-ranked candidate. The trap, exactly as with fare rules, is that an empty qualifier means unconstrained rather than no match. The model these rules express is covered in transfers and trip linking.

How a transfer rule is matched The optional qualifiers behave exactly like fare rule qualifiers: empty declines to constrain, populated must match exactly. Does this rule's qualifier match the change? qualifier empty Matches the rule does not constrain that axis qualifier equal Matches and ranks as more specific qualifier differs No match a more general rule may still apply

Root Cause Analysis

transfers.txt looks like the simplest file in a GTFS feed. It has eight columns, most feeds that carry it have a few hundred rows, and the semantics fit in a paragraph. It nonetheless produces a disproportionate share of routing bugs, for three reasons.

The qualifiers are optional and conjunctive. A row may name only two stops, or it may additionally name the routes, the trips, or both. An unpopulated qualifier does not constrain, so a bare stop-pair rule matches every change between those stops — including changes between the specific trips that a more specific rule also covers. Both rules match; only one should apply.

Absence is meaningful in the wrong direction. A missing transfers.txt is the common case, and it means the agency has said nothing. Code that treats “no rule” as “no transfer allowed” makes most feeds unroutable. Code that treats it as “transfer allowed instantly” promises impossible connections. The correct reading is “apply your own default”, which means the loader must return something a caller can distinguish from a rule.

transfer_type = 3 is a prohibition, not a cost. It is the only way a feed can say a connection cannot be made, and it is easy to lose. A loader that filters to rows with a min_transfer_time drops every type 3 row, because prohibitions carry no time. The connection then looks unconstrained, which is the exact opposite of what the agency said.

Production-Ready Python Implementation

python 
"""Load and index GTFS transfers.txt."""
from __future__ import annotations

import logging
from collections import defaultdict
from dataclasses import dataclass
from pathlib import Path
from zipfile import ZipFile

import pandas as pd

log = logging.getLogger("gtfs.transfers")

TRANSFER_DTYPES = {
    "from_stop_id": "string", "to_stop_id": "string",
    "from_route_id": "string", "to_route_id": "string",
    "from_trip_id": "string", "to_trip_id": "string",
    "transfer_type": "int8", "min_transfer_time": "float64",
}
QUALIFIERS = ("from_trip_id", "to_trip_id", "from_route_id", "to_route_id")

KIND_NAMES = {0: "recommended", 1: "timed", 2: "minimum time",
              3: "not possible", 4: "in-seat", 5: "in-seat not allowed"}
# transfer_type values that permit the change at all.
PERMITTED = {0, 1, 2, 4}


@dataclass(frozen=True)
class TransferRule:
    from_stop: str
    to_stop: str
    kind: int
    min_seconds: float | None
    from_trip: str | None
    to_trip: str | None
    from_route: str | None
    to_route: str | None

    @property
    def specificity(self) -> int:
        return sum(v is not None for v in
                   (self.from_trip, self.to_trip, self.from_route, self.to_route))

    @property
    def permitted(self) -> bool:
        return self.kind in PERMITTED

    @property
    def effective_seconds(self) -> float:
        """What a router should budget for this change."""
        if self.kind in (1, 4):        # timed or in-seat: the operator guarantees it
            return 0.0
        return self.min_seconds if self.min_seconds is not None else 0.0

    def describe(self) -> str:
        return (f"{self.from_stop}->{self.to_stop} {KIND_NAMES.get(self.kind, '?')}"
                f" ({self.effective_seconds:.0f}s, specificity {self.specificity})")


class TransferIndex:
    def __init__(self, by_pair: dict[tuple[str, str], list[TransferRule]]):
        self._by_pair = by_pair

    def resolve(self, from_stop: str, to_stop: str,
                from_trip: str | None = None, to_trip: str | None = None,
                from_route: str | None = None, to_route: str | None = None):
        """The single most specific rule covering this change, or None."""
        journey = {"from_trip_id": from_trip, "to_trip_id": to_trip,
                   "from_route_id": from_route, "to_route_id": to_route}
        for rule in self._by_pair.get((from_stop, to_stop), ()):
            actual = {"from_trip_id": rule.from_trip, "to_trip_id": rule.to_trip,
                      "from_route_id": rule.from_route, "to_route_id": rule.to_route}
            # An unpopulated qualifier does not constrain; a populated one must match.
            if all(actual[q] is None or actual[q] == journey[q] for q in QUALIFIERS):
                return rule
        return None

    def forbidden_pairs(self) -> set[tuple[str, str]]:
        return {pair for pair, rules in self._by_pair.items()
                if rules and rules[0].kind in (3, 5) and rules[0].specificity == 0}

    def __len__(self) -> int:
        return sum(len(v) for v in self._by_pair.values())


def _opt(value) -> str | None:
    return None if pd.isna(value) else str(value)


def load_transfers(feed_path: Path) -> TransferIndex:
    with ZipFile(feed_path) as archive:
        if "transfers.txt" not in archive.namelist():
            log.info("%s carries no transfers.txt — transfers are unconstrained",
                     feed_path.name)
            return TransferIndex({})
        with archive.open("transfers.txt") as fh:
            frame = pd.read_csv(fh, dtype=TRANSFER_DTYPES,
                                keep_default_na=False, na_values=[""])

    by_pair: dict[tuple[str, str], list[TransferRule]] = defaultdict(list)
    missing_time = 0
    for row in frame.itertuples(index=False):
        kind = int(row.transfer_type)
        if kind == 2 and pd.isna(row.min_transfer_time):
            missing_time += 1
        rule = TransferRule(
            from_stop=str(row.from_stop_id), to_stop=str(row.to_stop_id),
            kind=kind,
            min_seconds=None if pd.isna(row.min_transfer_time)
            else float(row.min_transfer_time),
            from_trip=_opt(row.from_trip_id), to_trip=_opt(row.to_trip_id),
            from_route=_opt(row.from_route_id), to_route=_opt(row.to_route_id),
        )
        by_pair[(rule.from_stop, rule.to_stop)].append(rule)

    # Most specific first, so resolve() can return the first rule that matches.
    for rules in by_pair.values():
        rules.sort(key=lambda r: r.specificity, reverse=True)

    if missing_time:
        log.warning("%d transfer(s) of type 2 carry no min_transfer_time — "
                    "they will be treated as instantaneous", missing_time)

    counts = frame["transfer_type"].value_counts().to_dict()
    log.info("%s: %d transfer rule(s) over %d stop pair(s): %s",
             feed_path.name, len(frame), len(by_pair),
             ", ".join(f"{KIND_NAMES.get(int(k), k)}={v}" for k, v in sorted(counts.items())))
    return TransferIndex(dict(by_pair))
From file to constant-time lookup Rules are read, bucketed by stop pair, sorted most specific first, and then resolved by returning the first rule whose populated qualifiers all match. Read explicit dtypes Bucket by stop pair Sort specificity first Resolve first match wins an absent transfers.txt yields an empty index, not a null

Step-by-Step Walkthrough

A missing file produces an empty index, not None. TransferIndex({}) answers every resolve with None, which is exactly what “the agency has not constrained this change” should look like to a caller. No null checks propagate outwards, and the one informational log line records why every lookup is going to miss.

Rules are sorted once, at load, most specific first. resolve then returns the first rule whose populated qualifiers all match, and that is by construction the most specific match. Sorting per query would work and would cost the same as the scan it was meant to avoid.

The match condition is actual[q] is None or actual[q] == journey[q]. This is the whole of the qualifier semantics in one line. A rule that does not name a trip matches regardless of which trip is being asked about; a rule that names one matches only that trip. Reversing the direction of the None test is the bug this line exists to make obvious.

effective_seconds folds the type into the duration. A timed or in-seat transfer costs nothing because the operator guarantees it; a minimum-time transfer costs its stated seconds; a recommended transfer with no time costs nothing. A router asking “how long does this take” gets one number without having to reimplement the enum.

Prohibitions keep their rows. TransferRule stores kind regardless of value, and permitted exposes it. Nothing filters type 3 or 5 out at load, because those rows carry the most important statement in the file. Filtering happens when a graph is built, and there the edge is omitted rather than weighted.

Type 2 with no time is warned about, not repaired. Treating it as instantaneous is the specification-neutral reading, and inventing a default here would hide a feed defect that the agency can fix.

Verification and Output

python 
def verify(index: TransferIndex, feed_path: Path) -> list[str]:
    with ZipFile(feed_path) as archive:
        with archive.open("stops.txt") as fh:
            stops = pd.read_csv(fh, dtype="string", usecols=["stop_id"])
        with archive.open("trips.txt") as fh:
            trips = pd.read_csv(fh, dtype="string", usecols=["trip_id", "route_id"])

    known_stops = set(stops["stop_id"])
    known_trips = set(trips["trip_id"])
    known_routes = set(trips["route_id"])

    problems, checked = [], 0
    for rules in index._by_pair.values():
        for rule in rules:
            checked += 1
            for stop in (rule.from_stop, rule.to_stop):
                if stop not in known_stops:
                    problems.append(f"transfer references unknown stop {stop}")
            for trip in (rule.from_trip, rule.to_trip):
                if trip is not None and trip not in known_trips:
                    problems.append(f"transfer references unknown trip {trip}")
            for route in (rule.from_route, rule.to_route):
                if route is not None and route not in known_routes:
                    problems.append(f"transfer references unknown route {route}")
            if rule.kind not in KIND_NAMES:
                problems.append(f"unknown transfer_type {rule.kind}")
            if rule.min_seconds is not None and rule.min_seconds < 0:
                problems.append(f"negative min_transfer_time on {rule.describe()}")

    log.info("verified %d transfer rule(s), %d problem(s)", checked, len(problems))
    return problems

Loading a large interchange feed:

text 
INFO gtfs.transfers: metro_20261102.zip: 1284 transfer rule(s) over 1102 stop pair(s): recommended=41, timed=18, minimum time=1198, not possible=27
WARN gtfs.transfers: 3 transfer(s) of type 2 carry no min_transfer_time — they will be treated as instantaneous

The twenty-seven prohibitions are the rows worth reading by hand. Each one is an agency telling you that a connection your routing engine would otherwise offer cannot be made, and each one is a wrong itinerary avoided.

Resolving a specific change:

python 
rule = index.resolve("plat_2", "plat_7", from_route="M1", to_route="M4")
print(rule.describe() if rule else "unconstrained")
# plat_2->plat_7 minimum time (240s, specificity 2)
What a large interchange feed actually contains Transfer rules by type in a metropolitan feed — the twenty-seven prohibitions are the rows worth reading by hand. Minimum time 1198 the ordinary large-interchange case Recommended 41 no time stated Not possible 27 connections a router would otherwise offer Timed 18 the vehicle is held each prohibition is one impossible itinerary avoided

Gotchas and Edge Cases

  • Two rules with identical specificity for the same pair. The sort is stable, so the file order decides. That is arbitrary but deterministic; if the two disagree on transfer_type the feed is contradictory and deserves a report.
  • A transfer whose stops are in different stations. Entirely normal — an out-of-station interchange across a road. It only becomes suspicious when the distance is large, which is the check in detecting impossible transfers.
  • from_route_id naming a route the from_trip_id does not belong to. Contradictory, and the verification above does not catch it because both identifiers exist. Cross-checking the trip’s own route is worth adding on feeds that populate both.
  • Very large transfers.txt. A few feeds enumerate every platform pair in every station, reaching tens of thousands of rows. The pair index handles it, but the verify scan above is linear and should be run once at load rather than per query.
  • Self-transfers with a long minimum time. A rule from a stop to itself with 600 seconds is the agency saying that changing vehicles at that platform genuinely takes ten minutes, usually because it is a terminus where vehicles lay over. It is real, and dropping it makes the router promise a connection that does not exist.

Frequently Asked Questions

What should happen when transfers.txt is missing?

Nothing should break. An absent file means the agency has not constrained transfers, not that transfers are impossible. Return an empty typed frame so every downstream filter still works, and let the routing engine apply its own minimum-connection defaults.

Which rule applies when several match the same change?

The most specific: the rule populating the largest number of the optional trip and route qualifiers. A rule naming both trips beats one naming both routes, which beats a bare stop-pair rule.

Is a transfer from a stop to itself meaningful?

Yes, and it is common. It models the minimum time needed to change vehicles at the same platform. Filtering out self-loops as though they were noise removes a real constraint that a routing engine needs.

Should min_transfer_time be trusted over a computed walking time?

Yes. It is the agency’s own statement about its own station, and it accounts for things no computation can see — fare gates, platform crowding, a long ramp. Use computed times only where the feed gives none.