Calendar and Service Exception Modeling

Every question a rider asks a transit application — when is the next train, does this bus run on Sunday, is there service on the holiday — resolves to a single internal question: which service_id values are active on this date? GTFS answers that question in two files that have to be read together, and reading only one of them is among the most common and most damaging defects in transit data pipelines. calendar.txt describes service as a weekly pattern bounded by a start and end date. calendar_dates.txt describes the exceptions: the holidays when the weekday pattern does not apply, the special-event Saturdays when it does, the single-day shutdowns for engineering work.

Neither file alone is the schedule. The schedule is the weekly pattern with the exception layer applied on top of it, and the arithmetic that combines them has to be done in date space, not in string space. This page covers the specification rules that govern both files, a Python implementation that expands them into a date-indexed service map, and the failure modes that appear once you run that code against real agency feeds rather than the spec examples.

Prerequisites

• Python 3.9 or later, with pandas for the tabular work and the standard library’s datetime module for date arithmetic
• A GTFS archive containing trips.txt and at least one of calendar.txt or calendar_dates.txt
• Familiarity with the static feed structure and how service_id links trips.txt to the calendar files
• An understanding of the GTFS service day, which is not the same thing as a calendar day

pip install pandas

Concept and Spec Background

A row in calendar.txt is a service pattern. It carries a service_id, seven boolean columns named for the days of the week, and a start_date / end_date pair in YYYYMMDD form. The row asserts that the service runs on every listed weekday that falls inside the inclusive date range. A typical feed has a handful of these rows — a weekday pattern, a Saturday pattern, a Sunday pattern, and perhaps a school-term variant — and tens of thousands of trips pointing at them.

A row in calendar_dates.txt is an exception to that assertion. It carries a service_id, a single date, and an exception_type that is either 1 (service added on this date) or 2 (service removed on this date). The exception is scoped to one service and one date; there is no range form, so an agency closing a line for a fortnight emits fourteen rows.

The two files interact in a way the specification states plainly but that pipelines routinely get wrong: calendar_dates.txt is applied after expansion, and it overrides whatever the weekly pattern said. An exception_type of 1 can add a date outside the start_date–end_date window entirely. An exception_type of 2 can remove a date the weekly mask included. There is no precedence subtlety and no partial application; the exception is absolute for that service on that date.

The calendar_dates-only feed

calendar.txt is conditionally required, which in practice means optional. A feed may omit it entirely and enumerate every single service date in calendar_dates.txt with exception_type 1. This is not an exotic edge case — it is how a number of large agencies publish, because it removes the weekly-pattern abstraction and lets the scheduling system emit dates directly. Any code that reads calendar.txt unconditionally will raise a FileNotFoundError on those feeds, and any code that treats a missing calendar.txt as “no service” will report an empty schedule for a feed that is perfectly valid. Both behaviours are covered in detail in handling calendar_dates-only feeds.

Dates are dates, not integers and not strings

YYYYMMDD sorts correctly as a string and converts cleanly to an integer, and both properties tempt engineers into skipping the parse. Both eventually bite. String comparison breaks the moment a feed pads inconsistently or a column is read with a stray whitespace character. Integer comparison silently permits arithmetic that is meaningless — adding one to 20251031 produces 20251032, a date that does not exist, and no exception is raised. Parse once, into a real date type, and do every comparison and every increment there. The same discipline applied to time fields applies here to date fields.

Step-by-Step Implementation

The goal is a mapping from each calendar date in the feed’s coverage window to the set of service_id values active on it. Every downstream question — which trips run today, what is the headway on a Sunday, is there service on 25 December — is a lookup in that map.

Step 1 — Read both files defensively

import logging
from datetime import date, timedelta
from pathlib import Path
from zipfile import ZipFile

import pandas as pd

log = logging.getLogger("gtfs.calendar")

CALENDAR_DTYPES = {
    "service_id": "string",
    "monday": "int8", "tuesday": "int8", "wednesday": "int8", "thursday": "int8",
    "friday": "int8", "saturday": "int8", "sunday": "int8",
    "start_date": "string", "end_date": "string",
}
EXCEPTION_DTYPES = {
    "service_id": "string",
    "date": "string",
    "exception_type": "int8",
}
DAY_COLUMNS = ["monday", "tuesday", "wednesday", "thursday",
               "friday", "saturday", "sunday"]


def _read_member(archive: ZipFile, name: str, dtypes: dict) -> pd.DataFrame:
    """Read one GTFS member, or return an empty typed frame when it is absent."""
    if name not in archive.namelist():
        log.info("%s is not present in this feed", name)
        return pd.DataFrame({col: pd.Series(dtype=dt) for col, dt in dtypes.items()})
    with archive.open(name) as handle:
        return pd.read_csv(handle, dtype=dtypes, keep_default_na=False, na_values=[""])


def read_calendar_tables(feed_path: Path) -> tuple[pd.DataFrame, pd.DataFrame]:
    with ZipFile(feed_path) as archive:
        calendar = _read_member(archive, "calendar.txt", CALENDAR_DTYPES)
        exceptions = _read_member(archive, "calendar_dates.txt", EXCEPTION_DTYPES)
    if calendar.empty and exceptions.empty:
        raise ValueError("feed defines no service: both calendar files are absent or empty")
    return calendar, exceptions

Reading the date columns as string rather than letting pandas infer them is deliberate. Inference turns 20251102 into an int64, and an int64 that later meets a genuinely null value becomes a float, at which point 20251102.0 no longer round-trips to the right date. Keep the raw text and parse it explicitly in the next step.

Step 2 — Parse the date bounds

def _to_date(series: pd.Series) -> pd.Series:
    """Parse a YYYYMMDD column into real dates, reporting anything unparseable."""
    parsed = pd.to_datetime(series, format="%Y%m%d", errors="coerce")
    bad = series[parsed.isna() & series.notna()]
    if len(bad):
        log.error("%d unparseable date value(s), e.g. %r", len(bad), bad.iloc[0])
    return parsed.dt.date

errors="coerce" turns a malformed value into NaT rather than raising, which lets the loader report every bad row at once instead of dying on the first. A feed with an unparseable end_date is broken, but it is more useful to say “these four rows are broken” than to stop at the first.

Step 3 — Expand the weekly patterns

def expand_calendar(calendar: pd.DataFrame) -> pd.DataFrame:
    """One row per (service_id, date) implied by the weekly patterns."""
    if calendar.empty:
        return pd.DataFrame({"service_id": pd.Series(dtype="string"),
                             "date": pd.Series(dtype="object")})

    starts = _to_date(calendar["start_date"])
    ends = _to_date(calendar["end_date"])
    rows: list[tuple[str, date]] = []

    for idx, service_id in enumerate(calendar["service_id"]):
        start, end = starts.iloc[idx], ends.iloc[idx]
        if pd.isna(start) or pd.isna(end) or end < start:
            log.error("service %s has an invalid date range %s..%s", service_id, start, end)
            continue
        mask = [bool(calendar[col].iloc[idx]) for col in DAY_COLUMNS]
        if not any(mask):
            log.warning("service %s runs on no weekday at all", service_id)
            continue
        current = start
        while current <= end:
            if mask[current.weekday()]:
                rows.append((service_id, current))
            current += timedelta(days=1)

    return pd.DataFrame(rows, columns=["service_id", "date"])

date.weekday() returns 0 for Monday through 6 for Sunday, which is exactly the order the GTFS columns appear in, so the mask indexes directly. That alignment is a happy accident of the specification and worth an explicit comment in production code, because the alternative convention — Sunday first — is common enough elsewhere that a future reader will wonder.

The loop is the honest implementation. It is also the slow one: a feed with 40 service patterns over a two-year window generates roughly 30,000 rows through a Python loop. For the feed sizes this section deals with that is a fraction of a second and not worth optimising, but the vectorised alternative is covered under performance below.

Step 4 — Apply the exception layer

def apply_exceptions(expanded: pd.DataFrame, exceptions: pd.DataFrame) -> pd.DataFrame:
    """Add exception_type 1 dates and remove exception_type 2 dates."""
    if exceptions.empty:
        return expanded.drop_duplicates()

    ex = exceptions.assign(date=_to_date(exceptions["date"]))
    ex = ex[ex["date"].notna()]

    unknown = set(ex["exception_type"].unique()) - {1, 2}
    if unknown:
        log.error("unknown exception_type value(s): %s", sorted(unknown))
        ex = ex[ex["exception_type"].isin([1, 2])]

    added = ex.loc[ex["exception_type"] == 1, ["service_id", "date"]]
    removed = ex.loc[ex["exception_type"] == 2, ["service_id", "date"]]

    combined = pd.concat([expanded, added], ignore_index=True).drop_duplicates()

    if len(removed):
        keys = set(zip(removed["service_id"], removed["date"]))
        keep = [
            (sid, d) not in keys
            for sid, d in zip(combined["service_id"], combined["date"])
        ]
        combined = combined[keep]

    return combined.reset_index(drop=True)

Additions are concatenated before removals are applied, which is the only ordering that respects the specification. A service that is both added and removed on the same date is a contradiction in the feed; applying removals last resolves it as “removed”, which is the safer reading — declining to promise service is a smaller failure than promising service that does not exist.

Step 5 — Index it, and join it to trips

def service_calendar(feed_path: Path) -> pd.DataFrame:
    calendar, exceptions = read_calendar_tables(feed_path)
    expanded = expand_calendar(calendar)
    return apply_exceptions(expanded, exceptions)


def trips_on(feed_path: Path, target: date) -> pd.DataFrame:
    """Every trip that runs on one calendar date."""
    active = service_calendar(feed_path)
    service_ids = set(active.loc[active["date"] == target, "service_id"])
    with ZipFile(feed_path) as archive, archive.open("trips.txt") as handle:
        trips = pd.read_csv(handle, dtype="string")
    return trips[trips["service_id"].isin(service_ids)]

Once the map exists, every calendar question becomes a filter on it. That is the point of building it: the arithmetic is done once, at load time, and the hot path is a set membership test rather than a re-derivation of the weekly pattern.

Validation and Verification

A service calendar is easy to build and easy to build wrongly, because a wrong one still returns plausible-looking trips. These four assertions catch the defects that actually occur.

def verify(active: pd.DataFrame, calendar: pd.DataFrame, exceptions: pd.DataFrame) -> None:
    # 1. Every service referenced anywhere produced at least one date.
    declared = set(calendar["service_id"]) | set(exceptions["service_id"])
    produced = set(active["service_id"])
    orphans = declared - produced
    assert not orphans, f"service(s) with no active dates at all: {sorted(orphans)[:5]}"

    # 2. No duplicate (service_id, date) pairs survived the merge.
    assert not active.duplicated(["service_id", "date"]).any(), "duplicate service dates"

    # 3. Every exception_type 2 date really is gone.
    removed = exceptions[exceptions["exception_type"] == 2]
    removed_keys = set(zip(removed["service_id"], _to_date(removed["date"])))
    live_keys = set(zip(active["service_id"], active["date"]))
    assert not (removed_keys & live_keys), "a removed date is still active"

    # 4. The coverage window is contiguous — a hole means missing service.
    days = sorted(set(active["date"]))
    gaps = [b for a, b in zip(days, days[1:]) if (b - a).days > 1]
    if gaps:
        log.warning("%d gap(s) in the service calendar, first at %s", len(gaps), gaps[0])

The fourth check is the one that finds real problems. A contiguous run of dates with a hole in the middle usually means an agency removed a week of service for engineering work and forgot to add the replacement pattern — the feed is internally valid and the schedule is wrong. Detecting it is the subject of detecting service gaps and feed expiry.

Failure Modes and Edge Cases

• calendar.txt absent entirely. Legal, and common. The expansion step must return an empty frame rather than raising, and the exception layer then supplies every date. See the dedicated guide on these feeds.
• exception_type 1 outside the pattern window. Also legal. An agency running a one-off service on a date beyond end_date emits exactly this. Do not filter exceptions to the calendar window; the window bounds the pattern, not the service.
• A service_id in trips.txt that appears in neither calendar file. This is a broken feed: those trips can never run. Quarantine them and report, rather than silently dropping or silently keeping them.
• start_date after end_date. Produces zero dates. Worth an explicit error, because the silent result — a service that exists but never runs — looks identical to a service the agency deliberately suspended.
• All seven weekday flags zero, with no exceptions. The row asserts a service that never operates. Some agencies use this as a placeholder for a pattern they intend to fill in later; treat it as a warning rather than an error, but never as normal.
• Duplicate service_id rows in calendar.txt. The specification makes service_id the primary key, so this is invalid, but it does occur. Expanding both rows and deduplicating the result is the forgiving reading; reporting it is mandatory either way.
• Feeds whose coverage has already expired. A feed whose largest date is in the past describes no current service at all. Serving it produces an empty schedule that looks like a bug in your application rather than a stale feed.

Performance and Scale Notes

The row-by-row expansion above is clear and slow. On a feed with 40 service patterns spanning two years it produces around 30,000 rows in roughly 200 milliseconds — irrelevant. It becomes relevant in two situations: a multi-agency batch expanding 200 feeds in one run, and a feed whose scheduling system emits one service_id per trip, which turns 40 patterns into 30,000 and the output into millions of rows.

The vectorised form replaces the loop with a cross join against a date range:

def expand_calendar_fast(calendar: pd.DataFrame) -> pd.DataFrame:
    starts, ends = _to_date(calendar["start_date"]), _to_date(calendar["end_date"])
    lo, hi = starts.min(), ends.max()
    all_dates = pd.date_range(lo, hi, freq="D")

    frames = []
    for weekday, column in enumerate(DAY_COLUMNS):
        days = all_dates[all_dates.weekday == weekday]
        if not len(days):
            continue
        active = calendar.loc[calendar[column] == 1, ["service_id"]].assign(key=1)
        candidates = pd.DataFrame({"date": days.date, "key": 1})
        merged = active.merge(candidates, on="key").drop(columns="key")
        frames.append(merged)

    out = pd.concat(frames, ignore_index=True)
    bounds = pd.DataFrame({"service_id": calendar["service_id"],
                           "start": starts.values, "end": ends.values})
    out = out.merge(bounds, on="service_id")
    return out[(out["date"] >= out["start"]) & (out["date"] <= out["end"])][["service_id", "date"]]

Seven merges — one per weekday — replace the day-by-day loop, and the range filter is applied once at the end. On a pathological feed with 30,000 single-trip service patterns this runs in about 1.4 seconds against roughly 90 seconds for the loop. It is worth having, and it is not worth reaching for first: the loop is easier to read and easier to trust, and for most feeds it is already fast enough. If the calendar map is being rebuilt on every request rather than once per feed version, the fix is caching, not vectorisation.

For pipelines that hold the whole feed in a warehouse, materialising the expanded calendar as its own table pays for itself immediately — it turns “which trips run on 2026-12-25” from a recomputation into an indexed lookup. That table partitions naturally by service_date, which is the same partition key recommended for partitioned Parquet output.

Frequently Asked Questions

Related

• Expanding GTFS Calendar to Service Dates in Python — the complete expansion script, with the vectorised form benchmarked against the loop
• Handling calendar_dates-Only Feeds — reading a feed that omits calendar.txt altogether without special-casing it everywhere
• Detecting Service Gaps and Feed Expiry — finding the holes and the horizon before a rider does
• Modeling Holiday Service in GTFS — why holidays are the hardest calendar case and how agencies actually encode them
• Timezone Handling and Schedule Normalization — the service day, which is what a calendar date means once times past 24:00 are involved

Up: GTFS Feed Architecture & Fundamentals | Home