1. Home
  2. GTFS Realtime Integration
  3. Handling GTFS Service Alerts
  4. Parsing Service Alerts In Python

Parsing GTFS Service Alerts in Python

Explode each alert’s informed_entity[] into one pandas row per affected route_id, stop_id, or trip_id, carrying the alert-level fields down onto every row; select the English TranslatedString with a language search that falls back to the untagged and then first translation; and convert each active_period bound from a POSIX integer to a UTC-aware datetime, mapping the protobuf default of 0 to NaT rather than a 1970 timestamp. The result is a tidy long-format DataFrame you can filter, join, and store directly.

Root Cause Analysis

A GTFS-Realtime alert is a deeply nested protobuf object, and the friction in parsing it comes from three specific mismatches with tabular analysis. The parent handling GTFS service alerts guide covers the message semantics in full; this page is the narrow, self-contained fix for turning one alert feed into one DataFrame.

The first mismatch is cardinality. A single Alert names many informed_entity selectors, so an alert is inherently a one-to-many structure. Flattening it into a frame means choosing long format — one row per (alert, selector) — and repeating the alert-level fields across those rows, so each affected entity is individually queryable.

The second is the protobuf zero-value trap. Protocol Buffers has no concept of null for scalar fields; an unset string reads as "" and an unset int reads as 0. For an active_period bound, that 0 is indistinguishable from a real timestamp unless you test HasField first — and epoch 0 renders as 1970-01-01, a value that silently poisons any time-window comparison.

The third is language ambiguity in TranslatedString. The translation[] list has no defined order and languages are optional, so reading element zero is a coin flip between English, French, or an untagged string. Correct extraction is an explicit search, not an index.

Production-Ready Python Implementation

The script below reads a raw protobuf payload, explodes the alerts, resolves text and enums, converts active periods to UTC-aware datetimes, and returns a tidy DataFrame. It depends only on gtfs-realtime-bindings and pandas.

python 
import logging
from datetime import datetime, timezone

import pandas as pd
from google.transit import gtfs_realtime_pb2

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")

Alert = gtfs_realtime_pb2.Alert

# Explicit column dtypes for the tidy output frame
ALERT_DTYPES = {
    "alert_id": "string",
    "cause": "string",
    "effect": "string",
    "header_text": "string",
    "description_text": "string",
    "agency_id": "string",
    "route_id": "string",
    "route_type": "Int64",
    "stop_id": "string",
    "trip_id": "string",
}


def _enum_name(enum_cls, value: int, fallback_prefix: str) -> str:
    """Resolve a protobuf enum integer to its name, tolerating unknown values."""
    try:
        return enum_cls.Name(value)
    except ValueError:
        return f"{fallback_prefix}_{value}"


def _pick_text(translated_string, language: str = "en") -> str | None:
    """Return the text for `language`, else the untagged, else the first translation."""
    translations = list(translated_string.translation)
    if not translations:
        return None
    for candidate in (
        [t for t in translations if t.language == language],
        [t for t in translations if not t.language],
        translations,
    ):
        if candidate:
            return candidate[0].text
    return None


def _epoch_to_utc(has_field: bool, epoch_value: int):
    """Convert a POSIX seconds value to a UTC-aware datetime, or NaT if unset."""
    if not has_field:
        return pd.NaT
    return datetime.fromtimestamp(epoch_value, tz=timezone.utc)


def parse_service_alerts(raw_protobuf: bytes, language: str = "en") -> pd.DataFrame:
    """
    Parse a GTFS-Realtime ServiceAlerts payload into a tidy long-format DataFrame.

    One row per (alert, informed_entity) selector. Active-period bounds are
    UTC-aware datetimes; unset bounds are NaT. Alerts with multiple active
    periods produce one row per (selector, period) combination.
    """
    feed = gtfs_realtime_pb2.FeedMessage()
    feed.ParseFromString(raw_protobuf)

    rows: list[dict] = []
    for entity in feed.entity:
        if not entity.HasField("alert"):
            continue
        alert = entity.alert

        header = _pick_text(alert.header_text, language)
        description = _pick_text(alert.description_text, language)
        cause = _enum_name(Alert.Cause, alert.cause, "UNRECOGNIZED_CAUSE")
        effect = _enum_name(Alert.Effect, alert.effect, "UNRECOGNIZED_EFFECT")

        # Normalise active_period; empty list => one always-active window
        periods = list(alert.active_period) or [None]

        selectors = list(alert.informed_entity)
        if not selectors:
            logging.warning("Alert %s has no informed_entity; emitting agency-wide row", entity.id)
            selectors = [None]

        for selector in selectors:
            base = {
                "alert_id": entity.id,
                "cause": cause,
                "effect": effect,
                "header_text": header,
                "description_text": description,
                "agency_id": (selector.agency_id or None) if selector is not None else None,
                "route_id": (selector.route_id or None) if selector is not None else None,
                "route_type": (
                    selector.route_type
                    if selector is not None and selector.HasField("route_type")
                    else pd.NA
                ),
                "stop_id": (selector.stop_id or None) if selector is not None else None,
                "trip_id": (
                    (selector.trip.trip_id or None)
                    if selector is not None and selector.HasField("trip")
                    else None
                ),
            }
            for period in periods:
                row = dict(base)
                row["active_start"] = _epoch_to_utc(
                    period is not None and period.HasField("start"),
                    period.start if period is not None else 0,
                )
                row["active_end"] = _epoch_to_utc(
                    period is not None and period.HasField("end"),
                    period.end if period is not None else 0,
                )
                rows.append(row)

    alerts = pd.DataFrame(rows).astype(ALERT_DTYPES)
    logging.info(
        "Parsed %d alerts into %d entity rows",
        alerts["alert_id"].nunique() if not alerts.empty else 0,
        len(alerts),
    )
    return alerts


if __name__ == "__main__":
    with open("feeds/service_alerts.pb", "rb") as handle:
        payload = handle.read()
    alerts_df = parse_service_alerts(payload, language="en")
    print(alerts_df.head(10).to_string(index=False))

Step-by-Step Walkthrough

The nested loop is the explode. The outer loop walks FeedEntity objects and skips any without an alert (checked with HasField, so all-default alerts survive). The middle loop walks informed_entity selectors, and the inner loop walks active_period windows. Emitting a row at the innermost level is what turns one nested Alert into many tidy rows — one per (selector, period) pair.

_pick_text searches, never indexes. It tries three candidate lists in priority order — the requested language, then untagged translations, then everything — and returns the first non-empty match. This is the correct reading of a TranslatedString; indexing translation[0] would return whichever language the producer happened to serialise first.

_epoch_to_utc guards the zero-value trap. It only converts when the caller confirms HasField, otherwise returning pd.NaT. That is what separates a genuinely unbounded window from the protobuf default 0 that would otherwise render as 1970-01-01. The conversion uses tz=timezone.utc because the POSIX timestamp is an absolute instant; keeping it UTC-aware makes downstream comparisons unambiguous, deferring any conversion to the agency’s local zone until display, exactly as timezone handling and schedule normalization recommends.

_enum_name tolerates future spec revisions. Alert.Cause.Name(value) raises ValueError on an integer the bindings do not know. Wrapping it and returning UNRECOGNIZED_CAUSE_<n> means one unfamiliar enum from a newer feed never aborts the whole parse — a resilience concern that mirrors the philosophy behind GTFS validation rules.

Explicit dtypes on the output. Casting to ALERT_DTYPES gives route_type a nullable Int64 and every identifier the pandas string type, so a downstream join against static route data does not silently coerce route_id to a float.

Verification and Output

After parsing, assert the frame’s structural invariants before using it:

python 
import pandas as pd

alerts_df = parse_service_alerts(payload, language="en")

# Every row belongs to an alert and carries resolved text
assert alerts_df["alert_id"].notna().all(), "Rows without alert_id present"
assert alerts_df["cause"].notna().all(), "Unresolved cause enum present"

# No active-period bound may be the epoch-0 sentinel
epoch_zero = pd.Timestamp("1970-01-01", tz="UTC")
assert (alerts_df["active_start"] != epoch_zero).all(), "Unset start leaked as 1970"
assert (alerts_df["active_end"] != epoch_zero).all(), "Unset end leaked as 1970"

# At least one identifying field per row (or a logged agency-wide row)
id_cols = ["agency_id", "route_id", "route_type", "stop_id", "trip_id"]
identified = alerts_df[id_cols].notna().any(axis=1)
print(f"{identified.sum()} of {len(alerts_df)} rows name a specific entity")

print(alerts_df[["alert_id", "effect", "route_id", "stop_id", "active_start", "active_end"]].head())

A healthy parse of a feed with, say, 40 alerts averaging three selectors each yields roughly 120 rows, every one carrying a resolved cause, effect, and English header_text, with active_start/active_end either a real UTC timestamp or NaT — never 1970-01-01.

Gotchas and Edge Cases

  • Multiple active periods multiply rows. An alert with two active_period windows and three selectors produces six rows. That is intentional for time-window filtering, but if you need one row per selector, collapse the periods into a list column with groupby(...).agg(list) after parsing rather than dropping windows.

  • route_type selectors carry no route_id. An alert scoped by route_type (e.g. “all ferries suspended”) sets route_type and leaves route_id null. Do not treat the null route_id as missing data — resolve the affected routes by matching route_type against routes.txt, as the route-type mapping guide describes.

  • trip selectors without a trip_id. A selector’s TripDescriptor may identify a run by route_id + start_time + start_date instead of trip_id, leaving trip_id null even though the selector is trip-scoped. Preserve the full TripDescriptor if you need to resolve those against the schedule via matching realtime to static schedules.

  • Empty feeds are valid. A feed with no active alerts is a normal, conformant response, not an error. The function returns an empty DataFrame; guard the .nunique() call (as the code does) so an empty frame does not raise.

Frequently Asked Questions

How do I get one row per affected entity from a service alert?

Iterate the alert’s informed_entity list and emit a dict per selector, repeating the alert-level fields (id, cause, effect, text) on each row. This normalizes the nested protobuf into a tidy long-format DataFrame where every route_id, stop_id, or trip_id gets its own record.

Why is my active_period showing 1970 dates?

A start or end of 0 is the protobuf default for an unset field, and epoch 0 renders as 1970-01-01. Test each bound with HasField before converting, and store an unset bound as NaT rather than converting the literal zero.

Should active_period times be stored as UTC or local time?

Store them as UTC-aware datetimes because the POSIX timestamps are absolute instants. Convert to the agency timezone only at display time, using the agency_timezone from agency.txt, so comparisons and storage stay unambiguous.