Tracking Vehicle Positions in Realtime

A GTFS-Realtime VehiclePositions feed is the moving layer that sits on top of a static schedule: a stream of coordinates, headings, and occupancy readings that tell you where every bus, tram, and train actually is right now. Turning that stream into something an operations dashboard or an arrival-prediction service can use means decoding the protobuf VehiclePosition message, flattening thousands of entities into a tabular structure, and joining each vehicle back to the routes.txt and trips.txt rows that give it meaning. This guide, part of the GTFS-Realtime Integration area, walks through that pipeline end to end for Python engineers building live tracking on production feeds.

The Live-Position Problem

A raw VehiclePositions feed is deliberately sparse. To keep payloads small enough to poll every 10–30 seconds, agencies transmit only what has changed and only the fields their AVL (automatic vehicle location) hardware populates. One entity may carry latitude, longitude, bearing, speed, and occupancy; the next may carry nothing but a coordinate pair and a vehicle.id. The protobuf wire format compounds the sparseness: absent fields are simply not present, so naive code that reads entity.vehicle.position.speed gets a silent default of 0.0 rather than an error, and a dashboard quietly reports a moving fleet as stationary.

The second half of the problem is context. A coordinate on its own is not actionable — an operator needs to know that vehicle 7421 is the 08:14 departure of route M15, running the northbound pattern, currently 60% full. That context lives entirely in the static feed. Recovering it means resolving the realtime trip descriptor against the schedule, which is straightforward when trip.trip_id is present and considerably harder when it is not. The workflow below builds the happy path first, then points to the two narrow techniques that handle the awkward feeds.

Prerequisites

Before running the code in this guide, make sure the following are in place:

• Python 3.9+ with the official protobuf bindings, pandas, and requests:

pip install gtfs-realtime-bindings pandas requests

• Familiarity with how a protobuf FeedMessage decodes into entities. If you have not built the decode layer yet, start with decoding GTFS-Realtime protobuf feeds and the step-by-step walkthrough on parsing GTFS-RT protobuf with Python, which cover the gtfs_realtime_pb2 module this page builds on.
• A polling loop or scheduler that fetches the feed at a sensible cadence. See polling GTFS-Realtime feeds efficiently for conditional requests and back-off so you do not hammer the endpoint.
• A matching static feed for the same agency, unpacked to disk, with at least trips.txt and routes.txt. The join keys depend on the identifiers described in understanding GTFS static feed structure.

Concept and Spec Background

The VehiclePosition Message

In GTFS-Realtime a feed is a FeedMessage containing a header and a repeated list of FeedEntity. For a positions feed, each entity’s vehicle field holds a VehiclePosition. Only the nested position is meaningful on its own; everything else is optional and frequently omitted.

The coordinates are always WGS84 (EPSG:4326), identical to stops.txt and shapes.txt. Any metric work — snapping a fix to a route line, measuring distance between fixes — requires projecting to a metric coordinate reference system first, exactly as in the static spatial pipeline.

The Enumerations

Two enums carry most of the operational signal, and both are transmitted as integers on the wire. Decoding them to labels is a required step, not a cosmetic one.

How the Realtime Entity References the Static Feed

The trip descriptor is the bridge between the moving fix and the schedule. trip.trip_id is a foreign key into trips.txt; from there you inherit route_id, service_id, direction_id, and shape_id. trip.route_id is a foreign key straight into routes.txt. The diagram below shows the entity and the two joins that give a coordinate its meaning.

Step-by-Step Implementation

Step 1: Fetch and Decode the FeedMessage

Fetch the protobuf payload and parse it into a FeedMessage. Iterate the entities and keep only those that actually carry a vehicle position — a single feed can interleave trip_update and alert entities alongside vehicle ones.

import logging
import requests
from google.transit import gtfs_realtime_pb2

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")

VEHICLE_POSITIONS_URL = "https://example-agency.gov/gtfs-realtime/vehicle-positions.pb"


def fetch_vehicle_feed(url: str, timeout: int = 30) -> gtfs_realtime_pb2.FeedMessage:
    """Fetch and decode a GTFS-Realtime VehiclePositions feed."""
    response = requests.get(url, timeout=timeout)
    response.raise_for_status()

    feed = gtfs_realtime_pb2.FeedMessage()
    feed.ParseFromString(response.content)

    vehicle_entities = [e for e in feed.entity if e.HasField("vehicle")]
    logging.info(
        "Header timestamp %s | %d entities | %d carry a vehicle position",
        feed.header.timestamp,
        len(feed.entity),
        len(vehicle_entities),
    )
    return feed

Step 2: Flatten Entities into a Positions DataFrame

Protobuf’s default-value behaviour is the trap here: reading an unset position.speed returns 0.0, and an unset trip.trip_id returns "". Guard every optional field with HasField and emit None when it is absent, so downstream code can tell “stationary” from “unknown”. Build a list of dicts and construct the DataFrame with an explicit dtype map.

import pandas as pd

POSITION_DTYPES = {
    "vehicle_id": "string",
    "trip_id": "string",
    "route_id": "string",
    "direction_id": "Int64",
    "latitude": "float64",
    "longitude": "float64",
    "bearing": "float64",
    "speed": "float64",
    "current_status": "Int64",
    "occupancy_status": "Int64",
    "timestamp": "Int64",
}


def _opt(container, field):
    """Return a field value only if the protobuf message actually set it."""
    return getattr(container, field) if container.HasField(field) else None


def positions_to_dataframe(feed: gtfs_realtime_pb2.FeedMessage) -> pd.DataFrame:
    """Flatten VehiclePosition entities into a typed positions DataFrame."""
    records = []
    for entity in feed.entity:
        if not entity.HasField("vehicle"):
            continue
        vp = entity.vehicle
        pos = vp.position
        records.append(
            {
                "vehicle_id": vp.vehicle.id or None,
                "trip_id": _opt(vp.trip, "trip_id"),
                "route_id": _opt(vp.trip, "route_id"),
                "direction_id": _opt(vp.trip, "direction_id"),
                "latitude": pos.latitude if vp.HasField("position") else None,
                "longitude": pos.longitude if vp.HasField("position") else None,
                "bearing": _opt(pos, "bearing"),
                "speed": _opt(pos, "speed"),
                "current_status": _opt(vp, "current_status"),
                "occupancy_status": _opt(vp, "occupancy_status"),
                "timestamp": _opt(vp, "timestamp"),
            }
        )

    vehicle_positions = pd.DataFrame.from_records(records).astype(POSITION_DTYPES)
    logging.info("Built positions frame with %d rows", len(vehicle_positions))
    return vehicle_positions

Step 3: Decode the Status and Occupancy Enums

Map the integer enums to readable labels using the generated protobuf enum tables rather than hard-coded dictionaries, so the labels stay in sync with the bindings you installed. Rows where the field was absent stay NA.

from google.transit import gtfs_realtime_pb2

_VP = gtfs_realtime_pb2.VehiclePosition


def _enum_label(enum_wrapper, value):
    if pd.isna(value):
        return pd.NA
    return enum_wrapper.Name(int(value))


def decode_enums(vehicle_positions: pd.DataFrame) -> pd.DataFrame:
    """Add human-readable label columns for current_status and occupancy_status."""
    vehicle_positions = vehicle_positions.copy()
    vehicle_positions["current_status_label"] = vehicle_positions["current_status"].map(
        lambda v: _enum_label(_VP.VehicleStopStatus, v)
    ).astype("string")
    vehicle_positions["occupancy_label"] = vehicle_positions["occupancy_status"].map(
        lambda v: _enum_label(_VP.OccupancyStatus, v)
    ).astype("string")

    # Convert speed to km/h for reporting, leaving the raw m/s column intact
    vehicle_positions["speed_kmh"] = vehicle_positions["speed"] * 3.6
    return vehicle_positions

Step 4: Join to Static trips.txt and routes.txt

Load the static tables with string keys and left-join the positions frame so that vehicles with an unresolved trip_id survive the join as NA rather than being dropped. Recover route_id from trips.txt when the realtime trip.route_id is missing, then attach the route naming and route type columns from routes.txt.

from pathlib import Path


def enrich_with_static(
    vehicle_positions: pd.DataFrame, feed_dir: str
) -> pd.DataFrame:
    """Join realtime positions to static trips.txt and routes.txt."""
    feed_path = Path(feed_dir)

    trips = pd.read_csv(
        feed_path / "trips.txt",
        dtype={
            "trip_id": "string",
            "route_id": "string",
            "service_id": "string",
            "shape_id": "string",
            "trip_headsign": "string",
        },
        usecols=lambda c: c
        in {"trip_id", "route_id", "service_id", "shape_id", "trip_headsign"},
    )
    routes = pd.read_csv(
        feed_path / "routes.txt",
        dtype={
            "route_id": "string",
            "route_short_name": "string",
            "route_long_name": "string",
            "route_type": "Int64",
        },
        usecols=lambda c: c
        in {"route_id", "route_short_name", "route_long_name", "route_type"},
    )

    # Join to trips first to recover the authoritative route_id and shape_id
    enriched = vehicle_positions.merge(
        trips, on="trip_id", how="left", suffixes=("", "_static")
    )
    # Prefer the realtime route_id, fall back to the one from trips.txt
    enriched["route_id"] = enriched["route_id"].fillna(enriched["route_id_static"])
    enriched = enriched.drop(columns=["route_id_static"])

    enriched = enriched.merge(routes, on="route_id", how="left")

    resolved = enriched["trip_id"].notna().sum()
    logging.info(
        "Resolved %d/%d positions to a trip_id (%.1f%%)",
        resolved,
        len(enriched),
        100 * resolved / max(len(enriched), 1),
    )
    return enriched

The positions that fail this join — a nonzero share on many feeds — are exactly the ones the two companion guides handle. Matching vehicle positions to GTFS trips resolves a trip_id from route_id, shape proximity, and the service window when the descriptor is missing, while vehicle speed and bearing from GTFS-RT fills the speed and bearing columns that Step 2 left as NA.

Validation and Verification

Run these gates on every poll before the frame reaches a dashboard or prediction service. They catch the three failure classes that quietly corrupt live tracking: out-of-bounds coordinates, stale fixes, and silent join loss.

import time


def validate_positions(enriched: pd.DataFrame, max_age_seconds: int = 120) -> dict:
    """Assert coordinate bounds, timestamp freshness, and join coverage."""
    coords = enriched.dropna(subset=["latitude", "longitude"])
    lat_ok = coords["latitude"].between(-90.0, 90.0)
    lon_ok = coords["longitude"].between(-180.0, 180.0)
    assert lat_ok.all() and lon_ok.all(), (
        f"{(~(lat_ok & lon_ok)).sum()} positions fall outside WGS84 bounds"
    )

    now = int(time.time())
    age = now - enriched["timestamp"]
    stale = enriched[age > max_age_seconds]

    duplicate_vehicles = enriched["vehicle_id"].duplicated().sum()

    results = {
        "positions": len(enriched),
        "with_coordinates": int(coords.shape[0]),
        "resolved_trip_id": int(enriched["trip_id"].notna().sum()),
        "stale_fixes": int(len(stale)),
        "duplicate_vehicle_ids": int(duplicate_vehicles),
        "median_age_seconds": float(age.median()),
    }
    assert duplicate_vehicles == 0, (
        "Duplicate vehicle_id values in a single snapshot — dedupe on latest timestamp"
    )
    logging.info("Validation: %s", results)
    return results

A single snapshot should contain each vehicle_id at most once; duplicates mean the feed republished a vehicle or your polling loop merged two snapshots. Resolve them by keeping the row with the newest timestamp per vehicle_id.

Failure Modes and Edge Cases

• Absent fields read as zeros. The most damaging trap in the whole pipeline. An unset position.speed decodes to 0.0 and an unset occupancy_status decodes to EMPTY (0). Without HasField guards, a dashboard shows a full, moving fleet as empty and stationary. Guard every optional field and emit None.

• Header timestamp versus entity timestamp. The FeedMessage.header.timestamp marks when the feed was assembled; each entity’s own timestamp marks when that fix was measured. On feeds with lagging AVL uplinks the two diverge by minutes. Always age-filter on the per-entity value.

• trip.route_id present but trip.trip_id absent. Common on frequency-run services where the agency knows the route but not which scheduled trip a vehicle is operating. These rows join to routes.txt cleanly but not to trips.txt; route them to the trip-matching workflow rather than discarding them.

• Duplicate or recycled vehicle.id. Some agencies reuse identifiers across depots or reset them nightly. If you accumulate positions across polls, key your history on (vehicle_id, trip_id) and treat a large coordinate jump between consecutive fixes as a new assignment rather than an impossibly fast vehicle.

• occupancy_percentage without occupancy_status. Newer feeds populate the numeric percentage but leave the coarse band unset, or vice versa. Read both and prefer the percentage when present, mapping it to a band yourself for consistent display.

• Coordinate order errors on re-projection. When you later project fixes for spatial work, geometry.x is longitude and geometry.y is latitude — the reverse of how humans say “lat, lon”. A swapped pair lands the vehicle in the wrong hemisphere. The spatial analysis and route geometry guide covers the projection idiom in full.

Performance and Scale Notes

A national or multi-agency positions feed can carry tens of thousands of entities per poll, and at a 15-second cadence that is a continuous, unbounded stream. Two strategies keep it tractable.

First, build the DataFrame once per poll and avoid per-row Python loops after the initial flatten. The enum decode and the static joins are fully vectorised; the only unavoidable loop is the protobuf iteration in Step 2, which is bounded by entity count.

Second, if you retain history for speed and bearing derivation or for playback, do not keep every snapshot in a growing DataFrame. Append each poll’s frame to a partitioned Parquet dataset keyed by service date and hour, then read back only the window you need.

import pyarrow as pa
import pyarrow.parquet as pq
from datetime import datetime, timezone

def append_snapshot_parquet(enriched: pd.DataFrame, root: str) -> None:
    """Append one poll's positions to a date/hour-partitioned Parquet dataset."""
    stamped = enriched.copy()
    poll_dt = datetime.now(timezone.utc)
    stamped["service_date"] = poll_dt.strftime("%Y-%m-%d")
    stamped["poll_hour"] = poll_dt.strftime("%H")

    table = pa.Table.from_pandas(stamped, preserve_index=False)
    pq.write_to_dataset(
        table,
        root_path=root,
        partition_cols=["service_date", "poll_hour"],
    )

For the broader patterns behind chunked reads and columnar caching on large feeds, see memory-efficient processing for large feeds. When you need to correlate positions with predicted arrivals, the matching realtime to static schedules area covers the join back to stop_times.txt.

Frequently Asked Questions

Related

• Matching Vehicle Positions to GTFS Trips — resolve a trip_id and route_id when the realtime descriptor is missing
• Vehicle Speed and Bearing from GTFS-RT — derive the speed and bearing columns from consecutive fixes when the feed omits them
• Decoding GTFS-Realtime Protobuf Feeds — the protobuf decode layer this pipeline builds on
• Coordinate Reference Systems for Transit Data — projecting WGS84 fixes to metric space for any spatial work
• Spatial Analysis and Route Geometry — snapping vehicles to route lines and working with shapes.txt

Up: GTFS-Realtime Integration | Home