How are streaming features different from batch features?

Batch features are computed over stored data on a schedule and are well suited to stable history, such as long-term spend or lifetime counts. Streaming features are updated as events arrive and are read during serving. They are useful when the model needs immediate context, but they cost more to operate and reason about.

What kinds of features are usually streamed?

Common examples include recent counters, rolling aggregates, last-seen values, current session attributes, and short-term intent signals. For example, a system might maintain recent purchase count, average transaction amount over a window, current session duration, or the most recent product category viewed. The best candidates are simple, explainable, and clearly freshness-sensitive.

What is the biggest correctness risk?

The biggest risk is mismatch between training and serving. If historical examples are built with one interpretation of time, windowing, deduplication, or lateness, while live features use another, the model is trained on different semantics from the ones it sees in production. Shared definitions, backfills, and explicit lateness rules reduce that risk.

Streaming Features for ML

Streaming features for ML are feature values maintained continuously from incoming events, keyed by an entity such as a user, account, device, product, or session. They let a model use very recent behaviour at prediction time, rather than relying only on batch-computed history that may already be stale.

The need appears when the thing being predicted changes faster than the batch pipeline. Fraud, recommendations, ranking, alerting, pricing, and personalisation often depend on what just happened: a burst of attempts, a category viewed moments ago, or a session that is still active. Without streaming features, the model may be technically correct on historical data but blind to the current state of the user or system.

Concretely, events arrive on a stream and are grouped by a key such as user, card, merchant, device, or session. For each key, the stream processor keeps state: counters, sums, last-seen values, recent items, or window contents. New events update that state, old state expires, and the latest value is written somewhere the serving path can read when it builds the model input.

The hard part is time. A rolling count, a fixed window, and a session window answer different questions, even if their names sound similar. Engineers must choose event time or processing time, define what happens to late or out-of-order events, and decide when a window is final enough. This is commonly misunderstood: streaming does not automatically mean correct, only fresher.

The trade-off is operational complexity. Streaming features require partitioned state, recovery, replay handling, duplicate protection, expiry, monitoring, and alignment with training data. The honest answer to whether a feature should be streaming is: it depends on whether freshness improves the decision enough to justify that machinery. Many systems work best with stable batch features plus a small number of valuable real-time signals.

Engineers meet streaming features in stream processors, online feature stores, model-serving code, and backfill jobs. The same definition should be usable for historical training and live serving, or the model learns one meaning and receives another in production. In practice, teams monitor freshness, null rates, volume, distribution shifts, and serving latency, because broken real-time features often fail silently.

Go deeper

Everything DataStreaming Features for MLDesign streaming feature computations for counters, rolling windows, session attributes, and real-time personalization.Open the lesson

Common questions

How are streaming features different from batch features?: Batch features are computed over stored data on a schedule and are well suited to stable history, such as long-term spend or lifetime counts. Streaming features are updated as events arrive and are read during serving. They are useful when the model needs immediate context, but they cost more to operate and reason about.
What kinds of features are usually streamed?: Common examples include recent counters, rolling aggregates, last-seen values, current session attributes, and short-term intent signals. For example, a system might maintain recent purchase count, average transaction amount over a window, current session duration, or the most recent product category viewed. The best candidates are simple, explainable, and clearly freshness-sensitive.
What is the biggest correctness risk?: The biggest risk is mismatch between training and serving. If historical examples are built with one interpretation of time, windowing, deduplication, or lateness, while live features use another, the model is trained on different semantics from the ones it sees in production. Shared definitions, backfills, and explicit lateness rules reduce that risk.

Every term