05.05 · Concept
Streaming Features for ML
Design streaming feature computations for counters, rolling windows, session attributes, and real-time personalization.
Streaming ML features are continuously maintained values keyed by user, account, device, product, or session, updated from incoming events and read at prediction time. Good designs separate event time from processing time, handle late data and replays, keep training and serving semantics aligned, and reserve online computation for signals whose freshness changes decisions.
What this lesson answers
- how to design streaming features for machine learning
- event time versus processing time for ML features
- when to use rolling windows for personalisation
Notes
Streaming features are feature values computed continuously from event streams so that machine learning systems can react to the most recent user, entity, or system behavior. Unlike batch features, which may be updated hourly or daily, streaming features are designed for low-latency use cases such as fraud detection, recommendations, ranking, dynamic pricing, alerting, or personalization.
Common questions
- What makes a feature worth computing in a stream?
- A streaming feature is worth it when recent behaviour materially improves the model decision and the definition is stable enough to operate safely. If freshness does not change the prediction, compute it in batch. Strong candidates are counters, rolling aggregates, session state, and immediate intent signals used by fraud, ranking, alerting, or personalisation systems.
- How should late or out-of-order events affect feature values?
- Late and out-of-order events need an explicit policy, not ad hoc correction. Define whether features are based on event time or arrival time, set an allowed lateness rule, and make replay behaviour deterministic. The same rule must apply when generating training data and when serving predictions, otherwise the model sees inconsistent feature semantics.
- Should all personalisation features be computed in real time?
- No. Real-time personalisation usually works best as a mix of stable batch features and a small set of fresh online signals. Long-lived preferences, profile attributes, and historical aggregates can be precomputed. Streaming should capture immediate intent, such as current session behaviour or the latest viewed category, where low latency changes the ranking outcome.
Short definition: what is Streaming Features for ML?