11.06 · Concept
Online Serving and Low-Latency Access
Design an online feature serving path using entity keys, freshness constraints, cache behavior, and latency budgets.
Online feature serving is the request-time path that turns entity keys into model-ready values within a latency budget. The design depends on freshness requirements, cache policy, missing-value handling, and the cost of each network or storage hop, especially when tail latency affects user-facing decisions.
What this lesson answers
- how to design online feature serving
- how fresh should online features be
- how should feature store caching work
Notes
Online serving is the path that retrieves features at request time for a user-facing decision, such as ranking, fraud checks, recommendations, or personalization. Unlike offline feature generation, the serving path is constrained by a strict latency budget and must return values for a specific entity key, such as user_id, account_id, merchant_id, or item_id. A good design starts by defining the entities the model needs, the features available for each entity, and the maximum acceptable staleness for each feature.
Common questions
- What is online feature serving?
- Online feature serving is the low-latency retrieval path used when a live request needs model inputs. The service receives or derives entity keys, fetches the required feature values, applies defaults or fallbacks where needed, and returns a feature vector quickly enough for a user-facing ranking, fraud, recommendation, or personalisation decision.
- How do freshness requirements affect feature serving design?
- Freshness requirements decide where each feature should come from. Very fresh values may need streaming updates or request-time computation. Less volatile values can be precomputed, stored, or cached. The key design choice is whether an older value is acceptable for the product decision, and what should happen when the freshest source is unavailable.
- Why is cache policy important for online features?
- A cache can lower load and improve slow-request behaviour, but it changes the data contract. Time-to-live settings must match feature freshness needs, and miss handling must be explicit. Teams should decide whether to serve stale values, fall back to defaults, retry another store, or fail the request when cached data is absent.
Short definition: what is Online Serving and Low-Latency Access?