06.08 · Concept
Multi-Tenancy and LoRA
Serve many fine-tunes from one base model by swapping low-rank adapters per request, and state what that buys over one deployment per tenant.
LoRA multi-tenancy keeps one base model resident and applies tenant-specific low-rank adapters at request time. It avoids duplicating the full weight footprint for every fine-tune, preserves more batching across tenants, and turns serving policy into adapter routing, caching, eviction and placement rather than separate deployments per customer.
What this lesson answers
- how does LoRA multi tenancy work
- why use adapters instead of separate model deployments
- when does multi LoRA serving stop helping
Notes
LoRA multi-tenancy serves many tenant-specific fine-tunes by keeping one frozen base weight matrix resident and selecting a low-rank delta per request, typically with , where , , and . At inference the layer computes , often without materializing .
Common questions
- What is LoRA multi-tenancy in model serving?
- It is a serving pattern where many tenant-specific fine-tunes share the same frozen base model. Each request carries an adapter identity, and the runtime applies the matching low-rank update during the forward pass. The base weights stay hot, while adapters are cached, loaded, evicted and scheduled as smaller per-tenant artefacts.
- Why is this better than one deployment per tenant?
- Separate deployments duplicate the largest object in the system: the base model weights. LoRA multi-tenancy shares that base across tenants and varies only the adapter. That reduces memory pressure, keeps more traffic in the same batching stream, and lets the fleet route requests by adapter rather than reserving whole replicas for each tenant.
- What can make multi-adapter serving slow?
- It gets slower when too many requests in the same decode step need different adapters, because batching locality falls apart. Cold adapter loads can also dominate if adapters are large or stored off GPU. High-rank adapters, changed embedding or output layers, and strict isolation requirements can make dedicated deployments or merged hot adapters preferable.
Short definition: what is Multi-Tenancy and LoRA?