Why is cold start a distribution rather than a single value?

Because each cold invocation may take a different path through placement, sandbox creation, runtime startup, artefact loading, and initialisation. A worker with cached image layers behaves differently from one that must fetch them. A burst may require many new sandboxes at once. Reporting only an average hides the slow tail that users often notice.

What is the difference between cold and warm invocation?

A warm invocation reuses an execution environment that already has its sandbox, runtime, and loaded code, so the platform mostly dispatches the event and runs the handler. A cold invocation must prepare some or all of that environment first. Warm does not mean free of latency, but it removes much of the boot path.

Does moving code out of the handler help cold starts?

It depends on what you move. Module-level initialisation runs before the handler on a cold start, so heavy imports, top-level await, model loading, or database connections there increase cold-start latency. Moving reusable setup outside the handler can help warm requests, but it can also make the first request slower and amplify burst connection pressure.

How do provisioned concurrency and minimum instances work internally?

They keep execution environments prepared before a request arrives. The platform performs sandbox creation, runtime startup, and code loading ahead of time, then routes traffic to those ready environments. This changes the latency distribution, especially the tail, but it is not magic: if traffic exceeds prepared capacity, new cold environments may still be needed.

Serverless Internals

Serverless internals are the platform mechanisms that allocate isolated compute, start a language runtime, load user code, run initialisation, and dispatch an event to a handler. They explain why an invocation may be fast when reused, slow when newly created, and variable across requests rather than having a single cold-start time.

Serverless exists to hide machines, but the machine has not disappeared. Before user code can run, the provider must place the invocation somewhere safe, isolate it from other tenants, attach the right runtime, and make the function’s files available. If that environment already exists, the request can be dispatched quickly. If it does not, the platform has to build enough of it on the request path, which is what engineers experience as a cold start.

A cold invocation is a chain of concrete steps: scheduling onto a worker, creating or reusing a sandbox, starting the runtime, loading the package or image, executing module-level code, then calling the handler. On Lambda that sandbox may be a Firecracker microVM with Node.js 20 or Python 3.12 attached. On container-based platforms, image layers may need to be fetched and unpacked. Top-level imports, database connections, and TLS setup happen before the handler if your code asks for them there.

The main trade-off is between operational simplicity and less control over latency shape. Cold start is commonly misunderstood as one number, but it is a distribution: placement, cache state, image size, runtime startup, networking, and burst concurrency all change the tail. Provisioned capacity or minimum instances move much of the boot work before the request, improving the slow cases, but you pay for readiness and still need to size it against actual traffic patterns.

Engineers meet these internals when production graphs show good median latency but poor tail latency after idle periods, deploys, regional cache misses, or sudden fan-out. They also appear when logs inside the handler omit time spent in imports or global initialisation. In practice, diagnosis means separating platform boot, runtime startup, package loading, module initialisation, and handler work, then choosing smaller artefacts, lazier initialisation, provisioned capacity, or a different execution model such as isolates.

Go deeper

DeploymentServerless InternalsDescribe what actually boots when a function is invoked, and why cold start is a distribution rather than a number.Open the lesson

Common questions

Why is cold start a distribution rather than a single value?: Because each cold invocation may take a different path through placement, sandbox creation, runtime startup, artefact loading, and initialisation. A worker with cached image layers behaves differently from one that must fetch them. A burst may require many new sandboxes at once. Reporting only an average hides the slow tail that users often notice.
What is the difference between cold and warm invocation?: A warm invocation reuses an execution environment that already has its sandbox, runtime, and loaded code, so the platform mostly dispatches the event and runs the handler. A cold invocation must prepare some or all of that environment first. Warm does not mean free of latency, but it removes much of the boot path.
Does moving code out of the handler help cold starts?: It depends on what you move. Module-level initialisation runs before the handler on a cold start, so heavy imports, top-level await, model loading, or database connections there increase cold-start latency. Moving reusable setup outside the handler can help warm requests, but it can also make the first request slower and amplify burst connection pressure.
How do provisioned concurrency and minimum instances work internally?: They keep execution environments prepared before a request arrives. The platform performs sandbox creation, runtime startup, and code loading ahead of time, then routes traffic to those ready environments. This changes the latency distribution, especially the tail, but it is not magic: if traffic exceeds prepared capacity, new cold environments may still be needed.

Every term