Long-Running Servers
A long-running server is a persistent application process that stays alive to handle many requests over its lifetime. Unlike a short-lived function invocation, it can keep memory warm, reuse network connections, and run background activity between requests, while accepting that this local state disappears when the process is restarted.
The need appears when each request would otherwise pay the full startup cost again. Creating database sessions, rebuilding caches, loading configuration, initialising SDK clients, or starting helper threads can dominate simple request handling. Some work also does not fit neatly inside a single request, such as polling a queue, refreshing credentials before expiry, or maintaining a WebSocket session. A persistent process gives the application somewhere to keep that work and state alive.
Concretely, the operating system runs a process containing your HTTP server, event loop, thread pool, heap, and open file or socket descriptors. Requests enter the same process repeatedly. A cache in a Node.js `Map`, a Postgres connection pool, or an OAuth client object can be created once and reused. A background worker can tick on a timer while request handlers continue serving traffic, rather than being recreated for each event.
The trade-off is that process-local state is convenient but not reliable or global. If the pod, container, or VM is restarted, its memory, in-process queue, and open sessions are gone. If you run several replicas, each has its own separate memory, so counters, caches, and locks can disagree unless they live in Redis, Postgres, or another shared system. Background loops also need bounds, shutdown handling, and health checks, or they become hidden production load.
Engineers meet long-running servers in Kubernetes Deployments, Cloud Run container instances, Rails, FastAPI, Express, Go HTTP services, Envoy, and database connection pooling. The common misunderstanding is to treat them as durable storage or as a single global process. The honest rule is: use the process for reusable, disposable, per-instance state, and use an external system for anything that must survive restarts or be consistent across replicas.
Common questions
- How is a long-running server different from a serverless function?
- A function is usually designed around one event at a time and may be frozen or removed after it finishes. A long-running server expects to stay alive and serve many requests. That lets it keep caches, clients, connection pools, and workers in memory, but only while that particular process remains running.
- Can I store user data or jobs in server memory?
- Only if losing it is acceptable. Server memory is process-local and disappears on restart, rescheduling, deployment, or crash. It is fine for derived caches, temporary coordination, and reusable clients. It is not fine for durable queues, payments, user records, locks, or anything that must be visible to every replica.
- Do long-running servers always perform better?
- It depends on what your application spends time doing. They help when reuse matters, such as database handshakes, TLS setup, cache warming, or client initialisation. They do not remove the need for scaling, backpressure, connection limits, or external storage, and a badly managed persistent process can leak memory or keep broken sockets.
- What should happen to background work during shutdown?
- The server should stop accepting new work, signal workers to finish or checkpoint, close pools, and hand unfinished durable work back to an external queue or database. In-process jobs should be treated as interruptible. If the work must complete exactly once, the ownership and retry logic need to live outside the process.