Skip to content
Klay

All papers

System Design28 min read

Large-scale cluster management at Google with Borg

Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, et al.2015EuroSys 2015

Read it on research.google(opens in a new tab)
  • scheduling
  • containers
  • operations

Why this one

If Kubernetes has ever felt like a pile of strange nouns, pods, jobs, services, requests, limits, controllers, this is the paper that makes the shapes less arbitrary. Borg is not interesting because Google had a big cluster. It is interesting because it treats cluster management as a product surface for thousands of engineers, not as a clever bin-packing script. The key idea is that scheduling, failure recovery, naming, rollout control, and resource isolation are one system, because production does not let you debug them separately. People often read Borg as Kubernetes origin lore and miss the more useful lesson: abstractions are allowed to be imperfect if they make operations predictable at fleet scale. Read this after The Google File System and MapReduce: Simplified Data Processing on Large Clusters, because Borg is the missing layer that explains how those systems actually get placed, restarted, and shared.

What to take away

  • Borg separates long-running services from batch jobs, then schedules both on the same machines to raise utilization.
  • Resource requests are contracts with the scheduler, while actual usage is measured and controlled after placement.
  • The paper explains why Kubernetes has controllers, declarative desired state, pods, labels, and service discovery.

Reads with