Skip to content
Klay

All papers

System Design28 min read

The Google File System

Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung2003SOSP 2003

Read it on research.google(opens in a new tab)
  • storage
  • distributed-systems
  • fault-tolerance

Why this one

Read this before MapReduce: Simplified Data Processing on Large Clusters, because MapReduce only looks simple once GFS has made huge files, cheap machines, and constant failure feel normal. The lasting idea is not “use a master” or “make big chunks.” It is that a storage system should be designed around the workload and the failure model you actually have, not the clean POSIX file system you wish you had. People often copy the architecture and miss the bargain: weak consistency in carefully chosen places, append as a first-class operation, replication as routine maintenance, and operational repair paths that assume disks and machines are always dying. If you build data platforms, queues, blob stores, or internal infra, this paper is worth an evening because it shows how much complexity disappears when you stop pretending generality is free.

What to take away

  • GFS optimizes for huge sequential reads and appends, not small random POSIX-style updates.
  • The single master works because it keeps metadata in memory and stays off the hot data path.
  • Its consistency model is a product decision: relaxed semantics buy throughput and simpler recovery.

Reads with