Erasure Coding
Practice this topic in a realistic system design interview
Disks and servers fail all the time in large storage systems, so data has to be stored in a way that survives those failures. The simplest approach is to keep multiple full copies. That works, but at large scale it becomes very expensive.
Erasure coding protects data with much less extra storage. Instead of storing full copies, it splits data into chunks, computes extra recovery chunks called parity chunks, and spreads all of them across different disks and servers. If some chunks are lost, the system rebuilds the original data from the ones that remain.
In this chapter, we will look at how erasure coding works, how it protects data against disk and server failures, and why large storage systems use it instead of simply keeping multiple copies of everything.