Communications of the ACM
Web Applications Crave Memory
Google not only keeps all of their search indexes in memory, but also keeps the entire Web in memory. Google has to do this to return search results with snippets back quickly.
Facebook caches everything (or almost everything) in memory. Facebook has to do this to render pages complete with news feeds quickly.
LinkedIn keeps their entire social graph in memory. LinkedIn has to do that to show nearby connections with real-time updates and to rank order search results by network distance.
Most web applications want rapid random access to many pieces of indexed data. They have only a hundred milliseconds or so to get what they need.
Only RAM and its slower but persistent cousin SSD can do that. For lots of data to be accessed fast enough for most web applications, it needs to be sitting in memory.
What most web applications need is commodity clusters with lots of low power and low cost RAM. As for CPU, they only need something cheap and low power, just enough to filter some data as they yank it out of memory. And they only need disk to get started when they wake up.
But manufacturers keep adding more and more cores and asking for more and more power. They keep building bigger and bigger disks spinning faster and faster, eating more and more power.
Big web applications using commodity clusters are hungry for RAM. Give us servers loaded with lots and lots of low power RAM. And little else.
Comments
Rakesh Barve
Greg,
Interesting article, thanks. I had a couple of questions:
(a) Something like Gmail search -- which requires per user indices -- presumably could still involve search from secondary memory. In general, if per user data is large and relevant only when that user is actively searching, then that would largely still reside on secondary memory and loaded on-demand for in-RAM indexing, or could rely on a secondary memory index ?
(b) You briefly mention SSD -- Given that random access on SSDs is still orders of magnitudes slower, I was wondering in what situations can they offload RAM.
Rakesh
Greg Linden
Hi, Rakesh. On GMail search, I am not sure about the particulars of Google's implementation but, talking more generally, it doesn't take a lot of cache misses that cause you to hit disk before the delays become intolerable. No doubt that many still use the well-worn strategy of using RAM just as a cache for disk, but, as in these examples, moving all the data to memory often is necessary to hit performance targets.
As for SSD, the most common use I've seen is as a secondary storage under a RAM caching layer (often with three layers RAM->SSD->disk, see for example Facebook's Flashcache). It's a faster but more expensive version of disk when used this way. In my opinion, its value is limited to allowing you to tolerate more cache misses, but others may have different opinions on that.
Tushar Gaikwad
As more and more sites store more data, the appetite for getting more memory will only increase. This will also force the disk manufacturers to get better products and will eventually bring new products to the market.
Displaying all 3 comments