Storage at Google

by Mr Jos Visser (Google)

Tuesday 13 Sept 2011, 14:00 → 15:30 Europe/Zurich

IT Auditorium (CERN)

Abstract:

Google sits on some of the largest repositories of data in the world:
detailed satellite imagery of the entire planet (Google Earth), more video than was ever broadcast on television since its invention (Youtube), entire libraries of scanned books (Google Books), hi-res omni-directional pictures of a large part of the western world
(Streetview) and of course a copy of the World Wide Web (Search). To store, access, update and backup that gigantic amount of information Google created a set of technologies ranging from a platform-independent way of representing structured data (protocol buffers) to its own file system (GFS) and even a distributed replicated database Bigtable). This talk covers these technologies and discusses their design rationales and limitations.

Biography:

Jos Visser is a staff software engineer in Google's Storage Site Reliability Engineering group.

 

Bigtable: A Distributed Storage System for Structured Data http://labs.google.com/papers/bigtable.html

Chubby http://labs.google.com/papers/chubby.html

MapReduce framework http://labs.google.com/papers/mapreduce.html

More papers from Googlers http://research.google.com/pubs/papers.html

Protocol Buffers http://code.google.com/p/protobuf/

The Google File System http://labs.google.com/papers/gfs.html http://www.theregister.co.uk/2009/08/12/google_fi le_system_part_deux/