Design Refinements in Bigtable

To fulfill our users’ high performance, availability, and reliability requirements, the system described in the preceding lessons needs to be refined in several ways. This lesson goes into further depth about specific implementation components to emphasize these improvements.

Locality groups

Another abstraction in our data model is the locality group, which is a partition of the columns. Recall that tables are partitioned horizontally where each shard is called a tablet. A locality group can be viewed as a vertical partition and what it does is combine the data in a specific set of columns (column families). Columns that are in different locality groups are stored separately from one another.

Regarding the implementation, different locality groups mean different files on GFS. Bigtable generates separate SSTables for each locality group, which gives clients the ability to control where their data is stored.

It also provides various other knobs, and a particular one that many clients have found helpful is that clients can state whether or not a locality group should be memory-mapped. That is, whether or not the server should serve the data directly out of memory rather than reading from the underlying Google File System.