Bigtable Basics: NoSQL Wide-Column Storage for Low-Latency Workloads

Bigtable is a NoSQL (Not Only SQL) wide-column database that Google designed to handle massive amounts of data with very low latency. NoSQL means it does not use the traditional table-and-join model of relational databases like MySQL or PostgreSQL. Instead, it trades away complex queries for horizontal scaling and speed. Wide-column means each row can have a different number of columns, and columns can be grouped into families. While a relational database forces every row to have the same columns, Bigtable lets rows be sparse—most cells are empty—which saves enormous storage space.

Every piece of data in Bigtable is addressed by three things: a row key, a column family, a column qualifier, and a timestamp. The row key is a unique identifier for the row, like a user ID or a device serial number. Bigtable stores all rows sorted by row key. This sort order is crucial because it determines how fast you can look up data. If your row key is designed well, a point lookup—asking for one exact row—completes in under 10 milliseconds. If you design the row key poorly, your queries can become slow scans that touch many rows you didn’t need.

Column families are groups of related columns that share the same storage settings. For example, you might have a column family called ‘metadata’ for things like user name and email, and another called ‘activity’ for pageview timestamps and click counts. All column families must be created before you can store data in them. Inside a column family, you can have any number of column qualifiers (the actual column name). You don’t need to define them in advance. This flexibility is what makes Bigtable schema-less within a family.

The timestamp allows Bigtable to store multiple versions of the same cell. If you update a user’s profile picture URL, the old URL isn’t immediately deleted—it stays as a previous version with a different timestamp. You can configure how many versions to keep, usually between 1 and a few hundred. This is extremely useful for auditing or showing historical data without building a separate versioning system.

Bigtable is built on top of three core Google technologies: Colossus (the distributed file system), Borg (the cluster management system), and Chubby (the distributed lock service). Colossus stores all the data on SSDs across thousands of machines. Borg schedules the work of serving that data onto the right machines. Chubby ensures that only one machine can serve as the master for a given tablet (a contiguous range of rows). These components together give Bigtable its high availability and automatic scaling. You don’t need to think about resharding; Bigtable splits and merges tablets automatically as data grows or shrinks.

When you write data, Bigtable first writes to a commit log (for durability), then to a memtable (an in-memory buffer). Periodically, the memtable is flushed to a sorted immutable file called an SSTable on Colossus. Read requests first check the memtable, then the SSTables. This architecture is what makes writes very fast (because they’re sequential writes) and reads fast when the row key is known (because the row is in a predictable location).

Bigtable is not designed for relational queries, joins, transactions across multiple rows, or ad-hoc analytics. Those workloads are better served by Cloud Spanner (for global strong consistency and SQL) or BigQuery (for analytics). Bigtable shines when you need to serve high-throughput, low-latency data to applications like ad serving, recommendation engines, IoT time-series data, and financial trading systems.

For PCDE, you must understand the relationship between row key design and performance. A monotonically increasing row key like ‘timestamp’ (e.g., 2024-01-01-00-00-00-001) causes all new writes to hit the same tablet server, creating a hotspot. A properly designed key spreads writes evenly across the cluster. One common pattern is to use a field-level prefix (like user ID) followed by a secondary field, or to reverse the key. For time-series data, prefixing with a hashed value or a geographic shard key distributes load.

An IT professional working at a large e-commerce company is tasked with building a real-time inventory tracking system. The company has 5 million products across 200 warehouses. Each time a warehouse scans a product, the system needs to record the event and allow a storefront dashboard to show the current stock level within 5 seconds. The existing relational database cannot handle the peak of 50,000 writes per second during Black Friday. The engineer chooses Bigtable.

First, she designs the row key. A naive approach would be ‘product_id + timestamp’ (e.g., product_789123_2025-11-27-10-30-00). This would cause all writes for product_789123 to hit one tablet server, creating a hotspot. Instead, she uses ‘warehouse_id + product_id’ (e.g., wh_42_product_789123). This spreads writes across all warehouse IDs. Because Bigtable stores rows sorted by row key, all events for a single warehouse are stored contiguously, which makes it fast to scan all products in one warehouse for reporting.

She creates two column families: ‘stock’ and ‘event’. The ‘stock’ family has column qualifiers like ‘current_count’, ‘reserved_count’, and ‘last_updated’. The ‘event’ family has one column per scan event, with the column qualifier being a scan ID and the value being a JSON blob of metadata. She sets the garbage collection policy on the ‘stock’ family to keep only the latest version (versions: 1), because only the current stock matters. On the ‘event’ family, she keeps 100 versions, because the warehouse team wants to audit the last 100 scans for any product.

When the system writes a new inbound scan, it increments the ‘current_count’ cell. When a customer buys the product, it decrements. The frontend dashboard queries Bigtable every 2 seconds using the warehouse_id + product_id key. With Bigtable’s consistent snapshot reads, the dashboard sees the latest committed write within milliseconds. If the dashboard accidentally requests a row that doesn’t exist (a product never scanned), the read still returns quickly—around 10 ms—because Bigtable returns an empty result instead of scanning.

The engineer configures the Bigtable cluster with 10 nodes during normal load, but uses the autoscaling feature to grow to 50 nodes during Black Friday. She sets a CPU utilisation target of 60% so that Bigtable adds nodes before the system becomes saturated. She also enables replication across two regions for disaster recovery, accepting an eventual consistency window of a few seconds between regions.

In this scenario, the biggest operational risk is a poorly designed row key. If she had used a monotonically increasing key (like a globally incrementing scan ID), all new writes would hit the last tablet server, causing overload and timeouts. She proactively tests the schema with a load simulator that writes 100,000 rows per second before go-live, ensuring the cluster doesn’t exceed 70% CPU utilisation at the peak rate.