If you design the data model poorly for Bigtable or Firestore, your database will become slow and expensive, and your application will frustrate users. Understanding how to model data for these two NoSQL databases is crucial because the Google Professional Cloud Database Engineer exam tests whether you can choose the right pattern for the right job, and because real-world systems rely on these decisions every day.
Jump to a section
A simple way to picture Schema Design for Bigtable and Firestore: NoSQL Data Modeling
A librarian is in charge of organising a massive library where readers ask for books in very different ways. Some readers want a single story by its exact title, like a novel. Others want every book by a specific author, or every book published in a certain year. The librarian knows she must arrange the books on shelves so that common questions can be answered instantly without flipping through every single page.
For the first kind of question (exact title lookup), she uses a magic filing cabinet that gives her the exact book as long as she knows a single unique ID, like 'JKR-HP1'. No matter how many books are added, she can find that one book in milliseconds. This is like Firestore, where each document has a unique ID, and if you know that ID you can retrieve the document instantly.
For the second kind of question (searching by author or year, or scanning a range of IDs), she uses a special recipe book where every recipe is written on a card, and the cards are stuck in order along a long spiral spine. To find all recipes by a certain chef, she just flips to where those cards start, then reads forward until they stop. But to make this fast, she must carefully choose the order in which the cards appear. This is like Bigtable, where you design the row key so that related data is stored next to each other on the disk, enabling super-fast scans.
Let us start with a core truth: traditional databases (called relational databases, like MySQL or PostgreSQL) store data in tables with rows and columns, and they use a language called SQL to ask questions. NoSQL databases are different. They do not use tables with fixed columns. Instead, they use flexible structures like documents, key-value pairs, or wide-column stores. Bigtable and Firestore are two NoSQL databases offered by Google Cloud. They are both powerful, but they are designed for completely different jobs. If you try to use one like the other, you will run into trouble.
Firestore is a document database. Think of it as a giant cloud-based filing cabinet. Each folder is called a 'document', and it contains data in key-value pairs. For example, a document representing a user might have the keys 'first_name', 'last_name', and 'email', with values like 'Alice', 'Smith', and 'alice@example.com'. Documents live inside 'collections'. A collection is like a drawer in the filing cabinet that holds many similar documents. So you might have a collection called 'Users' that contains one document for every user.
Firestore also supports 'subcollections'. A subcollection is a collection that lives inside a single document. For example, inside the document for user 'Alice', you could have a subcollection called 'Orders' that contains documents for each order Alice has placed. This lets you organise data in a hierarchical way that matches real-world relationships.
Now, how do you find data quickly? Firestore automatically creates 'indexes' for every field you query on. An index is like the index at the back of a textbook: it tells the database exactly where to find every occurrence of a value, so it does not have to read the whole book. You can also create 'composite indexes' for queries that filter on multiple fields at once.
Bigtable is a wide-column database. Think of it as a giant, always-sorted spreadsheet that never stops growing. Each row has a unique 'row key', and the rows are stored in alphabetical order by that key. The most important design decision you will make in Bigtable is choosing the row key. If you choose it well, related data ends up close together on disk, and queries that scan a range of rows are lightning fast. If you choose it poorly, all your data gets scattered and every query becomes slow.
Bigtable also has 'column families'. A column family is a group of related columns that are stored together on disk. For example, you might have a column family called 'profile_data' containing columns like 'name' and 'age', and another called 'metrics' containing columns like 'cpu_usage' and 'memory_usage'. Column families help you organise data and control which data is read from disk.
Here is a key difference to remember: Firestore is ideal for applications where you need to read and write individual documents, such as a user profile or a single chat message. Bigtable is ideal for applications that need to process large amounts of time-series data, such as IoT sensor readings, financial trading logs, or analytics data, where you need to scan millions of rows quickly.
Now, let us talk about the most common traps in the PCDE exam regarding these two databases:
The first trap is forgetting that Firestore queries are limited by the indexes you create. If a query does not use an existing index, Firestore will reject it. You cannot just query on any field without warning.
The second trap is assuming you can do complex joins across collections in Firestore. You cannot. You must design your data to be denormalised, meaning you duplicate data across documents to avoid joins.
The third trap is designing a Bigtable row key that causes 'hotspotting'. If you use a timestamp as the first part of the row key, all new writes go to the same region of the table, overwhelming one node. You must prefix the key with something that spreads writes evenly, like a user ID or a hash.
The fourth trap is forgetting that Bigtable is not great for small, frequent lookups of individual rows by a key. Firestore is much better for that.
To summarise the core model:
Firestore: Collections contain Documents. Documents can have Subcollections. Indexes make queries fast. Good for user-facing apps.
Bigtable: Rows are sorted by Row Key. Column Families group related columns. Row key design is everything. Good for analytical and time-series workloads.
Identify the workload type
First, decide whether the workload is primarily for low-latency reads and writes of individual records (favour Firestore) or for high-throughput scanning of large datasets over time (favour Bigtable). This is the foundational decision that guides everything else.
Design the row key for Bigtable
If using Bigtable, choose a row key that distributes writes evenly across nodes to avoid hotspotting. Typically, prefix the key with a high-cardinality field like a user ID or sensor ID, followed by a reversed timestamp if you need to query by time. Avoid starting the key with a timestamp, as that concentrates all new writes on one node.
Define column families in Bigtable
Group columns that are often read together into the same column family. For example, place all metadata columns in one family and all time-series metric columns in another. This optimises disk reads because Bigtable can skip reading entire column families that are not requested.
Plan collections, documents, and indexes in Firestore
If using Firestore, draw the hierarchy: collections at the top level, then documents, then optional subcollections. Decide which queries will be common, and create composite indexes for every query that filters on more than one field. Without a composite index, the query will fail at runtime.
Denormalise data for Firestore
Because Firestore does not support joins, duplicate relevant data across documents when needed. For instance, if you need to show a user's name alongside their order in a list, store the user's name inside the order document, even though it also exists in the user document. This ensures fast reads without extra queries.
Test and monitor performance
After implementing the schema, run realistic query workloads to ensure that the design meets latency and throughput requirements. For Bigtable, monitor the cluster's CPU utilisation to verify that hotspotting is not occurring. For Firestore, monitor the index usage report to see if any queries are being rejected due to missing indexes.
Iterate based on real-world usage
No schema is perfect from the start. As application usage patterns evolve, adjust row keys (Bigtable) or add new indexes and restructure documents (Firestore) to maintain performance. Remember that Bigtable row keys cannot be changed without rewriting the entire table, so careful upfront design is critical.
An IT professional working as a data engineer at a smart-home company is tasked with building two new systems: one for storing user profiles and home configurations, and another for storing temperature sensor readings from millions of devices.
For the user profiles system, the engineer chooses Firestore. Here is the step-by-step process:
The engineer creates a collection called 'Users'. Each document in this collection represents a single user. The document ID is the user's unique email address.
Inside each user document, there is a field called 'preferences', which is a nested map (a dictionary) containing the user's temperature settings and alarm schedule.
The engineer realises that customers often search for users by their last name or by their city. So he creates a composite index on the fields 'last_name' and 'city'. This makes those queries fast.
Inside each user document, he creates a subcollection called 'Devices'. Each document in that subcollection represents a smart thermostat or sensor owned by that user. This keeps the data organised and makes it easy to retrieve all devices for a specific user.
The engineer tests the system by simulating a user updating their home address. He finds that a single document read and write takes less than 200 milliseconds, which is perfect for the mobile app.
For the temperature sensor readings system, the engineer chooses Bigtable. Here is the step-by-step process:
The engineer decides on a row key format: 'sensor_id#reverse_timestamp'. The reverse timestamp ensures that the most recent reading from a sensor appears near the top of the table, because Bigtable sorts rows alphabetically. For example, a row key might look like 'sensor_001#20241015T143000Z'. By putting the sensor ID first, all readings from the same sensor are stored together on disk.
He defines two column families: 'metadata' (containing columns like 'firmware_version' and 'battery_level') and 'temperature_reading' (containing columns like 'celcius_reading' and 'humidity').
He sets up a cluster with three nodes to handle the write load. Because the row key starts with the sensor ID, the writes are spread evenly across the nodes, preventing hotspotting.
The engineer writes a data pipeline that streams raw sensor data from Cloud Pub/Sub into Bigtable using a Dataflow job. The job formats the data into the correct row key and column structure.
Later, a data analyst runs a query to find all temperature readings from a specific sensor over the last hour. Because all those rows are stored contiguously on disk (due to the sorted row key), Bigtable scans them in milliseconds.
Through this process, the engineer demonstrates how choosing the right database and designing the schema correctly directly impacts performance, cost, and developer productivity. The PCDE exam expects you to be able to make exactly these decisions.
The PCDE exam tests your ability to distinguish between when to use Bigtable and when to use Firestore, and how to model data correctly for each. Here is exactly what you need to know:
The exam loves to test the concept of 'row key design' for Bigtable. Expect multiple-choice questions that give you a scenario and ask which row key pattern is best. The most common patterns are: - 'field1#field2#field3' where the leading field spreads writes evenly. - Using a hash prefix to avoid hotspotting. - Using a reversed timestamp so that recent data appears first.
The traps in Bigtable questions usually involve:
Choosing a row key that has the timestamp first, which causes hotspotting on writes.
Choosing a row key that puts all data from one user in the same row, which can exceed Bigtable's maximum row size (100 MB).
Forgetting that Bigtable does not support transactions that span multiple rows.
For Firestore, the exam focuses on: - 'Collections', 'Documents', and 'Subcollections' and when to use each. - 'Indexes': You must remember that Firestore requires an index for every query that filters on multiple fields. The exam will ask you to identify whether a given query will work or not based on the indexes defined. - 'Data denormalisation': The exam will test if you know that you should duplicate data in Firestore to avoid needing joins.
The traps in Firestore questions usually involve:
Thinking you can query across subcollections of different documents easily. You can, but you must use collection group queries, which have their own indexing requirements.
Forgetting that Firestore has a maximum document size of 1 MB. You cannot store large binary files in a document.
Confusing 'subcollections' with 'nested maps'. They are different: a nested map is just a field inside a document, whereas a subcollection is a separate collection linked to a specific document.
Key definitions to memorise: - 'Hotspotting': When all write traffic hits a single Bigtable node because the row key does not distribute data evenly. - 'Denormalisation': Storing the same data in multiple places to avoid joins. - 'Composite index': An index on multiple fields used for queries that filter on those fields. - 'Column family': A group of columns in Bigtable that are stored together on disk. - 'Row key': The unique identifier for each row in Bigtable, which determines the order and distribution of data.
Finally, the exam often asks you to map a real-world requirement (like 'store one million time-series data points per second') to the correct database. The rule of thumb is: if you need high throughput writes and scans, use Bigtable. If you need low-latency reads of individual documents, use Firestore. If you need transactions or complex queries across multiple entities, use Firestore (but be aware of its limitations).
Firestore uses collections to group documents, and you cannot perform SQL-style joins across different collections, so you must duplicate data to answer multi-entity queries.
Bigtable row key design is the single most important factor for performance: a poorly designed key causes hotspotting and slow scans, while a good key enables lightning-fast data retrieval.
Firestore automatically creates indexes on single fields, but composite indexes for multi-field queries must be created manually before you can run those queries.
Bigtable does not support secondary indexes; you can only efficiently retrieve data by scanning a range of rows using the row key.
Use Firestore for low-latency, user-facing features like user profiles, chat messages, and real-time updates, and use Bigtable for high-throughput analytical workloads like IoT sensor data and financial logs.
Column families in Bigtable allow you to group related columns together on disk, which improves read performance when you only need certain categories of data per row.
The maximum size of a single Bigtable row is 100 MB, so you must ensure that designs that store large amounts of data per row (like all user events) do not exceed this limit.
These come up on the exam all the time. Here's how to tell them apart.
Firestore (Document Store)
Data is stored as documents inside collections, each with a unique ID.
Automatically indexes all single fields; composite indexes need manual creation.
Best for low-latency reads and writes of individual records, e.g., user profiles.
Bigtable (Wide-Column Store)
Data is stored as rows sorted by a row key, with columns grouped into families.
Does not automatically index any fields; queries rely on row key scans.
Best for high-throughput scans of large datasets, e.g., time-series sensor data.
Composite Index (Firestore)
An index on multiple fields, e.g., ['last_name', 'city'].
Required for queries that filter on more than one field.
Must be created manually and counts toward a maximum of 200 composite indexes.
Single-Field Index (Firestore)
An index on a single field, e.g., ['email'].
Automatically created for every field in every document.
Supports queries that filter on that one field only.
Subcollection (Firestore)
A separate collection of documents attached to a parent document.
Can be queried independently using collection group queries.
Useful for hierarchical data like 'User -> Orders'.
Nested Map (Firestore)
A field inside a single document that contains a dictionary or list.
Cannot be queried across documents; only usable within the parent document.
Useful for storing small, structured data like user preferences.
Column Family (Bigtable)
A group of related columns stored together on disk.
Controls read performance: you can read a whole family or skip it.
Example: 'metadata' family with columns 'firmware_version' and 'battery_level'.
Column (Bigtable)
A single data cell within a row, belonging to a column family.
Stored individually but physically co-located with other columns in the same family.
Example: a specific column like 'cpu_usage' under the 'metrics' family.
Hotspotting (Bad Row Key)
Occurs when all new writes go to the same Bigtable node.
Caused by a row key that starts with a monotonically increasing value like a timestamp.
Leads to performance bottlenecks and underutilisation of cluster nodes.
Even Distribution (Good Row Key)
Occurs when writes are spread evenly across all Bigtable nodes.
Achieved by prefixing the row key with a high-cardinality field like a user ID or hash.
Ensures optimal read and write performance across the cluster.
Mistake
NoSQL databases like Bigtable and Firestore do not need schema design at all, so you can just store data however you want.
Correct
Both Firestore and Bigtable have strict schema requirements. Firestore requires careful indexing and denormalisation, while Bigtable requires careful row key design and column family planning. Poor schema design leads to slow queries and high costs.
The word 'NoSQL' makes beginners think there are no rules, but each NoSQL database has its own set of design constraints that are different from SQL, not absent.
Mistake
You can use Firestore and Bigtable interchangeably for any workload because they are both NoSQL databases.
Correct
Firestore is optimised for low-latency reads and writes of individual documents in user-facing applications. Bigtable is optimised for high-throughput scans of time-series and analytical data. They serve completely different use cases.
Beginners see the label 'NoSQL' and assume all NoSQL databases are the same, ignoring the fundamental architectural differences between document stores and wide-column stores.
Mistake
In Bigtable, you can query on any column just as easily as in Firestore because both are indexed by default.
Correct
Bigtable does not automatically index columns. You can only efficiently query Bigtable by scanning rows using the row key. There is no secondary index. Filtering on a column value without scanning the entire table is extremely slow.
People are used to relational databases and Firestore, where indexes are created automatically. Bigtable's lack of secondary indexes is a major architectural difference that surprises beginners.
Mistake
Subcollections in Firestore are just a way to nest data inside a document, so they are the same as having a nested map field.
Correct
Subcollections are separate collections that can be queried independently, even across documents (using collection group queries). Nested maps are fields inside a single document and cannot be queried across documents. The two structures behave very differently and use different API calls.
The visual hierarchy looks similar, so beginners think they are interchangeable, but the query capabilities and performance characteristics are vastly different.
Mistake
If I add an index to Firestore for every possible query, my database will be perfectly fast and never have issues.
Correct
Too many indexes increase write latency and cost because each write must update every index. Firestore limits the number of composite indexes per database (default 200). You must design indexes judiciously based on actual query patterns.
Beginners think 'more indexes = faster', but in reality, indexes are a trade-off between read speed and write performance/cost.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Technically yes, but it is not recommended. Bigtable is not optimised for point lookups by an arbitrary field like email. Firestore is much better for that because it automatically indexes individual fields and can retrieve a single document in under 10 milliseconds.
A nested object is a field inside a single document, and you cannot query across documents by values inside nested objects. A subcollection is a separate collection of documents tied to a specific parent document, and you can query all documents across subcollections using collection group queries.
Firestore requires a composite index for queries that use equality filters on multiple fields. Single-field indexes only support queries that filter on one field at a time. Without the composite index, Firestore does not know how to efficiently combine the results.
You can add a hash prefix, such as the first few characters of a SHA-256 hash of the timestamp, or use a field with high cardinality like a user ID as the leading part of the row key. This distributes the writes across nodes rather than sending them all to one.
Yes, Bigtable is designed for extremely high throughput. With the right row key design and a sufficiently large cluster, it can handle millions of writes per second. This is one of its primary advantages over databases like Firestore.
Firestore automatically creates indexes for single-field queries. But for any query that uses multiple equality filters, a range filter, or an order by on a different field than the filter, you must manually create a composite index before the query will work.
You've finished Schema Design for Bigtable and Firestore: NoSQL Data Modeling. Continue through the PCDE study guide to build a complete picture of the exam.
Done with this chapter?