You are developing an IoT solution that stores device metadata (device ID, location, firmware version, last seen timestamp) in Azure Table Storage. Each device has a unique DeviceId and a Timestamp. You need to design the PartitionKey and RowKey to optimize query performance for the following query: Retrieve all firmware versions for devices in a specific city that were last seen within the last 24 hours. The query must be efficient (partition scan minimized). Which key design is most appropriate?
Trap 1: PartitionKey = City, RowKey = DeviceId_Timestamp (e.g.,…
This design allows querying by city but the row key is not sortable by timestamp, so you cannot do a range query on timestamp efficiently. You would have to scan all devices in the city and filter in the client.
Trap 2: PartitionKey = DeviceId, RowKey = Timestamp
Partitioning by DeviceId means each device's data is stored in a separate partition. While this allows for efficient queries for a single device's history using the Timestamp RowKey, it severely hinders queries that need to aggregate or filter devices based on a property like City. To find all devices in a specific city, the system would be forced to perform a full table scan across all partitions, which is highly inefficient and costly in Azure Table Storage, as it bypasses the optimized partition-level indexing.
Trap 3: PartitionKey = City, RowKey = DeviceId
This allows efficient partition query by city, but the row key is device ID. To filter by timestamp, you would need to scan all rows in the partition and apply a filter, which is not efficient if the partition has many devices.
- A
PartitionKey = City, RowKey = DeviceId_Timestamp (e.g., "device123_2023-10-01T12:00:00")
Why wrong: This design allows querying by city but the row key is not sortable by timestamp, so you cannot do a range query on timestamp efficiently. You would have to scan all devices in the city and filter in the client.
- B
PartitionKey = City, RowKey = Inverted timestamp (e.g., DateTime.MaxValue.Ticks - Timestamp.Ticks)
This design keeps all devices from the same city in one partition (efficient for city filtering). The row key, when sorted in ascending order, brings the most recent timestamps first. You can use a range query on the row key to get devices with last seen within the last 24 hours by comparing against the inverted timestamp of 24 hours ago.
- C
PartitionKey = DeviceId, RowKey = Timestamp
Why wrong: Partitioning by DeviceId means each device's data is stored in a separate partition. While this allows for efficient queries for a single device's history using the Timestamp RowKey, it severely hinders queries that need to aggregate or filter devices based on a property like City. To find all devices in a specific city, the system would be forced to perform a full table scan across all partitions, which is highly inefficient and costly in Azure Table Storage, as it bypasses the optimized partition-level indexing.
- D
PartitionKey = City, RowKey = DeviceId
Why wrong: This allows efficient partition query by city, but the row key is device ID. To filter by timestamp, you would need to scan all rows in the partition and apply a filter, which is not efficient if the partition has many devices.