A data engineer is designing a database for a sensor network that generates 20,000 readings per second. The schema is fixed, and each reading must be stored with minimal write latency while preserving the exact order of arrival for later time-series analysis. Which database structure should the engineer choose?
Trap 1: Document store with each sensor's readings embedded in a single…
Embedding readings in a growing document causes the document to be rewritten on every update, and document size limits will eventually be exceeded. Concurrent appends also create write contention on the same document. This approach cannot sustain 20,000 writes per second and does not guarantee the strict ordering needed for time-series analysis.
Trap 2: Graph database with sensor nodes connected by reading edges
A graph database excels at traversing complex relationships, but each reading would become a node or edge with its own storage and index overhead. At 20,000 readings per second, graph traversal structures add unnecessary cost, and the model does not naturally preserve linear arrival order for time-series queries, making it a poor fit.
Trap 3: Relational table with a composite primary key on sensor ID and…
A relational table with a composite key can enforce uniqueness and order logically, but row-by-row inserts at 20,000 per second introduce index maintenance and locking overhead. This design is better suited to moderate write volumes where transactional integrity matters more than raw ingest throughput, so it does not meet the minimal-latency requirement for this sensor stream.
- A
Document store with each sensor's readings embedded in a single growing document
Why it fails: Embedding readings in a growing document causes the document to be rewritten on every update, and document size limits will eventually be exceeded. Concurrent appends also create write contention on the same document. This approach cannot sustain 20,000 writes per second and does not guarantee the strict ordering needed for time-series analysis.
- B
Append-only log-structured store keyed by timestamp and sensor ID
An append-only log-structured store writes sequentially to disk, avoiding random I/O and in-place index updates, which keeps write latency low at high ingest rates. Keying entries by timestamp and sensor ID preserves arrival order and enables efficient range scans for time-series analysis, directly satisfying both the throughput and ordering requirements described.
- C
Graph database with sensor nodes connected by reading edges
Why it fails: A graph database excels at traversing complex relationships, but each reading would become a node or edge with its own storage and index overhead. At 20,000 readings per second, graph traversal structures add unnecessary cost, and the model does not naturally preserve linear arrival order for time-series queries, making it a poor fit.
- D
Relational table with a composite primary key on sensor ID and timestamp
Why it fails: A relational table with a composite key can enforce uniqueness and order logically, but row-by-row inserts at 20,000 per second introduce index maintenance and locking overhead. This design is better suited to moderate write volumes where transactional integrity matters more than raw ingest throughput, so it does not meet the minimal-latency requirement for this sensor stream.