Google ACE Practice Question: A data engineering team is deploying a streaming…

A data engineering team is deploying a streaming Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. They need to ensure that each event is processed exactly once, even in the event of failures. Which Dataflow feature should they use?

⚠ Common exam trap

Google Cloud often tests the misconception that exactly-once processing requires explicit configuration or external deduplication, when in fact Dataflow provides it as a default behavior for supported sources and sinks.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Rely on Dataflow's exactly-once processing guarantees

Dataflow's streaming engine provides built-in exactly-once processing guarantees for sources like Pub/Sub and sinks like BigQuery. This is achieved through a combination of checkpointing, deterministic replay, and idempotent writes, ensuring that each record is processed exactly once even during worker failures or pipeline updates. No additional configuration or external deduplication is required.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Enable at-least-once delivery on the Pub/Sub subscription
Why it's wrong here
Pub/Sub subscriptions already provide at-least-once delivery by default, so explicitly enabling this option does not alter the pipeline's correctness. At-least-once delivery permits duplicate messages when a subscriber acknowledges late or after a redelivery timeout, and Dataflow's exactly-once processing is achieved upstream of that by using checkpointed input state and deduplication, not by changing the Pub/Sub delivery guarantee.
✗
Set the Dataflow pipeline to use the 'exactly_once' parameter in the pipeline options
Why it's wrong here
Dataflow does not expose a pipeline option named 'exactly_once'; passing such a flag to the Dataflow runner would either be ignored or fail validation, because exactly-once processing is built into the Beam/Dataflow execution engine rather than being a user-toggleable parameter. The semantics are enforced automatically through Dataflow's consistent checkpointing and commit protocol, so the correct action is simply to rely on that native guarantee, not to set a nonexistent configuration.
✓
Rely on Dataflow's exactly-once processing guarantees
Why this is correct
Dataflow's streaming engine provides exactly-once processing guarantees by checkpointing pipeline state and input cursors into durable cloud storage and by committing each input element exactly once to that state. Because Dataflow is built on the Apache Beam model, it handles deduplication automatically even when the source, such as Pub/Sub, has at-least-once delivery semantics; no additional pipeline settings are required.
✗
Use Cloud Functions to deduplicate messages before sending to Dataflow
Why it's wrong here
Inserting Cloud Functions as a dedup front-end would require sharing a mutable deduplication store (for example Bigtable or Firestore) across invocations, and since Cloud Functions itself is invoked at-least-once, two invocations could both pass the dedup check and re-publish the same message. This adds latency, cost, and distributed-consistency complexity while providing no guarantee of exactly-once that Dataflow does not already have internally. The native pattern is to let Dataflow's stateful deduplication handle it in the pipeline.

Go deeper

Google ACE Practice Question: A data engineering team is deploying a streaming…

Why each option matters

Option-by-option breakdown

Related to this question