PDE Ingesting and Processing the Data Practice Question
A Dataflow pipeline reads from Pub/Sub, applies a keyed stateful ParDo that uses state variables to deduplicate events based on event ID, and writes to BigQuery. During a pipeline update, some events are duplicated in BigQuery. The state is not preserved across updates. Which configuration ensures exactly-once semantics during updates?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Take a snapshot of the pipeline before updating, then start the new pipeline from the snapshot.
Draining the pipeline stops it and completes processing in-flight, then the updated pipeline can start fresh. However, because state is lost, duplicates may still occur if the new pipeline processes events that were already committed. To preserve state, use snapshotting: take a snapshot before update and start the new pipeline from the snapshot. BigQuery's Storage Write API with exactly-once semantics can help at the sink but does not prevent duplicate processing if state is lost. As long as the deduplication state is recovered from the snapshot, duplicates are avoided.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Drain the pipeline and start the updated pipeline; all in-flight data will be processed.
Why it's wrong here
Draining does not preserve state; deduplication state is lost, leading to possible duplicates.
- ✗
Cancel the pipeline and restart it; Pub/Sub subscriptions will be rewound.
Why it's wrong here
Canceling loses state and the subscription may re-deliver unacknowledged messages, causing duplicates.
- ✗
Use the Storage Write API's exactly-once delivery mode.
Why it's wrong here
Ensures exactly-once writes to BigQuery but does not prevent duplicate processing if state is lost.
- ✓
Take a snapshot of the pipeline before updating, then start the new pipeline from the snapshot.
Why this is correct
Snapshots preserve the state of the pipeline, including deduplication state, allowing the new pipeline to resume without reprocessing duplicates.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 890 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.