PDE Ingesting and Processing the Data Practice Question
Your team is processing a large dataset with Apache Beam on Dataflow. The pipeline sometimes fails due to transient errors when writing to a BigQuery sink. You need to ensure that failed records are not lost and can be reprocessed later without blocking the pipeline. What is the best approach?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a side output (e.g., via TupleTag) to write failed records to a dead letter sink (e.g., GCS or Pub/Sub) and continue processing the main output.
Using a dead letter pattern with a side output to write failed records to a GCS bucket (or Pub/Sub) allows the pipeline to continue processing healthy records while failed records are stored for later analysis and reprocessing.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Configure the pipeline to use at-least-once semantics and rely on Dataflow to retry the entire bundle.
Why it's wrong here
This can cause duplicate processing and does not isolate failed records; the entire bundle may be retried, causing delays.
- ✗
Increase the number of workers to reduce the chance of transient errors.
Why it's wrong here
More workers do not prevent transient errors from the sink; they only improve throughput.
- ✗
Use a try-catch block in the DoFn and log the error; continue processing other elements.
Why it's wrong here
Logging alone does not persist the failed records for reprocessing; they are lost if the pipeline fails or logs are rotated.
- ✓
Use a side output (e.g., via TupleTag) to write failed records to a dead letter sink (e.g., GCS or Pub/Sub) and continue processing the main output.
Why this is correct
This pattern isolates bad records, allows the pipeline to continue, and stores the failed records for later reprocessing.
Go deeper
Related to this question
About these practice questions
This PDE question is part of Courseiva's 890-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.