mediumMultiple Choice
PDE Practice Question: A data pipeline uses Cloud Composer to…
A data pipeline uses Cloud Composer to orchestrate Dataflow and BigQuery jobs. The pipeline fails intermittently with dependency errors. Which design change can improve reliability?
⚠ Common exam trap
Google Cloud often tests the distinction between scaling compute resources (Dataflow workers) and improving orchestration reliability (retries), leading candidates to mistakenly choose option C when the problem is transient task failures, not resource bottlenecks.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use retries with exponential backoff
Cloud Composer (Apache Airflow) tasks can fail due to transient issues like API rate limits or resource contention. Implementing retries with exponential backoff allows the DAG to automatically re-attempt failed tasks with increasing delays, reducing the impact of intermittent failures without manual intervention. This is a standard Airflow pattern for improving reliability in orchestrated pipelines.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use retries with exponential backoff
Why this is correct
Intermittent dependency errors arise when upstream Dataflow or BigQuery jobs finish late, so retries with exponential backoff let dependent tasks re-attempt after transient delays rather than failing outright. This directly addresses the unreliable cross-service dependencies named in the stem, raising pipeline reliability.
- ✗
Switch to Cloud Functions for orchestration
Why it's wrong here
Cloud Functions is event-driven and stateless, so it cannot express the task dependencies, retries and scheduling that Cloud Composer's Airflow DAGs provide — exactly what the intermittent dependency failures require. It is tempting because Cloud Functions suits lightweight, single-trigger glue work, such as reacting to a Cloud Storage object arrival, where no orchestration graph exists.
- ✗
Increase worker count in Dataflow
Why it's wrong here
Increasing Dataflow worker count only raises parallel processing capacity; it cannot resolve dependency ordering between Composer tasks, Dataflow jobs and BigQuery loads, which is what causes the intermittent failures. It is tempting because horizontal scaling genuinely helps throughput-bound pipelines, and would be correct where a single Dataflow job is the bottleneck rather than orchestration sequencing.
- ✗
Use a simpler DAG with fewer dependencies
Why it's wrong here
Reducing dependency count does not resolve the underlying race conditions or missing task dependencies causing intermittent failures; it merely masks symptoms by removing orchestration logic. It tempts because simplifying DAGs genuinely helps when complexity itself causes maintenance burden or scheduling overhead. Here, the correct fix is explicit task dependencies and retries, not fewer tasks.
Go deeper
Related to this question
About these practice questions
This PDE question is part of Courseiva's 747-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.