Cloud Composer DAG Retry for Transient Failures — Dataflow, Quota Limits

Your team manages a production ML pipeline on Google Cloud that trains a fraud detection model every 6 hours using new transaction data. The pipeline steps are: (1) Cloud Function triggered by new files in Cloud Storage to validate data, (2) Dataflow job for feature engineering, (3) Vertex AI CustomJob for training, (4) Cloud Function to deploy the model to a Vertex AI endpoint after evaluation. You notice that the pipeline sometimes fails during the Dataflow job step with an error: 'Workflow failed. Causes: The job encountered a system error. Please try again later.' The error occurs sporadically, and retrying the pipeline manually usually succeeds. The team needs a reliable automated solution. What should you do?

Quick Answer

The answer is to orchestrate the pipeline using Cloud Composer with retry policies on the Dataflow operator. This is correct because Cloud Composer, built on Apache Airflow, allows you to define a Cloud Composer DAG retry for transient failures by setting parameters like `retries` and `retry_delay` directly on the Dataflow operator, which automatically re-launches the job when it encounters sporadic system errors like the one described. On the Google Professional Machine Learning Engineer exam, this scenario tests your understanding of production ML pipeline reliability and the distinction between transient versus permanent failures—a common trap is to over-engineer a solution with custom error handling or manual triggers when a simple retry policy suffices. Remember that for quota limit or system errors in Dataflow, the fix is often a retry, not a redesign. Memory tip: think "Composer retries compose the solution for transient Dataflow sighs."

⚠ Common exam trap

Many candidates confuse scaling solutions (Option C) with fault-tolerance mechanisms, or they choose a generic queuing service (Option B) instead of a dedicated orchestrator with built-in retry policies for pipeline steps.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Orchestrate the pipeline using Cloud Composer with retry policies on the Dataflow operator.

Cloud Composer (Apache Airflow) provides native retry policies on its Dataflow operators, enabling automatic retries of the Dataflow job when it fails due to transient system errors. This addresses the sporadic failure pattern without manual intervention, ensuring the pipeline runs reliably every 6 hours.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Schedule the pipeline to run less frequently to reduce load on the Dataflow service.
Why it's wrong here
Reducing frequency does not fix the sporadic errors and reduces model freshness.
✗
Use Cloud Tasks to queue the Dataflow job and retry on failure.
Why it's wrong here
Cloud Tasks is for asynchronous task execution, not orchestrating a multi-step pipeline with dependencies.
✗
Increase the number of Dataflow workers and use flexRS to handle transient errors.
Why it's wrong here
Transient system errors are not resolved by scaling resources; the job still may fail.
✓
Orchestrate the pipeline using Cloud Composer with retry policies on the Dataflow operator.
Why this is correct
Cloud Composer (Airflow) can manage the pipeline DAG with automatic retries and dependencies.

Visual reference

About these practice questions

Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →

How Courseiva writes practice questions · Editorial policy

Same concept, more angles

1 more way this is tested on PMLE

These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.

Variation 1. A team is using Cloud Composer to orchestrate ML workflows. They have a DAG that triggers a Vertex AI Training job, then a prediction deployment. The deployment step occasionally fails due to quota limits. What is the best way to handle this?

medium

A.Increase the quota manually
B.Use Vertex AI Pipelines instead of Cloud Composer
C.Create a custom sensor to wait for quota to be available
D.Catch the exception in the DAG and send an alert
✓ E.Implement exponential backoff retry in the DAG task

Why E: Cloud Composer (Apache Airflow) provides built-in retry mechanisms via task parameters like `retries` and `retry_delay`. Implementing exponential backoff in the DAG task is the best practice for handling transient quota errors, as it automatically retries the deployment step with increasing delays, reducing load on the quota system and increasing the chance of success without manual intervention. This approach aligns with Airflow's native error-handling capabilities and avoids unnecessary complexity or resource waste.

Last reviewed: Jun 11, 2026

Question Discussion

Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.

Loading comments…

This PMLE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PMLE exam.