PDE Designing Data Processing Systems Practice Question
A financial services firm runs a batch risk calculation on Dataproc. The job reads from Cloud Storage, processes data in memory, and writes results to BigQuery. The job must complete within a 2-hour window each night, and the cluster must be shut down automatically after completion to minimize cost. You want to orchestrate this with minimal operational overhead. What should you do?
⚠ Common exam trap
The trap here is assuming that any scheduler plus a cluster deletion script is equivalent, when only a workflow template provides native completion-based cluster deletion.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Create a Dataproc workflow template that runs the job and deletes the cluster upon completion, and trigger it with a Cloud Scheduler job that calls the Dataproc API.
Dataproc workflow templates natively support running a job on a cluster that is deleted automatically when the workflow completes, satisfying the auto-shutdown and cost requirements. Triggering the template with Cloud Scheduler provides a simple, serverless schedule. Persistent clusters, Composer, or fixed-delay deletion add cost or operational complexity without the same guarantee of completion-based cleanup.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Create a Dataproc workflow template that runs the job and deletes the cluster upon completion, and trigger it with a Cloud Scheduler job that calls the Dataproc API.
Why this is correct
Dataproc workflow templates can define a job and a cluster that is automatically deleted after the workflow finishes, which meets the auto-shutdown requirement. Cloud Scheduler can invoke the Dataproc API on a cron schedule, providing orchestration with minimal operational overhead. This combination directly addresses both the scheduling and cost cleanup needs.
- ✗
Submit the job to an ephemeral Dataproc cluster created with the gcloud dataproc clusters create command, and use a Cloud Function triggered by Cloud Scheduler to delete the cluster after a fixed 2-hour delay.
Why it's wrong here
This approach ties cluster deletion to a fixed delay rather than actual job completion, risking premature deletion if the job runs long or unnecessary cost if it finishes early. It also requires custom code in a Cloud Function, increasing operational overhead. A workflow template handles completion-based deletion natively.
- ✗
Use a persistent Dataproc cluster and submit the job via a cron job on the master node, relying on the cluster's idle timeout to shut it down.
Why it's wrong here
A persistent cluster incurs cost even when idle, and relying on idle timeout can leave the cluster running longer than needed. The requirement is to minimize cost by shutting down after completion, which a persistent cluster does not guarantee. This approach also adds operational overhead of managing the cluster and cron on the master node.
- ✗
Use Cloud Composer to create a Dataproc cluster, submit the job, and delete the cluster in a DAG, scheduling it daily.
Why it's wrong here
Cloud Composer can orchestrate the cluster lifecycle, but it introduces additional infrastructure and management overhead compared to a workflow template. While it meets the functional requirements, it is not the minimal-overhead solution. The question asks for minimal operational overhead, and Composer requires maintaining an Airflow environment.
Quick reference
Cloud Service Model Comparison
| Model | You Manage | Provider Manages | Examples |
|---|---|---|---|
| IaaS | OS, runtime, apps, data | Hardware, hypervisor, networking | EC2, Azure VMs, GCP Compute Engine |
| PaaS | Apps and data | OS, runtime, middleware, hardware | Elastic Beanstalk, Azure App Service |
| SaaS | Data and settings only | Everything else | Microsoft 365, Salesforce, Workday |
| FaaS / Serverless | Function code only | Infra, scaling, runtime | Lambda, Azure Functions, Cloud Run |
| CaaS | Containers and apps | Kubernetes, OS, hardware | EKS, AKS, GKE |
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 747 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Google Cloud exam blueprint
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.