hardMultiple Choice
PDE Practice Question: A data engineer is designing a batch ETL pipeline…
A data engineer is designing a batch ETL pipeline that reads CSV files from Cloud Storage, transforms them using Dataproc, and writes the results to BigQuery. The data volume is expected to grow 10x in the next year. Which design approach best balances cost and performance?
⚠ Common exam trap
Watch out — candidates often choose Dataflow (Option D) assuming it is always the best for cost and performance, but the question specifically involves Dataproc and batch ETL from Cloud Storage to BigQuery, where preemptible nodes with autoscaling provide a more direct and cost-effective solution without requiring a pipeline rewrite.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a Dataproc cluster with preemptible worker nodes and autoscaling enabled.
Preemptible worker nodes significantly reduce cost (up to 80% discount) while autoscaling dynamically adjusts cluster size to match the growing workload, ensuring performance without over-provisioning. This combination handles the 10x data growth efficiently by scaling out during peak loads and scaling in during lulls, using preemptible instances for fault-tolerant tasks like transformation.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Create a single large persistent Dataproc cluster to handle the peak load.
Why it's wrong here
A persistent cluster bills for idle nodes, so 10x growth means paying for peak capacity continuously rather than elastically. It is tempting because long-running clusters suit steady, predictable workloads where startup latency matters, but here ephemeral per-job clusters with autoscaling match variable batch volumes.
- ✗
Use Cloud Data Fusion to visually design the pipeline and run it on Dataproc.
Why it's wrong here
Cloud Data Fusion adds a graphical ETL layer and its own Dataproc-provisioned execution environment, introducing licensing and orchestration overhead without improving CSV-to-BigQuery throughput. It is tempting for code-free pipeline authoring by non-developers, but this scenario already specifies Dataproc transformations, so the abstraction adds cost rather than performance.
- ✓
Use a Dataproc cluster with preemptible worker nodes and autoscaling enabled.
Why this is correct
Preemptible worker nodes cut compute costs substantially for fault-tolerant batch ETL, while autoscaling adds or removes workers based on YARN load, matching capacity to the 10x volume growth without over-provisioning. Dataproc handles preemption by re-running lost tasks, so the pipeline stays reliable while satisfying the cost-performance balance.
- ✗
Migrate the pipeline to Dataflow with Apache Beam and use flexRS for cost savings.
Why it's wrong here
FlexRS applies only to batch Dataflow jobs and requires a six-hour minimum, so it cannot serve the pipeline's latency needs and adds scheduling delay. It is tempting because FlexRS genuinely cuts batch cost, but Dataproc on ephemeral clusters already matches this workload's cost and performance profile.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 747 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.