hardMultiple SelectObjective-mapped
PDE Ephemeral clusters Practice Question
A company is migrating their on-premises Apache Spark jobs to Google Cloud Dataproc. They want to minimize operational overhead and cost for jobs that run only a few times per day. Which TWO strategies should they adopt? (Choose TWO.)
⚠ Common exam trap
A common mistake is to think that a persistent cluster is needed for data durability or to avoid job startup latency. However, for jobs that run only a few times per day, ephemeral clusters with Cloud Storage are more cost-effective and operationally simpler.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Store all data in Cloud Storage instead of HDFS, and use the Cloud Storage connector to access it.
Storing data in Cloud Storage decouples storage from compute, allowing ephemeral clusters to be spun up and down without data loss. The Cloud Storage connector provides Hadoop-compatible file system access, eliminating the need for HDFS replication and reducing costs by avoiding persistent cluster storage. Option D is correct because ephemeral Dataproc clusters are created per job and deleted after completion, which minimizes cost and operational overhead for intermittent workloads, as there is no need to maintain a persistent cluster. Options A and B are incorrect: A proposes HDFS replication, which is unnecessary when using Cloud Storage, and B suggests rewriting jobs as Dataflow pipelines, which is not a required strategy for the stated goal of minimizing overhead for existing Spark jobs. Option E is incorrect because a persistent cluster incurs continuous costs and operational overhead, which is not optimal for jobs that run only a few times per day.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Configure HDFS replication factor to 3 to ensure data durability during cluster restarts.
Why it's wrong here
Incorrect. HDFS replication factor of 3 is unnecessary when using Cloud Storage as the primary storage because Cloud Storage provides data durability and availability by default. Moreover, HDFS is ephemeral if using ephemeral clusters, so replication does not persist after cluster deletion. This approach does not minimize operational overhead or cost.
- ✗
Rewrite the Spark jobs as Dataflow pipelines to take advantage of serverless processing.
Why it's wrong here
Incorrect. While Dataflow is serverless, rewriting existing Spark jobs as Dataflow pipelines would require significant redevelopment effort and is not necessary to minimize overhead for occasional jobs. The question specifically asks for strategies for existing Spark jobs on Dataproc.
- ✓
Store all data in Cloud Storage instead of HDFS, and use the Cloud Storage connector to access it.
Why this is correct
Correct. Storing data in Cloud Storage decouples storage from compute, enabling ephemeral clusters. The Cloud Storage connector provides Hadoop-compatible access, eliminating HDFS overhead and reducing cost because storage is billed separately and persists beyond cluster lifetime.
- ✓
Create an ephemeral Dataproc cluster for each job and delete it after completion.
Why this is correct
Correct. Ephemeral clusters are created per job and deleted after completion. This minimizes cost because you only pay for compute during job execution, and operational overhead is low because no persistent cluster management is needed.
- ✗
Use a small persistent cluster that runs continuously and submit jobs to it.
Why it's wrong here
Incorrect. A small persistent cluster incurs continuous costs even when idle, which is not cost-effective for jobs that run only a few times per day. It also requires ongoing management.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 890 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.