hardMultiple Select
PDE Ephemeral clusters Practice Question
A company is migrating their on-premises Apache Spark jobs to Google Cloud Dataproc. They want to minimize operational overhead and cost for jobs that run only a few times per day. Which TWO strategies should they adopt? (Choose TWO.)
⚠ Common exam trap
A common mistake is to think that a persistent cluster is needed for data durability or to avoid job startup latency. However, for jobs that run only a few times per day, ephemeral clusters with Cloud Storage are more cost-effective and operationally simpler.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Store all data in Cloud Storage instead of HDFS, and use the Cloud Storage connector to access it.
Option C is correct because storing data in Cloud Storage (GCS) and accessing it via the Cloud Storage connector decouples storage from compute, so ephemeral clusters can be deleted without losing data and you only pay for storage you actually use, which lowers cost and operational overhead. Option D is correct because ephemeral Dataproc clusters created per job and deleted on completion eliminate idle cluster costs and the patching/scaling overhead of managing long-running clusters, which fits jobs that run only a few times per day. Option A is not appropriate because HDFS replication factor 3 increases storage cost and is unnecessary when data should live in GCS for durability. Option B is not appropriate because rewriting Spark jobs as Dataflow pipelines is a significant re-engineering effort and not required to reduce Dataproc operational overhead. Option E is not appropriate because a continuously running persistent cluster incurs idle compute costs and ongoing management overhead, contradicting the goal of minimizing cost and operations for infrequent jobs.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Configure HDFS replication factor to 3 to ensure data durability during cluster restarts.
Why it's wrong here
Incorrect. HDFS replication factor of 3 is unnecessary when using Cloud Storage as the primary storage because Cloud Storage provides data durability and availability by default. Moreover, HDFS is ephemeral if using ephemeral clusters, so replication does not persist after cluster deletion. This approach does not minimize operational overhead or cost.
- ✗
Rewrite the Spark jobs as Dataflow pipelines to take advantage of serverless processing.
Why it's wrong here
Incorrect. While Dataflow is serverless, rewriting existing Spark jobs as Dataflow pipelines would require significant redevelopment effort and is not necessary to minimize overhead for occasional jobs. The question specifically asks for strategies for existing Spark jobs on Dataproc.
- ✓
Store all data in Cloud Storage instead of HDFS, and use the Cloud Storage connector to access it.
Why this is correct
Correct. Storing data in Cloud Storage decouples storage from compute, enabling ephemeral clusters. The Cloud Storage connector provides Hadoop-compatible access, eliminating HDFS overhead and reducing cost because storage is billed separately and persists beyond cluster lifetime.
- ✓
Create an ephemeral Dataproc cluster for each job and delete it after completion.
Why this is correct
Correct. Ephemeral clusters are created per job and deleted after completion. This minimizes cost because you only pay for compute during job execution, and operational overhead is low because no persistent cluster management is needed.
- ✗
Use a small persistent cluster that runs continuously and submit jobs to it.
Why it's wrong here
A continuously running cluster bills for idle capacity around the clock, defeating the cost goal for a few daily jobs. Persistent clusters suit latency-sensitive workloads needing warm workers, whereas sporadic batch jobs fit ephemeral, job-scoped clusters.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 747 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.