Cloud Digital Leader Practice Question: Google Cloud products, services, and solutions
A healthcare company needs to run a large batch processing job that analyzes patient records using Apache Spark, transforming data from Cloud Storage and writing results to BigQuery. The job runs once daily and requires a large cluster that should exist only during the job. Which Google Cloud product best handles this ephemeral large-batch Spark workload?
⚠ Common exam trap
The GCDL exam often tests the distinction between managed services that run native Spark (Dataproc) versus those that use different execution engines (Dataflow, BigQuery), leading candidates to confuse Dataflow's ability to run batch pipelines with running Spark code directly.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Cloud Dataproc, which runs managed Apache Spark clusters that can be created for the job and deleted on completion — paying only during the processing window
Cloud Dataproc is the correct choice because it provides managed Apache Spark clusters that can be created on demand for the batch job and automatically deleted upon completion, ensuring you only pay for the processing time. This ephemeral cluster model perfectly matches the requirement of a large cluster that exists only during the daily job, without manual infrastructure management.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Cloud Dataflow, for running the Apache Spark code as a streaming pipeline
Why it's wrong here
Apache Beam pipelines are the workload type that Cloud Dataflow executes; it has no Spark-compatible runtime for arbitrary pre-written PySpark/Scala Spark applications. Porting the existing Spark transformation to Beam would require rewriting the RDD/DataFrame operations and the job driver, which defats any 'run as-is' strategy. Moreover, the job is a daily batch workload, not an unbounded stream, so describing it as a streaming pipeline would change its processing semantics.
- ✓
Cloud Dataproc, which runs managed Apache Spark clusters that can be created for the job and deleted on completion — paying only during the processing window
Why this is correct
Dataproc is the correct choice for managed Apache Spark. The ephemeral cluster pattern (create cluster → run Spark job → delete cluster) is the recommended cost-optimization approach for batch jobs. The cluster exists only while needed, minimizing cost.
- ✗
Compute Engine VMs, by manually installing Apache Spark on a cluster of VMs each day before the job
Why it's wrong here
Standing up Compute Engine VMs, manually installing a Spark distribution, configuring cluster services such as YARN/HDFS, and repeating that every day would require substantial DevOps effort and lengthen job turnaround with setup/teardown scripting. It also lacks Dataproc's built-in integration for cluster initialization, automatic image versioning, and Stackdriver logging/metrics. The manual approach could eventually work, but it is far more expensive in operational time and error-prone than creating a managed Dataproc cluster per job.
- ✗
BigQuery, by running the Spark transformation directly within BigQuery's execution engine
Why it's wrong here
BigQuery's native execution engine is a massively parallel SQL query service; it does not interpret JVM or Python Spark code. Although BigQuery now offers Spark stored procedures, those procedures are executed by a Dataproc cluster behind the scenes, not within BigQuery's compute engine, and they still incur Dataproc pricing. Therefore running the existing PySpark/Scala Spark job directly in BigQuery's engine is not technically possible.
Go deeper
Related to this question
Learn chapter
Cloud Digital Transformation
Key term
Dataproc
Dataproc is a managed cloud service for running Apache Spark and Apache Hadoop clusters, allowing you to process large datasets quickly and economically.
Key term
Cloud storage
Cloud storage is a service that lets you save data on remote servers accessed over the internet instead of on your computer's hard drive.
About these practice questions
This GCDL question is part of Courseiva's 829-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This GCDL practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the GCDL exam.