Cloud Digital Leader Google Cloud Products and Services Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Cloud Dataflow

  Why it's wrong here

  Cloud Dataflow is a fully managed service for executing Apache Beam pipelines, not native Apache Spark workloads. Although it supports batch processing, it would require rewriting your Spark jobs into Beam translations using the Spark Runner compatibility layer, which is not a drop-in replacement and adds migration risk. For a nightly Spark batch job, Dataflow does not provide direct Spark runtime compatibility or the same cluster-based execution model.

• ✓

  Cloud Dataproc

  Why this is correct

  Cloud Dataproc is Google Cloud's managed Spark and Hadoop service, purpose-built to run workloads like Apache Spark directly. You can create a job-scoped cluster that automatically terminates as soon as the batch job finishes, so you only incur compute costs while the job is running, which is ideal for nightly analytics that do not need a persistent cluster. Dataproc also supports custom machine types, preemptible/spot workers, and integration with Cloud Storage, BigQuery, and Cloud Monitoring, making it the lowest-effort, cost-optimized choice.

• ✗

  Google Kubernetes Engine (GKE)

  Why it's wrong here

  While Google Kubernetes Engine (GKE) can run Apache Spark via the Spark-on-Kubernetes operator or by containerizing your jobs, it does not provide native auto-termination of the entire cluster after a single batch job. You must manage node pools, autoscaling, and cluster lifecycle manually, and the nodes remain active until you explicitly delete them or configured autoscaling scales them down. That operational overhead and the risk of idle compute costs make GKE a less suitable choice for a simple nightly Spark batch job compared to Dataproc.

• ✗

  Compute Engine with managed instance groups

  Why it's wrong here

  A managed instance group (MIG) is designed to maintain a fixed number of VM instances or to autoscale based on load, not to run ephemeral analytics clusters. To use it for Spark, you would have to manually install and configure Spark on each VM, coordinate a master and workers, and write separate scripts to shut down all instances after the job completes — something MIGs do not natively support. This approach lacks the integrated job scheduling and automatic cluster deletion of Dataproc, leading to high operational complexity and potential orphaned VMs.

Go deeper

Related to this question