PDE Ingesting and Processing the Data Practice Question
A company runs Apache Kafka on Dataproc for real-time event streaming. They want to archive the Kafka topics to Cloud Storage for long-term retention and later analysis in BigQuery. Which approach is the most cost-effective and operationally simple?
⚠ Common exam trap
A common mistake in Google exams is to think that streaming data to Cloud Storage requires a full streaming pipeline (Spark, Dataflow) or an intermediary service like Pub/Sub, when in fact Kafka Connect provides a native, lightweight, and cost-effective sink directly to GCS.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use Kafka Connect with the GCS Sink Connector to write directly to Cloud Storage
Kafka Connect with the GCS Sink Connector is purpose-built for exactly this use case: it directly streams Kafka topics to Cloud Storage in Avro, Parquet, or JSON format without requiring intermediate processing clusters or services. This approach minimizes operational overhead (no Spark or Dataflow jobs to manage) and is cost-effective since it runs as a lightweight connector within the existing Kafka ecosystem, leveraging Dataproc's managed Kafka cluster.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use Apache Spark streaming on Dataproc to read from Kafka and write to GCS
Why it's wrong here
Using Apache Spark streaming on Dataproc to read from Kafka and write to GCS introduces unnecessary compute overhead and operational complexity because Spark requires a persistent cluster to run the streaming job, whereas a purpose-built connector like the Kafka Connect GCS Sink can run serverlessly and scale automatically. This option is tempting because Spark streaming is a familiar, general-purpose tool for real-time ETL, and it would be the correct choice if the pipeline also required complex transformations, joins, or stateful aggregations before landing data in GCS.
- ✗
Use Kafka MirrorMaker to replicate topics to a second cluster that writes to GCS
Why it's wrong here
MirrorMaker is for cluster replication, not direct GCS export.
- ✗
Use the Pub/Sub connector to publish Kafka messages to Pub/Sub, then a Dataflow job to write to GCS
Why it's wrong here
This adds unnecessary hops and cost with Pub/Sub and Dataflow.
- ✓
Use Kafka Connect with the GCS Sink Connector to write directly to Cloud Storage
Why this is correct
Kafka Connect GCS Sink Connector is purpose-built, simple to configure, and runs on the same Dataproc cluster.
Go deeper
Related to this question
About these practice questions
One of 890 original PDE practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.