PDE Ingesting and Processing the Data Practice Question
A data engineer is designing a batch processing pipeline that runs daily. The pipeline reads CSV files from GCS, transforms them using Python, and writes the results to BigQuery. They need to parameterize the pipeline for different environments and run it on a schedule. Which THREE components should they use? (Choose 3)
⚠ Common exam trap
Google often tests the distinction between orchestration/scheduling services (Cloud Composer) and compute/processing services (Dataproc, Cloud Functions), leading candidates to mistakenly choose Dataproc for scheduling or Cloud Functions for batch processing.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Cloud Composer
Cloud Composer (A) is correct because it is a managed Apache Airflow service that provides DAG-based scheduling and parameterization, making it ideal for orchestrating a daily batch pipeline with environment-specific variables. Dataproc (B) is correct because it offers managed Spark/Hadoop clusters that can run Python-based transformations over the CSV data read from GCS at batch scale. Dataflow Flex Template (C) is correct because it packages a Dataflow pipeline (including Python transforms) into a reusable, parameterized template that can be invoked with different runtime parameters per environment. Storage Transfer Service (D) is not appropriate here because it only moves data between storage systems and performs no transformation or scheduling logic. Cloud Functions (E) is unsuitable because it is an event-driven, short-lived serverless function, not designed for orchestrating daily batch pipelines or large-scale data transformation.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Cloud Composer
Why this is correct
Cloud Composer orchestrates and schedules the pipeline, and allows parameterization via Airflow variables.
- ✓
Dataproc
Why this is correct
Dataproc can run Python jobs for transforming the data, and Composer can trigger and parameterize these jobs.
- ✓
Dataflow Flex Template
Why this is correct
Dataflow Flex Templates provide a reusable, parameterized pipeline that can be executed by Composer.
- ✗
Storage Transfer Service
Why it's wrong here
Storage Transfer Service is used for transferring data between storage systems, not for processing or scheduling.
- ✗
Cloud Functions
Why it's wrong here
Cloud Functions is event-driven and not suitable for long-running batch tasks or scheduled execution.
Quick reference
Cloud Service Model Comparison
| Model | You Manage | Provider Manages | Examples |
|---|---|---|---|
| IaaS | OS, runtime, apps, data | Hardware, hypervisor, networking | EC2, Azure VMs, GCP Compute Engine |
| PaaS | Apps and data | OS, runtime, middleware, hardware | Elastic Beanstalk, Azure App Service |
| SaaS | Data and settings only | Everything else | Microsoft 365, Salesforce, Workday |
| FaaS / Serverless | Function code only | Infra, scaling, runtime | Lambda, Azure Functions, Cloud Run |
| CaaS | Containers and apps | Kubernetes, OS, hardware | EKS, AKS, GKE |
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 747 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.