How should I use these Ingesting and Processing the Data practice questions?

Read each scenario carefully and choose your answer before revealing the explanation. Then check why your choice was right or wrong. Repeat until the reasoning feels automatic.

Can I practise just Ingesting and Processing the Data questions in a focused session?

Yes — use the session launcher on this page to start a 10-, 20-, 30- or 50-question session drawn entirely from the Ingesting and Processing the Data domain.

PDE · topic practice

Ingesting and Processing the Data practice questions

Practise Google Professional Data Engineer Ingesting and Processing the Data practice questions — original exam-style scenarios with answer choices, explanations, and analysis of common mistakes.

Courseiva uses original exam-style practice questions designed for learning and revision. The goal is to understand the concepts, recognise exam patterns, and improve through explanations — not memorise copied exam dumps.

Reviewed byJohnson Ajibi· MSc IT Security

20 questionsDomain: Ingesting and Processing the Data

Practice 10 questions Browse domain →

What the exam tests

What to know about Ingesting and Processing the Data

Ingesting and Processing the Data questions test whether you can apply the concept in context, not just recognise a definition.

How the topic appears in realistic exam-style scenarios.

Which detail in the question changes the correct answer.

How to eliminate plausible but wrong options.

How to connect the question back to the wider exam objective.

Watch out for

Common Ingesting and Processing the Data exam traps

▸Answering from memory before reading the full scenario.
▸Missing a constraint such as cost, availability, security, scope or command context.
▸Choosing a broad answer when the question asks for the most specific fix.
▸Ignoring why the wrong options are tempting.

Practice set

Ingesting and Processing the Data questions

20 questions · select your answer, then reveal the explanation

Question 1easymultiple choice

Read the full Ingesting and Processing the Data explanation →

Your company is migrating an on-premises Hadoop cluster to Google Cloud. You need to transform large datasets using Spark SQL. Which Google Cloud service should you use?

Trap 1: Dataflow

Dataflow runs Apache Beam pipelines, not Spark code.

Trap 2: BigQuery

BigQuery is a serverless data warehouse; you cannot run Spark code on it.

Trap 3: Cloud Dataprep

Cloud Dataprep is a visual data preparation tool, not for Spark.

Study all Ingesting and Processing the Data common traps →

A
Dataflow
Why wrong: Dataflow runs Apache Beam pipelines, not Spark code.
B
Dataproc
Dataproc provides managed Spark clusters where you can run Spark SQL, DataFrames, and RDDs.
C
BigQuery
Why wrong: BigQuery is a serverless data warehouse; you cannot run Spark code on it.
D
Cloud Dataprep
Why wrong: Cloud Dataprep is a visual data preparation tool, not for Spark.

Full breakdown with real-world context →

Question 2mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

Your organization uses dbt (data build tool) for transformations on BigQuery. You need to run dbt models on a schedule and manage versions. Which Google Cloud service can execute dbt jobs in a serverless manner?

Trap 1: Dataflow

Dataflow runs Beam pipelines, not dbt models.

Trap 2: Cloud Build

Cloud Build is a CI/CD service, not for scheduling periodic dbt runs.

Trap 3: Cloud Run

Cloud Run can run dbt as a container, but it lacks built-in scheduling and versioning for multiple dbt models.

Study all Ingesting and Processing the Data common traps →

A
Dataflow
Why wrong: Dataflow runs Beam pipelines, not dbt models.
B
Cloud Build
Why wrong: Cloud Build is a CI/CD service, not for scheduling periodic dbt runs.
C
Cloud Run
Why wrong: Cloud Run can run dbt as a container, but it lacks built-in scheduling and versioning for multiple dbt models.
D
Cloud Composer
Cloud Composer (Airflow) is designed for orchestrating data pipelines and can schedule dbt runs with dependencies and version control.

Full breakdown with real-world context →

Question 3mediummulti select

Read the full Ingesting and Processing the Data explanation →

Which TWO statements are true about BigQuery Data Transfer Service? (Choose 2)

Trap 1: It supports data transformation during transfer.

It does not transform data; it only loads data as-is.

Trap 2: It can transfer data from Cloud SQL to BigQuery directly.

No direct transfer; you need to export from Cloud SQL first.

Trap 3: It is only available in the US and EU regions.

It is available in many regions globally.

Study all Ingesting and Processing the Data common traps →

A
It supports data transformation during transfer.
Why wrong: It does not transform data; it only loads data as-is.
B
It can transfer data from Cloud SQL to BigQuery directly.
Why wrong: No direct transfer; you need to export from Cloud SQL first.
C
It is only available in the US and EU regions.
Why wrong: It is available in many regions globally.

Full breakdown with real-world context →

Question 4hardmulti select

Read the full Ingesting and Processing the Data explanation →

You are building a Dataflow pipeline that reads from Pub/Sub, applies transformations, and writes to BigQuery. The pipeline must handle late-arriving data and ensure that the windowing and triggering are correct. Which THREE configurations should you consider? (Choose 3)

Trap 1: Enable Dataflow Streaming Engine for exactly-once processing.

Streaming Engine improves scalability but does not guarantee exactly-once; committed mode does.

Trap 2: Use side inputs to enrich streaming data with static data.

Side inputs are not related to windowing or late data handling.

Study all Ingesting and Processing the Data common traps →

A
Enable Dataflow Streaming Engine for exactly-once processing.
Why wrong: Streaming Engine improves scalability but does not guarantee exactly-once; committed mode does.
B
Use side inputs to enrich streaming data with static data.
Why wrong: Side inputs are not related to windowing or late data handling.

Full breakdown with real-world context →

Question 5easymultiple choice

Read the full Ingesting and Processing the Data explanation →

A data engineer needs to schedule recurring nightly loads from Amazon S3 to Google Cloud Storage. The data is in CSV format and the volume is approximately 500 GB per night. Which Google Cloud service should they use?

Trap 1: Transfer Appliance

Transfer Appliance is for offline petabyte-scale transfers, not for regular nightly transfers over the network.

Trap 2: BigQuery Data Transfer Service

BigQuery Data Transfer Service is for loading data directly into BigQuery from various sources, not for transferring files between cloud storage.

Trap 3: Datastream

Datastream is for change data capture from databases, not for file transfers.

Study all Ingesting and Processing the Data common traps →

A
Transfer Appliance
Why wrong: Transfer Appliance is for offline petabyte-scale transfers, not for regular nightly transfers over the network.
B
Storage Transfer Service
Storage Transfer Service is designed for online transfers between storage systems including S3 to GCS.
C
BigQuery Data Transfer Service
Why wrong: BigQuery Data Transfer Service is for loading data directly into BigQuery from various sources, not for transferring files between cloud storage.
D
Datastream
Why wrong: Datastream is for change data capture from databases, not for file transfers.

Full breakdown with real-world context →

Question 6mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A company runs a Dataflow pipeline that reads from Pub/Sub, transforms data, and writes to BigQuery. The pipeline uses classic templates and is deployed in batch mode. They notice that the pipeline does not scale well under high load, causing a backlog in Pub/Sub. Which improvement would BEST address the scaling issue?

Trap 1: Change the pipeline to use batch mode instead of streaming

Batch mode does not address real-time processing requirements and may increase latency.

Trap 2: Use a larger machine type for all workers

Uniform machine type may waste resources during low load and still fail under high load.

Trap 3: Increase the number of workers manually in the pipeline…

Manual scaling is not as responsive and may not handle load spikes efficiently.

Study all Ingesting and Processing the Data common traps →

A
Change the pipeline to use batch mode instead of streaming
Why wrong: Batch mode does not address real-time processing requirements and may increase latency.
B
Switch to Dataflow Prime to enable vertical autoscaling and right-fitting
Dataflow Prime automatically adjusts resources for optimal performance under varying loads.
C
Use a larger machine type for all workers
Why wrong: Uniform machine type may waste resources during low load and still fail under high load.
D
Increase the number of workers manually in the pipeline configuration
Why wrong: Manual scaling is not as responsive and may not handle load spikes efficiently.

Full breakdown with real-world context →

Question 7mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A company needs to load data from a MySQL database into BigQuery daily. The data volume is 10 GB per day and the schema changes occasionally. They want to minimize costs and operational overhead. What is the MOST appropriate approach?

Trap 1: Use BigQuery Data Transfer Service for MySQL

BigQuery Data Transfer Service does not support MySQL as a source.

Trap 2: Export MySQL data to CSV, upload to GCS, and use BigQuery load jobs

This approach requires manual handling of schema changes and daily exports, which is more operational overhead.

Trap 3: Use Cloud SQL federated query from BigQuery

Federated queries are suitable for ad-hoc analysis but not for daily scheduled loads, and they may incur high costs for repeated queries.

Study all Ingesting and Processing the Data common traps →

A
Use Datastream to stream changes from MySQL to BigQuery
Datastream handles schema changes automatically and provides low-latency streaming with minimal manual intervention.
B
Use BigQuery Data Transfer Service for MySQL
Why wrong: BigQuery Data Transfer Service does not support MySQL as a source.
C
Export MySQL data to CSV, upload to GCS, and use BigQuery load jobs
Why wrong: This approach requires manual handling of schema changes and daily exports, which is more operational overhead.
D
Use Cloud SQL federated query from BigQuery
Why wrong: Federated queries are suitable for ad-hoc analysis but not for daily scheduled loads, and they may incur high costs for repeated queries.

Full breakdown with real-world context →

Question 8mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A media company streams real-time viewer data from Pub/Sub to BigQuery using a Dataflow pipeline. They need to handle occasional malformed messages without losing valid data. Which pattern should they implement?

Trap 1: Raise an exception in the pipeline and stop processing

Stopping the pipeline on malformed data would cause data loss for valid messages.

Trap 2: Use retry logic in the pipeline to reprocess malformed messages…

Indefinite retries may cause backpressure and do not fix malformed data.

Trap 3: Discard malformed messages and log an error

Discarding data loses potentially recoverable information and does not allow for reprocessing.

Study all Ingesting and Processing the Data common traps →

A
Raise an exception in the pipeline and stop processing
Why wrong: Stopping the pipeline on malformed data would cause data loss for valid messages.
B
Use retry logic in the pipeline to reprocess malformed messages indefinitely
Why wrong: Indefinite retries may cause backpressure and do not fix malformed data.
C
Implement a dead letter sink to store malformed messages for later analysis
Dead letter sinks store problematic records without blocking the pipeline, enabling later inspection and reprocessing.
D
Discard malformed messages and log an error
Why wrong: Discarding data loses potentially recoverable information and does not allow for reprocessing.

Full breakdown with real-world context →

Question 9hardmultiple choice

Study the full Python automation breakdown →

A data engineer is designing a Dataflow pipeline in Python that reads from Pub/Sub, applies complex transformations using external libraries, and writes to BigQuery. The pipeline must be deployed as a reusable, version-controlled template that can be easily updated without re-uploading the pipeline code each time. Which approach should they use?

Trap 1: Use a Classic Template and store the pipeline code in a Cloud…

Classic Templates do not support custom Docker containers and require re-uploading the pipeline code for changes.

Trap 2: Use Cloud Composer to trigger the pipeline each time with updated…

Cloud Composer orchestrates workflows but does not address version-controlled template deployment.

Trap 3: Use Dataflow Prime and deploy the pipeline directly from the Apache…

Dataflow Prime is a runtime optimization feature, not a template deployment mechanism.

Study all Ingesting and Processing the Data common traps →

A
Use a Flex Template with a custom Docker image that contains the pipeline code and dependencies
Flex Templates allow packaging the pipeline in a Docker image, enabling version control and easy updates via image tags.
B
Use a Classic Template and store the pipeline code in a Cloud Storage bucket
Why wrong: Classic Templates do not support custom Docker containers and require re-uploading the pipeline code for changes.
C
Use Cloud Composer to trigger the pipeline each time with updated parameters
Why wrong: Cloud Composer orchestrates workflows but does not address version-controlled template deployment.
D
Use Dataflow Prime and deploy the pipeline directly from the Apache Beam SDK
Why wrong: Dataflow Prime is a runtime optimization feature, not a template deployment mechanism.

Full breakdown with real-world context →

Question 10hardmultiple choice

Read the full Ingesting and Processing the Data explanation →

A financial services company needs to ingest real-time trade data from multiple sources into BigQuery for immediate fraud detection. The data volume is high (1 million messages per second) and each message must be available for queries within seconds. They are considering the Storage Write API. Which stream mode should they choose to balance data availability and cost?

Trap 1: Legacy streaming inserts

Legacy streaming inserts have a 1 MB limit per row and higher latency, and are being deprecated.

Trap 2: Pending mode

Pending mode is for writing data that is not immediately visible; it is used for batch loads, not real-time.

Trap 3: Committed mode

Committed mode writes data directly to BigQuery storage with strong durability, but at higher cost and latency than buffered mode.

Study all Ingesting and Processing the Data common traps →

A
Legacy streaming inserts
Why wrong: Legacy streaming inserts have a 1 MB limit per row and higher latency, and are being deprecated.
B
Pending mode
Why wrong: Pending mode is for writing data that is not immediately visible; it is used for batch loads, not real-time.
C
Buffered mode
Buffered mode provides low-latency streaming with data available within seconds, and is cost-effective for high-volume ingestion.
D
Committed mode
Why wrong: Committed mode writes data directly to BigQuery storage with strong durability, but at higher cost and latency than buffered mode.

Full breakdown with real-world context →

Question 11hardmultiple choice

Read the full Ingesting and Processing the Data explanation →

A team uses dbt on BigQuery to transform data in their data warehouse. They have a large table with nested and repeated fields (arrays and structs). The transformation needs to normalize this data into a star schema. Which dbt feature and BigQuery SQL feature should they use together?

Trap 1: dbt hooks with BigQuery STRUCT access

STRUCT access retrieves fields from structs but does not normalize arrays into rows.

Trap 2: dbt snapshots with BigQuery JSON functions

JSON functions are for JSON data, not for native arrays and structs.

Trap 3: dbt seeds with BigQuery ARRAY_AGG

ARRAY_AGG does the opposite: it aggregates rows into arrays.

Study all Ingesting and Processing the Data common traps →

A
dbt hooks with BigQuery STRUCT access
Why wrong: STRUCT access retrieves fields from structs but does not normalize arrays into rows.
B
dbt models with BigQuery UNNEST and CROSS JOIN
UNNEST flattens arrays into rows, and CROSS JOIN with UNNEST is the standard way to normalize nested data in BigQuery.
C
dbt snapshots with BigQuery JSON functions
Why wrong: JSON functions are for JSON data, not for native arrays and structs.
D
dbt seeds with BigQuery ARRAY_AGG
Why wrong: ARRAY_AGG does the opposite: it aggregates rows into arrays.

Full breakdown with real-world context →

Question 12easymulti select

Read the full Ingesting and Processing the Data explanation →

A data engineer needs to load data from CSV files in Cloud Storage into BigQuery. The CSV files have a header row and some columns contain nested JSON strings. Which TWO methods can they use to load this data into BigQuery?

Trap 1: Use Datastream to load CSV files

Datastream is for CDC from databases, not for file loading.

Trap 2: Create a federated query using an external table

Federated queries read data directly from GCS without loading, but the question asks for loading.

Trap 3: Use gsutil to copy files into BigQuery

gsutil is for Cloud Storage, not for loading into BigQuery.

Study all Ingesting and Processing the Data common traps →

A
Use Datastream to load CSV files
Why wrong: Datastream is for CDC from databases, not for file loading.
B
Use the Storage Write API to write rows from a custom application
The Storage Write API can be used to stream data from CSV after parsing.
C
Create a federated query using an external table
Why wrong: Federated queries read data directly from GCS without loading, but the question asks for loading.
D
Create a BigQuery load job with the CSV format
BigQuery load jobs support CSV files with header rows.
E
Use gsutil to copy files into BigQuery
Why wrong: gsutil is for Cloud Storage, not for loading into BigQuery.

Full breakdown with real-world context →

Question 13mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A data engineer needs to transfer 5 PB of historical data from an on-premises Hadoop cluster to Cloud Storage. The network bandwidth is limited to 1 Gbps, and the transfer must complete within 30 days. Which transfer method should they use?

Question 14easymultiple choice

Read the full Ingesting and Processing the Data explanation →

A company wants to stream real-time user click events from their web application into BigQuery for immediate analysis. Which combination of services is the most scalable and cost-effective for this use case?

Question 15mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A data engineer needs to load 2 TB of Avro files stored in Cloud Storage into BigQuery on a daily schedule. The schema is static and the data should overwrite the existing table each day. What is the most efficient way to accomplish this?

Question 16mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A company needs to run a Spark ML training job on a Dataproc cluster with high memory per node, but the cluster should automatically scale down when idle to save costs. Which configuration should they use?

Question 17easymultiple choice

Read the full Ingesting and Processing the Data explanation →

Which BigQuery feature allows you to write data with exactly-once semantics, high throughput, and the ability to buffer data before making it available for queries?

Question 18mediummultiple choice

Read the full Ingesting and Processing the Data explanation →

A company wants to transform data using dbt (data build tool) on BigQuery. They have a CI/CD pipeline and need to version-control their transformations. Which setup is recommended?

Question 19mediummulti select

Read the full Ingesting and Processing the Data explanation →

Which three of the following are valid BigQuery data loading methods? (Choose THREE.)

Question 20mediummultiple choice

Study the full Python automation breakdown →

A data engineer is building a Dataflow pipeline in Python that reads from BigQuery, transforms data, and writes to Cloud Storage. The pipeline will be deployed in production. Which approach should they use to ensure the pipeline is reusable across environments with different configuration parameters?

Trap 1: Create a separate pipeline for each environment with hardcoded…

Not reusable; violates DRY principles.

Trap 2: Use a Dataflow Classic Template

Classic Templates are less flexible; they require recompilation for different parameters.

Trap 3: Run the pipeline using the DirectRunner for each environment

DirectRunner is for local testing only, not production.

Study all Ingesting and Processing the Data common traps →

A
Create a separate pipeline for each environment with hardcoded values
Why wrong: Not reusable; violates DRY principles.
B
Use a Dataflow Classic Template
Why wrong: Classic Templates are less flexible; they require recompilation for different parameters.
C
Use a Dataflow Flex Template
Flex Templates support runtime parameters and are best for production deployment across environments.
D
Run the pipeline using the DirectRunner for each environment
Why wrong: DirectRunner is for local testing only, not production.

Full breakdown with real-world context →

Continue with 20-question session →

Free account

Track your progress over time

Create a free account to save your results and see which topics improve across sessions.

Focused Ingesting and Processing the Data sessions

Start a Ingesting and Processing the Data only practice session

Every question in these sessions is drawn from the Ingesting and Processing the Data domain — nothing else.

10 questions 20 questions 30 questions 50 questions

Browse all Ingesting and Processing the Data questions →Mixed PDE session

Frequently asked questions

What does the PDE exam test about Ingesting and Processing the Data?: Ingesting and Processing the Data questions test whether you can apply the concept in context, not just recognise a definition.
How should I use these practice questions?: Select your answer before revealing the explanation. Then read why each option is right or wrong — this active recall approach builds retention far faster than re-reading notes.
Can I practise just Ingesting and Processing the Data questions in a focused session?: Yes — the session launcher on this page draws every question from the Ingesting and Processing the Data domain. Use a 10-question session first to gauge your baseline, then move to 20 or 30 once the weak spots are clear.
Where can I practise other PDE topics?: Use the topic links above to move to related areas, or go back to the PDE question bank to see all topics.
Are these real exam questions or dumps?: These are original practice questions written to test the same concepts the PDE exam covers. They are not copied from any real exam or dump site.

Ingesting and Processing the Data only

10 questions 20 questions 30 questions 50 questions

Mixed PDE session

Track your progress

A free account saves results across sessions and highlights which topics need work.

Study resources

All PDE questions Ingesting and Processing the Data domain overview PDE exam guide

Exam traps to avoid

▸Answering from memory before reading the full scenario.
▸Missing a constraint such as cost, availability, security, scope or command context.
▸Choosing a broad answer when the question asks for the most specific fix.
▸Ignoring why the wrong options are tempting.

Ingesting and Processing the Data practice questions

What to know about Ingesting and Processing the Data

Common Ingesting and Processing the Data exam traps

Ingesting and Processing the Data questions

Your company is migrating an on-premises Hadoop cluster to Google Cloud. You need to transform large datasets using Spark SQL. Which Google Cloud service should you use?

Your organization uses dbt (data build tool) for transformations on BigQuery. You need to run dbt models on a schedule and manage versions. Which Google Cloud service can execute dbt jobs in a serverless manner?

Which TWO statements are true about BigQuery Data Transfer Service? (Choose 2)

You are building a Dataflow pipeline that reads from Pub/Sub, applies transformations, and writes to BigQuery. The pipeline must handle late-arriving data and ensure that the windowing and triggering are correct. Which THREE configurations should you consider? (Choose 3)

A data engineer needs to schedule recurring nightly loads from Amazon S3 to Google Cloud Storage. The data is in CSV format and the volume is approximately 500 GB per night. Which Google Cloud service should they use?

A company needs to load data from a MySQL database into BigQuery daily. The data volume is 10 GB per day and the schema changes occasionally. They want to minimize costs and operational overhead. What is the MOST appropriate approach?

A media company streams real-time viewer data from Pub/Sub to BigQuery using a Dataflow pipeline. They need to handle occasional malformed messages without losing valid data. Which pattern should they implement?

A team uses dbt on BigQuery to transform data in their data warehouse. They have a large table with nested and repeated fields (arrays and structs). The transformation needs to normalize this data into a star schema. Which dbt feature and BigQuery SQL feature should they use together?

A data engineer needs to load data from CSV files in Cloud Storage into BigQuery. The CSV files have a header row and some columns contain nested JSON strings. Which TWO methods can they use to load this data into BigQuery?

A data engineer needs to transfer 5 PB of historical data from an on-premises Hadoop cluster to Cloud Storage. The network bandwidth is limited to 1 Gbps, and the transfer must complete within 30 days. Which transfer method should they use?

A company wants to stream real-time user click events from their web application into BigQuery for immediate analysis. Which combination of services is the most scalable and cost-effective for this use case?

A data engineer needs to load 2 TB of Avro files stored in Cloud Storage into BigQuery on a daily schedule. The schema is static and the data should overwrite the existing table each day. What is the most efficient way to accomplish this?

A company needs to run a Spark ML training job on a Dataproc cluster with high memory per node, but the cluster should automatically scale down when idle to save costs. Which configuration should they use?

Which BigQuery feature allows you to write data with exactly-once semantics, high throughput, and the ability to buffer data before making it available for queries?

A company wants to transform data using dbt (data build tool) on BigQuery. They have a CI/CD pipeline and need to version-control their transformations. Which setup is recommended?

Which three of the following are valid BigQuery data loading methods? (Choose THREE.)

Track your progress over time

Start a Ingesting and Processing the Data only practice session

Related PDE topic practice pages

Designing Data Processing Systems practice questions