PDE Practice Test 4 — 25 Questions

Question 1

Which Google Cloud service provides a fully managed, serverless Spark environment without requiring cluster provisioning?

Accepted Answer

Dataproc Serverless. Dataproc Serverless is a fully managed, serverless Spark environment on Google Cloud that eliminates the need to provision or manage clusters. It automatically scales resources and charges only for the duration of the workload, making it ideal for running Spark jobs without infrastructure overhead.

Answer

Dataproc on GKE

Answer

Dataflow

Answer

Cloud Data Fusion

Question 2

A company uses Cloud Monitoring to track application latency. They notice a spike in latency every 30 minutes. What is the best initial step to diagnose the issue?

Accepted Answer

Check if scheduled jobs or cron tasks overlap.. Recurring spikes at regular intervals often indicate a scheduled process (e.g., cron job, batch job) that runs every 30 minutes. Checking for overlapping scheduled jobs is the most efficient first step before scaling or other actions.

Answer

Increase the number of instances to handle the load.

Answer

Enable Cloud Trace for all requests.

Answer

Change the alert threshold to ignore the spikes.

Question 3

Which TWO are common causes of prediction bias in a deployed machine learning model in production?

Accepted Answer

Data drift between training and serving data distributions.. Option B is correct because data drift occurs when the statistical distribution of the input features or target changes between the training environment and the live serving environment, causing the model's learned mappings to become stale and its predictions to be systematically biased. Option E is correct because training-serving skew arises when feature engineering logic differs between the training pipeline and the production inference path (for example, different imputation, normalization, or aggregation code), so the model receives inputs at serving time that do not match what it learned from, producing biased predictions. Option A is not a cause of bias — high accuracy is generally desirable and does not by itself indicate or produce prediction bias. Option C is not correct in this context because overfitting primarily harms generalization and variance rather than being a canonical cause of prediction bias in production, and it is a training-time issue rather than the drift/skew mechanisms described. Option D is not correct because low latency is a performance characteristic of the serving system and has no direct causal relationship with prediction bias.

Answer

Model accuracy is too high.

Answer

Model is overfitted to training data.

Answer

Low latency predictions.

Question 4

Your company uses Cloud Composer to orchestrate a data pipeline that includes Dataproc Spark jobs and BigQuery load operations. You need to pass the output file path from the Spark job to the next BigQuery task in the DAG. Which two mechanisms can you use to share data between tasks? (Choose TWO.)

Accepted Answer

Write the output path to a Cloud Storage object and read it in the next task.. Option E is correct because Airflow XComs are the native, built-in mechanism for passing small pieces of data such as a file path between tasks in a DAG: the Spark task calls xcom_push (or returns a value) and the BigQuery task uses xcom_pull with the task_id/key to retrieve it. Option C is correct because writing the output path to a Cloud Storage object and reading it in the next task is a durable, decoupled way to share data between tasks, especially when the value is larger or must persist beyond the DAG run; Cloud Composer tasks can use the GCS operators/sensors or the google-cloud-storage client to write and read the object. Option A is not appropriate because Cloud Composer (Airflow) Variables are global key-value configuration stores, not per-run task-to-task data channels, so using them for a run-specific output path risks race conditions and stale values. Option B is not appropriate because Pub/Sub is an asynchronous messaging service, not a synchronous task-to-task handoff mechanism, and it adds unnecessary complexity and latency for passing a single path within one DAG run. Option D is not appropriate because using BigQuery as an intermediary to store a transient output path is heavyweight and semantically wrong; BigQuery is for analytical data, not for orchestrating task metadata.

Answer

Store the output path as a Cloud Composer variable.

Answer

Publish the output path to a Pub/Sub topic and subscribe in the next task.

Answer

Use BigQuery as an intermediary to store the output path.

Question 5

You need to orchestrate a simple, linear workflow that calls several Cloud Functions and API endpoints sequentially with conditional logic. The workflow should be defined as code and have minimal overhead. Which GCP service should you use?

Accepted Answer

Workflows. Workflows is a serverless orchestration service that uses YAML/JSON to define workflows. It is ideal for simpler, linear or conditional orchestrations without the need for full Airflow infrastructure.

Answer

Cloud Tasks

Answer

Dataflow

Answer

Cloud Composer

Question 6

A company processes CSV files that are uploaded to Cloud Storage by external partners. Each file is around 500 MB, and they need to be parsed and loaded into BigQuery. The processing must start as soon as the file arrives. What is the most efficient serverless architecture?

Accepted Answer

Cloud Storage triggers a Cloud Function that publishes events to Pub/Sub; a Dataflow streaming pipeline reads from Pub/Sub and writes to BigQuery.. It combines Cloud Storage event-driven triggers with Pub/Sub for reliable asynchronous message delivery, and uses Dataflow streaming with autoscaling to handle 500 MB files efficiently. This serverless architecture ensures processing starts immediately upon file arrival, scales to handle large files without manual intervention, and leverages BigQuery's streaming inserts for near-real-time data loading.

Answer

Use Cloud Scheduler to periodically check for new files and process them with Dataflow batch jobs.

Answer

Cloud Storage triggers a Dataproc job that reads the file and loads it into BigQuery.

Answer

Cloud Storage triggers a Cloud Function that directly loads the data into BigQuery using the BigQuery API.

Question 7

Which TWO actions are recommended to improve the reliability of a Cloud Dataflow streaming pipeline that processes event data from Pub/Sub?

Accepted Answer

Enable Dataflow Streaming Engine.. Option B is correct because Dataflow Streaming Engine moves pipeline state and shuffle execution out of the worker VMs and into the Dataflow service, which improves reliability by decoupling state management from worker lifecycle events such as restarts, autoscaling, and updates. Option C is correct because enabling exactly-once processing sinks (for example, BigQuery with row-level deduplication via insertion IDs) prevents duplicate or lost records when retries occur, which is essential for reliable event processing from Pub/Sub. Option A is not recommended because a 10-second acknowledgment deadline is too short for many streaming workloads and would cause premature redelivery and duplicate processing rather than improving reliability. Option D is incorrect because disabling autoscaling does not improve reliability and can actually reduce throughput and resilience during load spikes. Option E is incorrect because micro-batching with a small batch size is not a Dataflow reliability best practice and can increase overhead and latency without addressing state or delivery guarantees.

Answer

Use a pull subscription with a 10-second acknowledgment deadline.

Answer

Disable autoscaling to prevent worker churn.

Answer

Use micro-batch processing with a small batch size.

Question 8

A data engineer needs to design a streaming pipeline that ingests events from multiple sources, enriches them with a lookup table stored in BigQuery (updated every hour), and writes the results to a BigQuery table for real-time dashboards. The pipeline must handle late-arriving data up to 1 hour. Which Dataflow feature should be configured to manage late data?

Accepted Answer

Allowed lateness on the window. Allowed lateness on the window is the Dataflow feature that explicitly defines how long the pipeline will retain window state and accept late-arriving elements after the watermark has passed the end of the window. Setting allowed lateness to 1 hour lets Dataflow keep the window's state for that duration, so events arriving up to 60 minutes late are still processed and emitted with correct results. This directly satisfies the requirement to handle late data up to 1 hour without dropping or misassigning events.

Answer

A custom watermark estimation function

Answer

Using side inputs with a periodic refresh

Answer

A trigger that fires on every late element

Question 9

A retail company ingests point-of-sale events from thousands of stores into Cloud Pub/Sub. They need to process these events in a streaming Dataflow pipeline that enriches each event with store metadata from a slowly changing BigQuery table. The enrichment table is updated only once per day. The pipeline must minimize latency and avoid querying BigQuery for every event. Which approach should they use?

Accepted Answer

Load the BigQuery metadata table into a side input as a key-value map, refreshing it periodically via a scheduled pipeline.. The correct approach is to load the slowly changing metadata into a side input that is refreshed periodically. This allows the streaming pipeline to enrich each event without querying BigQuery per event, keeping latency low. Side inputs are a standard Dataflow pattern for enriching streaming data with reference data that changes infrequently, and they avoid the high cost and latency of per-element external calls.

Answer

Use a CoGroupByKey transform to join the streaming events with a bounded read of the metadata table.

Answer

Configure the pipeline to call the BigQuery Storage Read API for each event to fetch the latest metadata.

Answer

Use BigQueryIO.Read with a query that joins the events to the metadata table, and apply the join in the pipeline.

Question 10

A company runs a critical batch pipeline using Cloud Dataflow. The pipeline processes financial transactions and runs every hour. Recently, some runs have failed due to transient errors (e.g., network timeouts). The engineer wants to automatically retry failed runs without manual intervention. The pipeline is launched from a Cloud Composer DAG using DataflowPythonOperator. What is the BEST way to handle retries?

Accepted Answer

Set the 'retries' parameter in the DAG's default_args to a positive integer.. Cloud Composer (Apache Airflow) natively supports task-level retries via the 'retries' parameter in default_args. When a DataflowPythonOperator fails due to a transient error, Airflow automatically re-executes the task up to the specified number of retries, without requiring custom sensors or external triggers. This is the simplest and most reliable mechanism for handling transient failures in a DAG-driven pipeline.

Answer

Add a DataflowJobStatusSensor in the DAG that waits for job completion and retries if failed.

Answer

Configure the Dataflow pipeline to automatically retry on failure using the --numberOfWorkerHarnessThreads option.

Answer

Use a Cloud Function triggered by Cloud Scheduler to re-launch the pipeline if the Dataflow job fails.

Question 11

An organization uses BigQuery on-demand pricing. To control costs, they want to estimate the bytes processed by a query before running it. Which command or method should they use?

Accepted Answer

Use the bq query --dry_run command. The bq query --dry_run flag validates a query and returns the estimated bytes that would be processed without actually executing it or incurring charges. This is the standard method to estimate on-demand BigQuery costs before running a query, since on-demand pricing is based on bytes processed.

Answer

Use bq ls to list table sizes

Answer

Use BigQuery reservations to get cost estimate

Answer

Use INFORMATION_SCHEMA.JOBS_BY_PROJECT to view past costs

Question 12

A team uses Vertex AI AutoML Tables to train a model. They need to deploy the model for real-time predictions with high availability. Which deployment configuration should they use?

Accepted Answer

Deploy to a Vertex AI Endpoint with multiple replicas and auto-scaling. For real-time predictions with high availability, you need a deployment that can handle traffic spikes and failover. Deploying to a Vertex AI Endpoint with multiple replicas and auto-scaling ensures that the model is served from multiple instances, providing redundancy and the ability to scale up or down based on demand. This configuration meets the high-availability requirement by distributing load and automatically recovering from instance failures.

Answer

Export as a Cloud Function

Answer

Deploy to a Vertex AI Endpoint with 1 replica

Answer

Use a Vertex AI Batch Prediction job

Question 13

A data engineer needs to inspect a BigQuery table for sensitive data such as credit card numbers and email addresses before sharing it with a third party. The engineer also wants to de-identify the data by masking the sensitive columns. Which Google Cloud service should be used?

Accepted Answer

Cloud DLP. Cloud DLP (Data Loss Prevention) is the correct service because it is specifically designed to inspect, classify, and de-identify sensitive data such as credit card numbers and email addresses. It provides built-in infoType detectors for over 150 types of sensitive data and supports masking, tokenization, and other de-identification techniques. The engineer can use Cloud DLP to scan BigQuery tables and then apply transformations to mask the sensitive columns before sharing.

Answer

Dataplex

Answer

BigQuery column-level security

Answer

Data Catalog

Question 14

An organization wants to integrate BigQuery Omni to query data stored in AWS S3. They have set up the necessary connections. What is the primary benefit of using BigQuery Omni over simply copying the data to BigQuery?

Accepted Answer

Ability to use BigQuery ML models on data in S3 without moving data.. BigQuery Omni allows you to query data across clouds without moving it, providing a unified analytics experience. It reduces data egress costs and avoids duplication.

Answer

Automatic encryption of data at rest in S3.

Answer

Lower latency queries due to in-memory caching.

Answer

Support for real-time streaming inserts into S3.

Question 15

A media company streams real-time viewer data from Pub/Sub to BigQuery using a Dataflow pipeline. They need to handle occasional malformed messages without losing valid data. Which pattern should they implement?

Accepted Answer

Implement a dead letter sink to store malformed messages for later analysis. A dead letter sink (e.g., a separate Pub/Sub topic or a BigQuery error table) allows the Dataflow pipeline to route malformed messages out of the main processing path while continuing to process valid data. This pattern ensures no valid data is lost and provides a durable location for later analysis or reprocessing of the malformed records, which is essential for streaming pipelines where data quality issues are intermittent.

Answer

Raise an exception in the pipeline and stop processing

Answer

Use retry logic in the pipeline to reprocess malformed messages indefinitely

Answer

Discard malformed messages and log an error

Question 16

A company uses Workflows to orchestrate a series of Google Cloud services for data processing. They need to call an external HTTP API as part of the workflow and handle potential failures with retries. Which Workflows feature should they use?

Accepted Answer

Retry policy on the step. Workflows provides a built-in retry policy that can be configured on individual steps to automatically retry an HTTP call upon transient failures (e.g., 5xx server errors or network timeouts). This allows the workflow to handle external API failures without custom code, using exponential backoff and a maximum retry count.

Answer

Subworkflows

Answer

Parallel steps

Answer

Conditional steps

Question 17

You need to track the lineage of data in BigQuery, showing how tables are derived from other tables via queries. Which service provides this capability?

Accepted Answer

BigQuery Lineage API. The BigQuery Lineage API provides programmatic access to lineage information, showing how tables are derived from other tables through queries, including column-level lineage. It captures dependencies from SQL queries, scheduled queries, and other BigQuery operations.

Answer

Cloud Composer

Answer

Cloud Data Catalog

Answer

Dataflow

Question 18

A company is using Cloud Storage to store raw logs. They want to use Cloud Data Fusion to transform and load the data into BigQuery on a daily schedule. The transformations are complex and involve joining multiple datasets. What is the most efficient way to run these pipelines?

Accepted Answer

Use Cloud Data Fusion to design the pipeline and schedule it to run on a Dataproc cluster. Cloud Data Fusion is a fully managed, code-free ETL/ELT service built on CDAP that provides a visual pipeline designer with a rich set of plugins (including BigQuery sinks) and can execute pipelines on ephemeral Dataproc clusters. For complex transformations involving joins across multiple datasets, Data Fusion's Wrangler and Joiner transforms plus its scheduling capability make it the most efficient native choice. Scheduling the pipeline to run on a Dataproc cluster gives the compute needed for joins while keeping orchestration managed.

Answer

Use Cloud Composer to orchestrate Dataproc jobs that run the transformations

Answer

Use Cloud Functions to trigger a Dataflow job that does the transformations

Answer

Use Cloud Dataprep to design the transformation and export to BigQuery

Question 19

You need to create a BigQuery table that stores customer transaction data. The table will be queried frequently by a customer_id column to retrieve recent transactions (last 30 days). Which table design optimizes query performance and cost?

Accepted Answer

Partition by transaction_date and cluster by customer_id. Partitioning by transaction_date aligns with the query filter (last 30 days), so BigQuery prunes all partitions outside that window, scanning only the relevant date range. Clustering by customer_id then sorts data within each partition, so the customer_id filter benefits from block-level pruning. This combination minimizes both bytes scanned (cost) and query latency.

Answer

Partition by customer_id and cluster by transaction_date

Answer

Partition by ingestion_time and cluster by customer_id

Answer

Cluster by transaction_date and customer_id without partitioning

Question 20

You are migrating an on-premises PostgreSQL database to Cloud SQL. You need to continuously replicate changes to BigQuery for real-time analytics with minimal latency. Which service should you use?

Accepted Answer

Datastream. Datastream is Google Cloud's serverless change data capture (CDC) service that continuously replicates changes from sources like PostgreSQL, MySQL, Oracle, and SQL Server into BigQuery or Cloud Storage with low latency. It reads the source's replication log and streams changes without impacting production workloads significantly. This directly matches the requirement for continuous, near-real-time replication into BigQuery.

Answer

Dataflow with JDBC source

Answer

Pub/Sub with a Cloud Function that writes to BigQuery

Answer

Storage Transfer Service