PDE Practice Test 14 — 15 Questions

Question 1

A data engineer configures the above lifecycle rule on a Cloud Storage bucket that stores daily log files. After 60 days, they notice that files older than 30 days have been transitioned to Nearline, but files older than 90 days are still present. What is the most likely cause?

Accepted Answer

The delete rule's condition includes `matchesStorageClass`: `STANDARD`, which does not match the Nearline storage class of transitioned objects.. Option D is correct because the lifecycle delete rule includes a `matchesStorageClass` condition set to `STANDARD`. Once objects are transitioned to Nearline (which is a different storage class), they no longer match the `STANDARD` condition, so the delete rule does not apply to them. As a result, files older than 90 days that were moved to Nearline remain in the bucket.

Answer

The delete rule is missing `isLive: true` condition, so it does not apply to live objects.

Answer

The `age` condition in the delete rule is calculated from the transition date, not creation date.

Answer

The bucket has object versioning enabled, and the delete rule only applies to non-current versions.

Question 2

A company is designing a real-time clickstream analytics pipeline using Pub/Sub and Dataflow. The pipeline must handle late-arriving data (up to 1 hour) and ensure exactly-once processing. Which Dataflow feature should be configured to handle late data correctly?

Accepted Answer

Configure the trigger with allowed lateness of 1 hour.. Option A is correct because Dataflow's allowed lateness feature explicitly controls how long the pipeline waits for late-arriving data before closing a window. By setting allowed lateness to 1 hour, the watermark is held back, and late data within that period is still processed with exactly-once semantics. This directly addresses the requirement for handling late data up to 1 hour while ensuring no duplicates or data loss.

Answer

Use fixed windows with a 1-hour period and enable data discarding.

Answer

Use session windows with a gap duration of 1 hour.

Answer

Set the watermark estimate to 1 hour.

Question 3

A financial services company uses Vertex AI to serve a fraud detection model. The model was trained on historical data that is updated daily. The team wants to automate retraining when data drift is detected. Which approach best operationalizes this requirement with minimal manual intervention?

Accepted Answer

Enable Vertex AI Model Monitoring for feature drift and skew, then create a Cloud Function that triggers a Vertex AI Pipeline to retrain and deploy the model after validation.. Option D is correct because it uses Vertex AI Model Monitoring to automatically detect feature drift or skew, then triggers a Cloud Function that invokes a Vertex AI Pipeline to retrain and redeploy the model after validation. This approach minimizes manual intervention by automating both the detection of data drift and the subsequent retraining and deployment lifecycle.

Answer

Use Cloud Monitoring alerts on prediction latency to trigger a retraining pipeline.

Answer

Manually monitor model performance metrics in Vertex AI Experiments and retrain when accuracy drops.

Answer

Use scheduled Vertex AI Pipelines to retrain the model every night, then deploy automatically.

Question 4

A Dataflow batch job frequently fails with 'OutOfMemoryError'. Which THREE are common causes? (Choose 3)

Accepted Answer

Inefficient GroupByKey with hot keys. Option B is correct because a hot key in a GroupByKey operation causes all values for that key to be processed by a single worker, leading to memory exhaustion when the key's associated data exceeds the worker's memory capacity. This is a common cause of OutOfMemoryError in Dataflow batch jobs, as the SDK buffers all values for a key before emitting the result.

Answer

Too many parallel workers

Answer

Using Dataflow Shuffle

Question 5

A company uses Cloud Dataflow to process streaming data from Pub/Sub into BigQuery. The pipeline uses a side input from a Cloud Bigtable table containing user profile information to enrich the events. The side input is updated every hour. Which approach should the company use to ensure that the pipeline uses the latest profile data without causing high memory usage?

Accepted Answer

Use a side input that is periodically refreshed by reading the Cloud Bigtable table at a regular interval.. Option A is correct because Cloud Dataflow supports periodically refreshing side inputs by reading from an external source like Cloud Bigtable at a specified interval. This approach keeps the profile data up-to-date without storing the entire side input in memory for the lifetime of the pipeline; instead, the side input is rebuilt and cached only when refreshed, controlling memory usage.

Answer

For each incoming event, read the corresponding profile from Cloud Bigtable using a synchronous call.

Answer

Use a CoGroupByKey transform to join the stream with a bounded PCollection created from the Cloud Bigtable table.

Answer

Stream the profile updates into a separate BigQuery table and use a BigQuery streaming query to join in real-time.

Question 6

A company wants to replicate a Cloud SQL (PostgreSQL) database to BigQuery in near real-time for analytics. The volume is about 10GB per day with frequent updates and deletes. They need to capture changes with low latency and ensure exactly-once delivery to BigQuery. Which approach should they use?

Accepted Answer

Use Cloud Data Fusion with a Debezium streaming source to capture CDC from Cloud SQL and a BigQuery sink with exactly-once mode.. Option C is correct because Cloud Data Fusion with a Debezium streaming source provides native change data capture (CDC) from PostgreSQL, capturing inserts, updates, and deletes with low latency. The BigQuery sink in exactly-once mode ensures no duplicate records, meeting the requirement for near real-time analytics with frequent updates and deletes.

Answer

Export the entire database to Cloud Storage as CSV files every hour and load them into BigQuery using a load job with WRITE_TRUNCATE.

Answer

Use a Dataflow pipeline with JDBCIO to read from Cloud SQL every minute and write changes to BigQuery using upserts.

Answer

Use Cloud SQL's change data capture feature to write changes to a Pub/Sub topic and use a Dataflow pipeline to stream into BigQuery.

Question 7

A data engineering team uses Cloud Pub/Sub to ingest clickstream events and Cloud Dataflow to process them. They need to maintain strict event ordering per user session, and the processing output must be written to a BigQuery table with exactly-once semantics. Which configuration should the team implement?

Accepted Answer

Enable message ordering in Pub/Sub with a session ID as the ordering key, and in Dataflow use a global window with a custom trigger that fires on watermark and uses a BigQuery sink with 'exactly-once' mode enabled.. Option A is correct because it combines Pub/Sub message ordering (using a session ID as the ordering key) with Dataflow's exactly-once sink to BigQuery. The global window with a watermark-based trigger ensures all events for a session are processed in order before writing, while the BigQuery 'exactly-once' mode prevents duplicate rows even if the pipeline retries. This satisfies both strict per-session ordering and exactly-once semantics.

Answer

Use a Pub/Sub pull subscription with a subscriber that acknowledges messages immediately after processing, and a Dataflow pipeline with a sliding window.

Answer

Assign a unique session ID as the message ordering key in Pub/Sub, use a Dataflow pipeline with session windows and .withAllowedLateness(0), and write to BigQuery using a batch load.

Answer

Use a Pub/Sub push subscription with an acknowledgment deadline of 600 seconds and enable exactly-once delivery on the subscription.

Question 8

A company is ingesting real-time sensor data from thousands of devices into Cloud Pub/Sub. They need to process this data with low latency (seconds) and exactly-once semantics. Which data processing service should they use?

Accepted Answer

Dataflow streaming with exactly-once processing. Dataflow streaming with exactly-once processing is the correct choice because it provides exactly-once semantics for Pub/Sub sources via checkpointing and idempotent sinks, and it meets the low-latency (seconds) requirement through its streaming engine that minimizes per-element overhead. Cloud Dataflow's integration with Pub/Sub ensures that each message is processed exactly once, even in the presence of failures, by using snapshots and consistent state management.

Answer

Cloud Run with Pub/Sub push

Answer

Cloud Functions triggered by Pub/Sub

Answer

Dataproc with Spark Streaming

Question 9

A team wants to ingest streaming data from millions of IoT devices and store historical data in BigQuery for analysis. They need near real-time analytics on the most recent data, with sub-second latency. Which architecture should they use?

Accepted Answer

Use Pub/Sub, then a Dataflow pipeline that filters and transforms data, writing to Cloud Bigtable for real-time queries and to Cloud Storage for periodic BigQuery loads.. Option B is correct because it uses Cloud Bigtable for sub-second latency on recent data, which is ideal for near real-time analytics on streaming IoT data. Dataflow provides the necessary stream processing, filtering, and transformation before writing to Bigtable for low-latency queries and to Cloud Storage for periodic batch loads into BigQuery for historical analysis. This architecture decouples real-time and historical paths, meeting both latency and storage requirements.

Answer

Use Pub/Sub to receive data, then stream directly into BigQuery using the streaming API, and use standard SQL queries for real-time analytics.

Answer

Use Pub/Sub to ingest data into a Dataproc Spark Streaming job that writes to both Bigtable and BigQuery.

Answer

Use Cloud SQL to store the latest data and periodically move historical data to BigQuery via cron jobs.

Question 10

A company processes large volumes of GPS sensor data stored in Cloud Storage. Each hour, they run an Apache Spark job that aggregates the data by geohash region. The job must be cost-effective and scale automatically. Currently, they are using a Dataproc cluster with preemptible workers. Which improvement would best reduce costs while maintaining performance?

Accepted Answer

Use Dataproc Serverless Spark. Dataproc Serverless Spark (Option D) eliminates the need to manage a cluster, automatically scaling resources to match job demand and charging only for the resources consumed during execution. This removes the overhead of preemptible worker management and idle cluster costs, directly reducing expenses while maintaining performance for the hourly aggregation job.

Answer

Use a larger Dataproc cluster with standard workers

Answer

Migrate the job to BigQuery scheduled queries

Answer

Switch to Dataflow batch pipeline with Apache Beam

Question 11

A team needs to orchestrate a complex ETL workflow that includes conditional branching (if new data arrives, run transformation A, else run transformation B), error handling, and coordination across multiple services. Which service should they use?

Accepted Answer

Cloud Composer (Apache Airflow). Cloud Composer (Apache Airflow) is the correct choice because it is designed for orchestrating complex, multi-step ETL workflows with conditional branching, error handling, and cross-service coordination. Airflow's directed acyclic graphs (DAGs) natively support conditional logic (e.g., BranchPythonOperator), retries, and dependency management across heterogeneous services, making it ideal for this use case.

Answer

Cloud Functions

Answer

Cloud Workflows

Answer

Cloud Scheduler

Question 12

A company is migrating their on-premises Apache Spark jobs to Google Cloud Dataproc. They want to minimize operational overhead and cost for jobs that run only a few times per day. Which TWO strategies should they adopt? (Choose TWO.)

Accepted Answer

Store all data in Cloud Storage instead of HDFS, and use the Cloud Storage connector to access it.. Option C is correct because storing data in Cloud Storage decouples storage from compute, allowing ephemeral clusters to be spun up and down without data loss. The Cloud Storage connector provides Hadoop-compatible file system access, eliminating the need for HDFS replication and reducing costs by avoiding persistent cluster storage.

Answer

Configure HDFS replication factor to 3 to ensure data durability during cluster restarts.

Answer

Rewrite the Spark jobs as Dataflow pipelines to take advantage of serverless processing.

Answer

Use a small persistent cluster that runs continuously and submit jobs to it.

Question 13

A streaming Dataflow pipeline ingests events from Cloud Pub/Sub and writes to BigQuery. The event schema evolves occasionally (new columns added). The pipeline fails when new columns appear. What is the best long-term solution?

Accepted Answer

Configure the BigQuery sink to use stored 'dynamic' schema by setting create_disposition to CREATE_NEVER and writing to a temporary table with schema auto-detection. Option A is correct because it leverages BigQuery's schema auto-detection with a temporary table to handle schema evolution dynamically. By setting create_disposition to CREATE_NEVER, the pipeline writes to a table that already exists, while the temporary table with auto-detection allows the pipeline to infer new columns from the incoming data. This approach avoids pipeline failures when new columns appear, as the sink can adapt without manual intervention or pipeline restarts.

Answer

Stop the pipeline and update the BigQuery schema manually whenever a new column appears

Answer

Switch to Dataproc to process the data with Spark and write to BigQuery using the Avro format

Answer

Use a Cloud Function to transform the data and add null columns for missing fields

Question 14

Which THREE Google Cloud services are considered fully managed serverless data processing services? (Choose THREE.)

Accepted Answer

Cloud Functions. Cloud Functions is a fully managed serverless data processing service because it executes code in response to events without requiring any server provisioning or management. It automatically scales from zero to thousands of instances based on incoming requests, and you pay only for compute time used while your code runs. This makes it ideal for lightweight, event-driven data processing tasks such as transforming data in Cloud Storage or reacting to Pub/Sub messages.

Answer

Cloud Dataproc

Answer

Cloud Composer

Question 15

You are optimizing a Dataflow pipeline that performs a group-by-key transformation on a large, skewed dataset. The pipeline is experiencing high latency due to data skew (some keys have many more values). Which TWO actions can help mitigate the skew? (Choose two.)

Accepted Answer

Use hot key detection and split the hot key into multiple sub-keys (e.g., append a random number).. Option A is correct because splitting a hot key into multiple sub-keys (e.g., by appending a random number) distributes the values across multiple shards during the shuffle phase, reducing the load on any single worker. This technique, often called "salting," is a standard pattern in Dataflow and Apache Beam to handle data skew by breaking the bottleneck caused by a single key with disproportionately many values.

Answer

Enable the Dataflow service's automatic reshuffling feature.

Answer

Use CoGroupByKey to reduce the number of keys.

Answer

Increase the number of worker machines.

PDE Practice Test 14 — 15 Questions

Exhibit

PDE Practice Test 14 — 15 Questions

Exhibit