PDE Practice Test 25 — 15 Questions

Question 1

A Dataflow pipeline as described in the exhibit has increasing lag. Which optimization is most likely to reduce the lag?

Accepted Answer

Use FileLoads instead of StreamingInserts for BigQuery output. The exhibit shows increasing lag in a Dataflow pipeline writing to BigQuery. StreamingInserts (the default) use the BigQuery Storage Write API, which can throttle under high throughput, causing backpressure and lag. Switching to FileLoads writes data to temporary files in Cloud Storage and then loads them into BigQuery via batch load jobs, which decouples the write path from the streaming insert quota and reduces lag by avoiding per-row insert limits.

Answer

Increase the number of workers

Answer

Use global windows instead of fixed windows

Answer

Add additional ParDo transforms

Question 2

You have two versions of a classification model (v1 and v2) deployed on a Vertex AI Endpoint. You want to gradually roll out v2 to 10% of traffic, monitor performance, and if metrics are better, increase traffic to 100%. You have set up model monitoring for skew and drift. Which configuration should you use?

Accepted Answer

Use the Vertex AI Endpoint 'traffic_split' parameter to assign 10% of traffic to v2 and 90% to v1.. The Vertex AI Endpoint 'traffic_split' parameter allows you to direct a percentage of inference requests to different model versions deployed on the same endpoint. Setting 10% to v2 and 90% to v1 enables a gradual rollout while monitoring skew and drift, and you can adjust the split as needed. This is the native, supported method for canary deployments in Vertex AI, avoiding the complexity and latency of external load balancers.

Answer

Deploy v2 to a separate endpoint and use a load balancer to route 10% of traffic.

Answer

Create a new deployment with v2 on the same endpoint and set the 'min_replica_count' to 1 for both versions.

Answer

Enable Vertex AI Model Monitoring on the endpoint and set up alerting for performance drop.

Question 3

A data team uses Cloud Composer to orchestrate Airflow DAGs. They need to ensure that a downstream task runs only if at least two out of three upstream sensor tasks succeed. Which TWO configurations should they combine?

Accepted Answer

Set trigger_rule to 'none_failed_or_skipped' and use a condition.. The 'none_failed_or_skipped' trigger rule triggers the downstream task when all upstream tasks have succeeded or been skipped. Combined with a condition (e.g., using a PythonOperator or BranchPythonOperator) that checks whether at least two of the three sensor tasks succeeded, this ensures the downstream task runs only when the required threshold is met. This approach leverages Airflow's built-in trigger rules and conditional logic to implement a quorum-based dependency.

Answer

Set trigger_rule to 'one_success'.

Answer

Set trigger_rule to 'all_done'.

Answer

Set trigger_rule to 'none_failed'.

Question 4

A team has trained a scikit-learn model and wants to deploy it to AI Platform Prediction for online predictions. What is the required format for the model artifact?

Accepted Answer

A model.joblib file (or model.pkl) along with any custom code.. AI Platform Prediction (now Vertex AI) supports scikit-learn models natively. The required artifact format is a serialized model file (model.joblib or model.pkl) optionally accompanied by any custom code dependencies. This is because scikit-learn models are pickled objects, and the platform deserializes them using the same Python environment specified in the runtime version.

Answer

A single .h5 file containing the model weights.

Answer

A SavedModel directory containing the model for TensorFlow.

Answer

A model.pt file for PyTorch models.

Question 5

You need to deploy a reusable Dataflow pipeline that can be executed with different parameters from Cloud Composer. Which TWO components should you use? (Choose 2)

Accepted Answer

Dataflow Flex Template. Dataflow Flex Templates allow you to create custom templates that can accept runtime parameters. Cloud Composer can trigger these templates using the DataflowStartFlexTemplateOperator. Direct runner options are not needed.

Answer

Direct runner

Answer

Dataflow Classic Template

Answer

Cloud Scheduler

Question 6

A company uses BigQuery's Storage Write API in committed mode to stream data. They notice that some writes are failing with 'DEADLINE_EXCEEDED' errors during peak traffic. The pipeline is a Dataflow job using the Beam SDK. What is the MOST likely cause and solution?

Accepted Answer

The default RPC timeout is too low for the write throughput; increase the timeout in the Storage Write API configuration.. Committed mode requires immediate acknowledgment from BigQuery. Under high traffic, the default timeout may be exceeded. The solution is to increase the timeout or switch to buffered mode, which provides higher throughput by batching. The error is not due to schema mismatch or permissions; those would cause different errors. Pub/Sub is not involved in the write path.

Answer

The Dataflow workers lack sufficient memory; increase worker memory.

Answer

The Pub/Sub subscription is not sending acknowledgments; check the subscription.

Answer

The row schema has changed; update the schema before writing.

Question 7

A data engineer is designing a batch ETL pipeline using Cloud Composer and Dataflow. The pipeline must be self-healing and retry on failures. Which Composer feature should they configure?

Accepted Answer

Retry policy on the DAG. Cloud Composer (based on Apache Airflow) allows you to configure a retry policy directly on the DAG or individual tasks. This enables the pipeline to automatically retry failed tasks according to parameters like `retries`, `retry_delay`, and `retry_exponential_backoff`, making the ETL pipeline self-healing without external services.

Answer

Use Cloud Tasks for retries

Answer

Cloud Composer with high availability

Answer

Dataflow retries

Question 8

A data science team needs to ensure that a deployed Vertex AI model can handle varying traffic patterns with minimal latency and cost. What should they do?

Accepted Answer

Use Vertex AI Prediction with autoscaling. Vertex AI Prediction with autoscaling dynamically adjusts the number of serving instances based on incoming traffic, ensuring minimal latency during spikes and cost efficiency during lulls. This is the recommended approach for handling variable traffic patterns in production, as it leverages Google Cloud's managed infrastructure to scale from zero to thousands of nodes automatically.

Answer

Use batch prediction instead of online

Answer

Pre-warm all instances

Answer

Deploy to a single large machine type

Question 9

A retail company uses Cloud Dataflow for a streaming pipeline that aggregates sales events from thousands of stores. The pipeline writes aggregated results to BigQuery every 5 minutes. Recently, the Dataflow job has been restarting multiple times a day with the error: 'Worker ran out of memory' in the logs. The streaming engine is enabled. The pipeline uses keyed state (ParDo with stateful processing) to maintain per-store counters. The average event size is 2KB, and the throughput is 2,000 events/sec. You need to resolve the out-of-memory issues without losing data. What should you do?

Accepted Answer

Increase the number of workers in the pipeline configuration and ensure the maximum worker count is set higher to allow better distribution of state.. Increasing the number of workers distributes the keyed state (per-store counters) across more VMs, reducing the memory pressure on each individual worker. With streaming engine enabled, state is still held in worker memory for low-latency access, so adding workers is the direct way to scale the state footprint. This avoids data loss because the pipeline continues processing with exactly-once semantics and state is preserved via checkpointing.

Answer

Disable stateful processing and use side inputs from BigQuery to get per-store aggregates.

Answer

Modify the pipeline to use sliding windows with a shorter duration to reduce the state size.

Answer

Reduce the number of workers to limit the overhead of data shuffling.

Question 10

Drag and drop the steps to create a Cloud Function triggered by Cloud Storage events into the correct order.

Accepted Answer

Step 1: Create a Cloud Storage bucket. Step 2: Write the Cloud Function code. Step 3: Deploy the Cloud Function with the appropriate event trigger. Step 4: Test the function by uploading a file to the bucket.. The correct sequence for creating a Cloud Function triggered by Cloud Storage events is: first ensure the bucket exists (create if needed), then write the function code, deploy the function with the appropriate event trigger (e.g., google.storage.object.finalize), and finally test by uploading a file. Common mistakes include deploying before creating the bucket, testing before deployment, or deploying without code.

Answer

Step 1: Write the Cloud Function code. Step 2: Create a Cloud Storage bucket. Step 3: Deploy the Cloud Function with the appropriate event trigger. Step 4: Test the function by uploading a file to the bucket.

Answer

Step 1: Create a Cloud Storage bucket. Step 2: Write the Cloud Function code. Step 3: Test the function by uploading a file to the bucket. Step 4: Deploy the Cloud Function with the appropriate event trigger.

Answer

Step 1: Deploy the Cloud Function with the appropriate event trigger. Step 2: Create a Cloud Storage bucket. Step 3: Write the Cloud Function code. Step 4: Test the function by uploading a file to the bucket.

Question 11

A data engineer needs to design a data pipeline that ingests streaming data from Cloud Pub/Sub, performs real-time aggregations, and loads the results into BigQuery for dashboarding. Which Google Cloud service should they use for the streaming aggregation step?

Accepted Answer

Dataflow. Dataflow is a fully managed service for stream and batch processing that integrates with Pub/Sub and BigQuery. It supports exactly-once processing and low-latency streaming.

Answer

Cloud Functions

Answer

Cloud Dataproc

Answer

Cloud Composer

Question 12

Drag and drop the steps to deploy a Cloud Dataflow pipeline from a template into the correct order.

Accepted Answer

Navigate to the Dataflow section of the Cloud Console, then click 'Create Job from Template', then select the desired pipeline template, then specify the required parameters, then click 'Run'.. Deploying a Dataflow pipeline from a template via the Cloud Console involves a specific sequence: first, navigate to the Dataflow page, then create a job from a template, select the appropriate template, fill in the required parameters (such as input/output and temp location), and finally run the job. This order ensures all necessary configurations are applied correctly before execution.

Answer

Select the desired pipeline template, then navigate to the Dataflow section of the Cloud Console, then click 'Create Job from Template', then specify the required parameters, then click 'Run'.

Answer

Navigate to the Dataflow section of the Cloud Console, then specify the required parameters, then select the desired pipeline template, then click 'Create Job from Template', then click 'Run'.

Answer

Navigate to the Dataflow section of the Cloud Console, then click 'Run', then select the desired pipeline template, then specify the required parameters, then click 'Create Job from Template'.

Question 13

You have a batch prediction job on Vertex AI that processes millions of records. The job is failing with an out-of-memory error. What is the best way to resolve this?

Accepted Answer

Use a machine type with more memory for the batch prediction job. A batch prediction job on Vertex AI runs on a single machine (or a cluster of machines) and an out-of-memory (OOM) error indicates that the model or data processing exceeds the available RAM of the chosen machine type. Increasing the machine's memory directly addresses the root cause by providing more heap space for loading the model and processing large batches of predictions, without altering the job's parallelism or data partitioning.

Answer

Increase the minNodes and maxNodes for the batch prediction job

Answer

Split the input data into smaller files and run multiple batch prediction jobs

Answer

Enable autoscaling on the batch prediction job

Question 14

You are designing a disaster recovery strategy for a critical streaming data processing pipeline. The pipeline reads from Cloud Pub/Sub, processes with Dataflow streaming, and writes to BigQuery. The required RPO is less than 1 minute, and RTO is less than 5 minutes. Which architecture should you implement?

Accepted Answer

Use cross-region replication with two separate Dataflow pipelines reading from a Pub/Sub cross-region subscription and writing to a BigQuery cross-region dataset. Cross-region replication for Pub/Sub ensures messages are available in a secondary region with sub-second latency, and a separate Dataflow pipeline reading from a cross-region subscription provides active-active processing. BigQuery cross-region dataset replication (using the 'cross-region' dataset location, e.g., EU or US multi-region, or a specific dual-region configuration) ensures data durability and availability within the RPO of <1 minute. This architecture meets both RPO and RTO by eliminating single points of failure and enabling automatic failover without manual intervention.

Answer

Run the pipeline using Dataflow batch mode with a 1-minute trigger and store intermediate results in Cloud Storage

Answer

Deploy resources in a single region with regular backups to Cloud Storage

Answer

Use a single Dataflow pipeline with a standby cluster in another region, but failover is manual

Question 15

You are designing a streaming data pipeline that must guarantee exactly-once processing semantics for financial transactions. The pipeline reads from Cloud Pub/Sub and writes to Cloud Bigtable. Each transaction has a unique transaction ID. Which features do you need to implement to ensure exactly-once semantics end-to-end?

Accepted Answer

Use Dataflow with exactly-once processing, and ensure the Bigtable sink uses idempotent mutations based on the transaction ID.. Dataflow's exactly-once processing guarantees that each record is processed precisely once, and idempotent Bigtable mutations (keyed by transaction ID) ensure that even if a mutation is retried, the result is the same. This combination provides end-to-end exactly-once semantics: Dataflow handles source-side deduplication and checkpointing, while Bigtable's idempotent writes prevent duplicates at the sink.

Answer

Use Cloud Pub/Sub with synchronous pull and manually commit offsets after successfully writing to Bigtable.

Answer

Use Dataflow with at-least-once processing and implement deduplication in a windowed transform using the transaction ID.

Answer

Use Cloud Pub/Sub with exactly-once delivery enabled, and write to Bigtable using single-row transactions.