PDE Practice Exam 58 — 20 Questions

Question 1

A company runs a critical batch pipeline using Cloud Dataflow. The pipeline processes financial transactions and runs every hour. Recently, some runs have failed due to transient errors (e.g., network timeouts). The engineer wants to automatically retry failed runs without manual intervention. The pipeline is launched from a Cloud Composer DAG using DataflowPythonOperator. What is the BEST way to handle retries?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Set the 'retries' parameter in the DAG's default_args to a positive integer.. Cloud Composer (Apache Airflow) natively supports task-level retries via the 'retries' parameter in default_args. When a DataflowPythonOperator fails due to a transient error, Airflow automatically re-executes the task up to the specified number of retries, without requiring custom sensors or external triggers. This is the simplest and most reliable mechanism for handling transient failures in a DAG-driven pipeline.

Answer

Add a DataflowJobStatusSensor in the DAG that waits for job completion and retries if failed.

Answer

Configure the Dataflow pipeline to automatically retry on failure using the --numberOfWorkerHarnessThreads option.

Answer

Use a Cloud Function triggered by Cloud Scheduler to re-launch the pipeline if the Dataflow job fails.

Question 2

Match each Google Cloud monitoring/logging service to its function.

Accepted Answer

Cloud Monitoring: Monitors the performance, uptime, and health of cloud applications and services.. Cloud Monitoring provides metrics and dashboards; Cloud Logging manages logs; Error Reporting aggregates errors; Cloud Trace provides tracing. Common mistakes include mixing up Logging with Trace and Monitoring with Error Reporting.

Answer

Cloud Logging: Provides distributed tracing and latency analysis.

Answer

Cloud Monitoring: Counts, analyzes, and aggregates errors.

Question 3

A data pipeline uses Cloud Pub/Sub to ingest events and Cloud Functions to transform and write to BigQuery. The system is experiencing data loss during Pub/Sub subscription outages. Which design change improves reliability?

Accepted Answer

Use Dataflow with at-least-once delivery and checkpointing. Dataflow with at-least-once delivery and checkpointing ensures that messages are not lost during Pub/Sub subscription outages because Dataflow tracks processing progress via checkpoints and can replay unacknowledged messages from the last checkpoint. This decouples the processing from the subscription's transient failures, providing fault-tolerant, exactly-once or at-least-once semantics depending on the sink.

Answer

Use a pull subscription with a custom app that polls frequently

Answer

Use long ack deadlines to keep messages in the subscription

Answer

Increase the timeout in Cloud Functions

Question 4

A company needs to deploy a trained model for real-time predictions with low latency. Which Vertex AI resource should they use?

Accepted Answer

Vertex AI Endpoints. Vertex AI Endpoints are designed for online prediction, providing a managed service that hosts models for real-time inference with low latency. They automatically scale resources and handle traffic routing, making them the correct choice for deploying a trained model that needs to respond to individual prediction requests quickly.

Answer

Cloud TPU

Answer

Vertex AI Batch Prediction

Answer

Cloud Run

Question 5

Your company is evaluating managed messaging services for a new event-driven application. The application requires pub/sub semantics, high throughput (millions of messages per second), and integration with Google Cloud services like Cloud Functions and Dataflow. Which TWO services should you consider? (Choose two.)

Accepted Answer

Cloud Pub/Sub. Cloud Pub/Sub (A) is the correct choice because it provides fully managed, highly scalable pub/sub messaging with exactly-once delivery semantics and support for millions of messages per second. It integrates natively with Cloud Functions and Dataflow, making it ideal for event-driven architectures requiring high throughput and decoupled communication.

Answer

Cloud Scheduler

Answer

Cloud Functions

Answer

Cloud Tasks

Question 6

A company uses Dataplex to manage data quality across multiple BigQuery datasets. They need to define data quality rules that check for null values in critical columns and enforce uniqueness constraints. Which two Dataplex features should they use? (Choose TWO)

Accepted Answer

Dataplex Data Quality Rules. Dataplex Data Quality can define rules (including null check and uniqueness) and schedule them as Data Quality Tasks. Data Quality Rules are defined in YAML and can be attached to entities.

Answer

Dataplex Lake

Answer

Dataplex Data Lineage

Answer

Dataplex Universal Catalog

Question 7

A Dataflow pipeline reads events from Pub/Sub and transforms them. Some events contain invalid product IDs that should be filtered out. The list of valid product IDs is stored in a frequently updated BigQuery table. What is the best approach to filter out invalid events?

Accepted Answer

Read the BigQuery table as a side input and refresh it periodically using a global window with a periodic trigger. Reading the BigQuery table as a side input with a global window and periodic trigger allows the pipeline to refresh the list of valid product IDs at a configurable interval without reprocessing the entire stream. This pattern is idiomatic for Beam/Dataflow when the reference data changes frequently and must be kept reasonably current while maintaining low latency for streaming events.

Answer

Use a Combine.PerKey to group by product ID and then filter

Answer

Use a custom pipeline option to read the valid IDs at startup and cache them

Answer

Use a ParDo with a side input that is a MapSideInput of valid IDs, and refresh it on each element

Question 8

A company needs to process sensitive data in BigQuery with column-level security. They want to allow analysts to see aggregated data but not individual records. What approach?

Accepted Answer

Use authorized views with aggregation functions. Authorized views in BigQuery allow you to define SQL queries that aggregate data (e.g., using SUM, COUNT, AVG) and expose only the aggregated results to analysts, while hiding individual records. This approach enforces column-level security by granting access to the view rather than the underlying table, ensuring analysts cannot query the raw data directly. It meets the requirement of seeing aggregated data without seeing individual records, leveraging BigQuery's native authorization and SQL capabilities.

Answer

Use table-level access controls

Answer

Use column-level access controls with masking

Answer

Use Cloud Data Loss Prevention to de-identify data

Question 9

Based on the exhibit, what is the most likely cause of the out-of-memory error?

Accepted Answer

The worker machine type has insufficient memory for the message size and throughput.. The out-of-memory error in a Dataflow pipeline is most likely caused by the worker machine type having insufficient memory for the message size and throughput. When messages are large or the throughput is high, each worker must hold data in memory for processing, windowing, and shuffling. If the worker's memory is too small, the JVM heap runs out of memory, leading to an OOM error.

Answer

The BigQuery output table schema does not match the transformed data, causing write failures.

Answer

The Pub/Sub subscription is not acknowledging messages quickly enough, causing a backlog.

Answer

The fixed window duration of 1 minute is too short, causing excessive state overhead.

Question 10

A company wants to implement model monitoring for a deployed classification model. Which three types of monitoring should they set up? (Select 3)

Accepted Answer

Training-serving skew. Training-serving skew (B) is correct because it detects discrepancies between the data used for training and the data the model sees in production, which can cause performance degradation. This is a critical monitoring type for classification models to ensure the model's assumptions remain valid in the live environment.

Answer

Infrastructure cost monitoring

Answer

Model version comparison

Question 11

A data scientist needs to provide explanations for each prediction made by a deployed autoML model to comply with regulatory requirements. Which Vertex AI feature should they use?

Accepted Answer

Vertex AI Explainable AI. Vertex AI Explainable AI is the correct feature because it provides feature attributions and explanations for each prediction, enabling compliance with regulatory requirements that demand interpretability. It uses techniques like Shapley value approximations or integrated gradients to quantify the contribution of each input feature to the model's output, which is essential for auditing and transparency in deployed autoML models.

Answer

Vertex AI Model Monitoring

Answer

Vertex AI Vizier

Answer

Vertex AI Feature Store

Question 12

A financial services company uses Cloud Composer to orchestrate a daily workflow that includes a Dataproc job for risk analysis. The workflow sometimes fails because the Dataproc cluster creation times out. The cluster creation typically takes 3 minutes, but occasionally takes over 10 minutes. What is the most effective way to handle this variability?

Accepted Answer

Create a long-running Dataproc cluster that remains idle and reuse it for each workflow.. Creating a long-running Dataproc cluster and reusing it eliminates the variable cluster creation time that causes timeouts. Cloud Composer (Airflow) can manage cluster lifecycle separately from the workflow, ensuring the cluster is always available when the Dataproc job runs. This approach decouples cluster provisioning from job execution, making the workflow resilient to creation delays.

Answer

Implement a retry loop with exponential backoff in the DAG.

Answer

Use preemptible VMs for the cluster to reduce cost and improve creation speed.

Answer

Increase the cluster creation timeout in the Airflow configuration.

Question 13

Your team is running a Dataflow streaming pipeline that reads from Pub/Sub, transforms data, and writes to BigQuery. You notice that the pipeline's backlog is growing and the processing latency has increased from seconds to minutes. You need to diagnose and resolve the issue. Which TWO actions should you take? (Choose two.)

Accepted Answer

Increase the maximum number of workers in the pipeline's autoscaling configuration to allow more compute resources.. Increasing the maximum number of workers in the autoscaling configuration allows Dataflow to scale out horizontally, adding more compute resources to handle the increased backlog and reduce processing latency. Dataflow's autoscaling algorithm uses metrics like backlog bytes and CPU utilization to decide when to add workers, but it is capped by the max workers setting. Raising this cap enables the pipeline to allocate more VMs, thus processing more messages per second and reducing the backlog.

Answer

Stop the pipeline, increase the number of workers in the streaming engine configuration, and restart it.

Answer

Increase the batch size in the WriteToBigQuery transform to reduce I/O operations.

Answer

Configure a dead-letter queue in Cloud Storage for failed messages to reduce reprocessing load.

Question 14

A data engineer needs to create a Dataflow pipeline that reads from Pub/Sub, applies a Python transformation, and writes to BigQuery. The pipeline should be reusable across environments with different parameters. Which deployment method is most appropriate?

Accepted Answer

Flex Template. Flex Templates (Option B) are the most appropriate deployment method because they allow you to package a custom Docker image containing your Python transformation code and dependencies, making the pipeline reusable across environments with different runtime parameters. Unlike Classic Templates, Flex Templates support arbitrary pipeline code and can be parameterized at runtime via the Dataflow UI or API, which is essential for a multi-environment deployment strategy.

Answer

Classic Template

Answer

Direct pipeline submission with gcloud dataflow jobs run

Answer

Cloud Composer to trigger Dataflow jobs

Question 15

You are designing a streaming pipeline that ingests events from Pub/Sub, enriches them with a machine learning model, and writes the results to BigQuery. The ML model is deployed on Cloud Run and has a high latency (500ms per request). You need to minimize the impact of slow ML inference on the overall pipeline throughput. Which approach should you take?

Accepted Answer

Use Dataflow to write events to Pub/Sub, then use a separate Dataflow pipeline that batches calls to Cloud Run.. It uses Dataflow to batch events before sending them to Cloud Run, which amortizes the 500ms per-request latency over multiple events, significantly increasing throughput. By writing events to Pub/Sub and then processing them in a separate Dataflow pipeline with batched calls, you decouple the ingestion from the inference and avoid blocking on each individual request.

Answer

Increase the number of Dataflow workers to compensate for the latency.

Answer

Use Cloud Functions to call Cloud Run and write directly to BigQuery.

Answer

Use Dataflow's ParDo with synchronous calls to Cloud Run for each element.

Question 16

A data engineer is designing a batch ETL pipeline that reads CSV files from Cloud Storage, transforms them using Dataproc, and writes the results to BigQuery. The data volume is expected to grow 10x in the next year. Which design approach best balances cost and performance?

Accepted Answer

Use a Dataproc cluster with preemptible worker nodes and autoscaling enabled.. Preemptible worker nodes significantly reduce cost (up to 80% discount) while autoscaling dynamically adjusts cluster size to match the growing workload, ensuring performance without over-provisioning. This combination handles the 10x data growth efficiently by scaling out during peak loads and scaling in during lulls, using preemptible instances for fault-tolerant tasks like transformation.

Answer

Create a single large persistent Dataproc cluster to handle the peak load.

Answer

Use Cloud Data Fusion to visually design the pipeline and run it on Dataproc.

Answer

Migrate the pipeline to Dataflow with Apache Beam and use flexRS for cost savings.

Question 17

A financial services company needs to process high-frequency trading data with strict ordering guarantees. They use Pub/Sub with ordering keys and Dataflow. The pipeline occasionally produces out-of-order results. What is the most likely cause?

Accepted Answer

The window trigger allows late data to be included after the main output. Dataflow's default window trigger behavior allows late data to arrive after the main pane is emitted. When using Pub/Sub with ordering keys, late-arriving events (e.g., due to network delays or retries) can be assigned to the correct window but emitted in a separate pane, causing the final output to appear out-of-order relative to the event time. This is a known behavior when combining event-time windows with late data handling.

Answer

Dataflow does not preserve order when using multiple workers

Answer

Dataflow uses at-least-once processing, which can reorder events

Answer

Pub/Sub does not guarantee message ordering

Question 18

A data science team has built a model using scikit-learn. They want to operationalize it on Google Cloud without rewriting the code. Which approach should they take?

Accepted Answer

Package the model in a custom container and deploy to Vertex AI Endpoints. Vertex AI Endpoints support custom containers, allowing you to package your scikit-learn model with its dependencies (e.g., a Flask or FastAPI inference server) and deploy it without rewriting any code. This approach directly meets the requirement to operationalize the existing model on Google Cloud without modification.

Answer

Export the model as a PMML file and use BigQuery ML

Answer

Use AI Platform Training to host the model directly

Answer

Convert the scikit-learn model to TensorFlow SavedModel format

Question 19

A data scientist deploys a new version of a fraud detection model (model2) alongside the existing model (model1) on the same Vertex AI endpoint with a 70/30 traffic split. After 24 hours, the team notices that model2's predictions are significantly different from model1's, and the fraud detection rate has increased. What is the most likely explanation for the change in predictions?

Accepted Answer

Model2 uses a different model artifact (fraud_detection_v2) that produces different predictions.. The most straightforward explanation for a significant change in predictions and an increased fraud detection rate is that model2 uses a different model artifact (fraud_detection_v2) that was designed to produce different outputs. In Vertex AI, deploying a new model version with a traffic split means both models receive the same input data, but each model artifact independently processes it. If model2's predictions differ substantially, it indicates the model artifact itself has been updated or replaced, not that there is a data or routing issue.

Answer

Model2 was trained on data that leaked future information, causing unrealistic results.

Answer

Model2 is receiving corrupted input data due to a bug in the traffic routing.

Answer

The traffic split is misconfigured and sending all traffic to model2.

Question 20

Match each data lifecycle stage to its description.

Accepted Answer

Data Collection: The process of gathering raw data from various sources.. Data collection and data storage are distinct stages. Collection is about obtaining raw data, while storage is about preserving it for future use. Common confusions arise due to the sequential nature of these processes.

Answer

Data Collection: The process of storing data in a database.

Answer

Data Storage: The process of collecting data from sensors.