PDE Practice Exam 56 — 20 Questions

Question 1

A Dataflow batch job fails consistently with the error shown. The job uses a custom container image and runs in a VPC with a private IP. What should the engineer do to resolve the issue?

Accepted Answer

Verify that the VPC has Private Google Access enabled and that Cloud NAT is configured for outbound internet access if needed.. The error indicates that the Dataflow batch job cannot access required resources (e.g., container image, dependencies) because the VPC with private IPs lacks outbound internet connectivity. Option B is correct because enabling Private Google Access allows the VMs to reach Google APIs (like Container Registry) via the Google network, and Cloud NAT provides outbound internet access for non-Google APIs or external dependencies. Without these, the job fails to pull the custom container image or download necessary artifacts.

Answer

Request a CPU quota increase in the region.

Answer

Rebuild the custom container image and upload it to Container Registry.

Answer

Check that the custom image is based on the latest Dataflow SDK version.

Question 2

You manage a large-scale machine learning system that recommends products to users. The model is a deep neural network trained on TensorFlow and deployed on Vertex AI Endpoint with global load balancing. The model receives over 10,000 requests per second. Recently, the team added a new feature: the user's current geographic location (latitude/longitude). After deploying the updated model, you notice that the average prediction latency has doubled, and the error rate has increased, particularly for requests from regions far from the model's primary training data (North America). You suspect the location feature is causing issues. What should you do to diagnose and mitigate the problem?

Accepted Answer

Examine the latency breakdown using Cloud Monitoring to see if the location feature is causing computationally expensive operations, then consider feature engineering like bucketing coordinates.

Answer

Remove the location feature from the model and retrain without it to restore performance.

Answer

Increase the number of replicas for the endpoint to handle the increased latency.

Answer

Switch to a regional endpoint in North America to reduce latency for the majority of users.

Question 3

Your Looker dashboard uses a BigQuery connection. You notice that some queries take over a minute. Which service can you enable to cache results in memory for sub-second Looker queries?

Accepted Answer

BigQuery BI Engine. BI Engine is an in-memory analysis service that accelerates BigQuery queries by caching data in memory. It integrates with Looker and Looker Studio. Cloud Memorystore is a Redis/Memcached cache, not directly for BigQuery. BigQuery BI Engine is the correct service.

Answer

Cloud SQL

Answer

Cloud Bigtable

Answer

Cloud Memorystore

Question 4

A company needs to process streaming data from IoT devices with sub-second latency and exactly-once processing guarantees. Which Google Cloud service should they use?

Accepted Answer

Cloud Dataflow. Cloud Dataflow is the correct choice because it provides a unified stream and batch processing model with exactly-once processing guarantees and sub-second latency via its Apache Beam SDK. It supports event-time processing, watermarks, and triggers to handle out-of-order data from IoT devices while ensuring each record is processed exactly once, even in the case of failures.

Answer

BigQuery

Answer

Cloud Dataproc

Answer

Cloud Pub/Sub

Question 5

You are using AI Platform Prediction (now Vertex AI) for online predictions. You notice that some requests are failing with a 503 status code. Which is the most likely cause?

Accepted Answer

The model is experiencing high traffic and the underlying nodes are still scaling up. A 503 status code in Vertex AI (formerly AI Platform Prediction) indicates that the prediction service is temporarily unavailable, most commonly due to autoscaling latency. When a model receives a sudden spike in traffic, the underlying nodes (compute instances) may still be provisioning and initializing, causing requests to be rejected until the new nodes are ready to serve. This is a transient condition that resolves once scaling completes.

Answer

The input data format does not match the model's expected schema

Answer

The project has exceeded its prediction requests quota

Answer

The service account used for prediction does not have the required permissions

Question 6

A company is building a continuous training pipeline that retrains a model daily using new data from a feature store. The training data must include features computed up to the timestamp of each training run. Which architecture should be used to ensure time-consistent feature values without label leakage?

Accepted Answer

Use Vertex AI Feature Store with point-in-time lookup enabled to retrieve features as of the training timestamp.. Vertex AI Feature Store's point-in-time lookup retrieves the exact feature values as they existed at the specified training timestamp, ensuring time-consistency and preventing label leakage. This mechanism avoids using future data that would not have been available at the time of prediction, which is critical for realistic model evaluation and production performance.

Answer

Train on a fixed window of the most recent features without considering timestamps.

Answer

Store all features in a Cloud SQL database and perform a join at training time.

Answer

Use Pub/Sub to stream new features into Cloud Storage and train on the latest snapshot.

Question 7

An e-commerce company deploys a recommendation model on Vertex AI Endpoints. The endpoint receives a high volume of requests with a large payload. They notice high latency and occasional timeouts. Which action should they take to improve performance without sacrificing accuracy?

Accepted Answer

Enable request batching on the endpoint. Enabling request batching on the Vertex AI endpoint allows multiple inference requests to be grouped into a single prediction call, reducing per-request overhead and improving throughput. This directly addresses high latency and timeouts caused by a high volume of large payloads without altering the model or its accuracy.

Answer

Switch to a smaller machine type

Answer

Reduce the model size by pruning

Answer

Increase the number of replicas

Question 8

An e-commerce application uses Cloud SQL (MySQL) for transaction processing. To improve read performance for reporting queries, the team wants to offload read traffic to a separate database instance that stays in sync with the primary. Which Cloud SQL feature should they use?

Accepted Answer

Create a cross-region read replica. Cloud SQL read replicas are designed to offload read traffic from the primary instance while staying in sync using asynchronous replication. Cross-region read replicas specifically allow you to place a replica in a different region, improving read performance for geographically distributed reporting queries without affecting the primary's transaction processing.

Answer

Configure a high availability (HA) replica

Answer

Use Cloud SQL's failover replica

Answer

Enable automatic backups and point-in-time recovery

Question 9

A company is designing a streaming pipeline using Dataflow to process real-time clickstream data. The pipeline reads from Pub/Sub, performs user sessionization using Apache Beam's Session window, and writes to BigQuery. The team notices that the pipeline's lag is growing and the worker utilization is low. What is the most likely cause and recommended fix?

Accepted Answer

The session window gap duration is too large, causing excessive state per key; reduce the gap duration.. D is correct because a large session window gap duration causes Dataflow to maintain excessive state per key (user session), leading to high memory pressure and slow processing. This results in growing pipeline lag despite low worker utilization, as workers spend more time managing state than processing data. Reducing the gap duration limits the state size and improves throughput.

Answer

Too many workers are created; reduce the number of workers.

Answer

The pipeline is not using autoscaling; enable autoscaling.

Answer

Insufficient disk space per worker; increase the boot disk size.

Question 10

A company wants to migrate 500 TB of on-premises archival data to Cloud Storage. The data is stored on a SAN and the network link is limited to 1 Gbps. The migration must complete within 10 days. What is the MOST cost-effective approach?

Accepted Answer

Order a Transfer Appliance, copy data locally, and ship it to Google for ingestion.. The Transfer Appliance is designed for large-scale data migrations where network bandwidth is insufficient. With 500 TB at 1 Gbps, the theoretical transfer time is over 46 days, far exceeding the 10-day window. The appliance allows you to physically ship the data, bypassing network constraints entirely, making it the most cost-effective and timely solution.

Answer

Set up a Cloud VPN and use rsync over the encrypted connection.

Answer

Use BigQuery Data Transfer Service to load the data directly into BigQuery.

Answer

Use Storage Transfer Service to copy data from on-premises to GCS over the existing network.

Question 11

Your team is using Cloud Data Fusion to build batch ETL pipelines that load data from Cloud Storage into BigQuery. You have several pipelines that run daily. Recently, one pipeline started failing with a 'Permission denied' error when trying to read a new CSV file uploaded to a specific Cloud Storage bucket. Other pipelines using the same bucket succeed. The failing pipeline has a Cloud Storage source plugin that uses a service account with the roles/storage.objectViewer role. The bucket has uniform bucket-level access enabled. What is likely causing the issue?

Accepted Answer

Check that the service account used by the failing pipeline's Data Fusion instance has the correct permissions, and ensure that the service account is the same as the one used by working pipelines.. The error is likely due to the Data Fusion instance's service account, not the source plugin's service account. In Cloud Data Fusion, the pipeline execution uses the service account attached to the Data Fusion instance itself to access Cloud Storage, even if the source plugin specifies a different service account. Since other pipelines using the same bucket succeed, the issue is that the failing pipeline's Data Fusion instance uses a service account that lacks the roles/storage.objectViewer role on the bucket, while working pipelines use an instance with the correct permissions.

Answer

Create a custom IAM role with storage.buckets.get and storage.objects.get permissions and assign it to the service account.

Answer

Disable uniform bucket-level access and add bucket ACLs for the service account.

Answer

Add the service account as a member of the Cloud Storage bucket with the roles/storage.objectViewer role.

Question 12

Your Dataflow streaming pipeline is processing financial transactions and writing results to BigQuery. You need to monitor the pipeline for data freshness (end-to-end latency) and alert if it exceeds 5 minutes. The pipeline uses fixed windows of 1 minute. Which metrics should you use for alerting?

Accepted Answer

System Lag metric from Dataflow monitoring.. System Lag in Dataflow measures the maximum time that a data element waits in the pipeline before being processed, which directly reflects end-to-end latency. For a streaming pipeline with 1-minute fixed windows, System Lag indicates how far behind the pipeline is from real-time, making it the correct metric to alert on when data freshness exceeds 5 minutes.

Answer

Data Freshness metric from BigQuery monitoring.

Answer

Element Count metric from Dataflow monitoring.

Answer

Worker Threads Utilization metric from Dataflow monitoring.

Question 13

A company stores IoT sensor readings in BigQuery. The table is partitioned by day and clustered by sensor_id. Query performance has degraded as data grows; many queries filter by a date range and a single sensor_id. Which optimization should be applied first?

Accepted Answer

Recluster the table to ensure data is sorted by sensor_id within each partition.. Reclustering the table ensures that within each daily partition, the data is physically sorted by sensor_id. This optimizes the performance of queries that filter by a date range and a single sensor_id, as BigQuery can use the clustering metadata to prune blocks and read only the relevant data, reducing the amount of data scanned and improving query speed.

Answer

Remove clustering on sensor_id as it may cause overhead.

Answer

Add a WHERE clause to filter by partition date even if the query already filters by a date range.

Answer

Increase the number of BigQuery slots assigned to the project.

Question 14

A data engineer needs to design a BigQuery dataset for a multi-team environment. Each team should have read access only to specific tables, and the data must be protected from accidental deletion. Which THREE steps should they take?

Accepted Answer

Create authorized views for each team and grant access to the views. Authorized views allow sharing specific table data without direct table access. Dataset-level IAM is too broad. Deletion protection prevents accidental table drops. Clustering improves performance but not access control.

Answer

Cluster tables by team_id to improve performance

Answer

Grant bigquery.dataViewer on the dataset to all teams

Question 15

A data engineer needs to process a large dataset (500 TB) stored in Cloud Storage using Dataproc. The processing job requires reading the entire dataset and writing results back to Cloud Storage. The job is expected to run for 6 hours. Which configuration minimizes cost?

Accepted Answer

Use a cluster with a mix of standard and preemptible VMs.. Preemptible VMs cost about 80% less than standard VMs, and mixing them with standard VMs provides fault tolerance for the job's 6-hour duration. Since the job reads and writes to Cloud Storage (not local HDFS), local SSDs are unnecessary, and a single-node cluster would lack the parallelism needed to process 500 TB efficiently within 6 hours. Using a mix of standard (for critical master/worker nodes) and preemptible VMs (for worker nodes) minimizes cost while ensuring job completion.

Answer

Use a single-node cluster with standard VMs.

Answer

Use a cluster with local SSDs for faster I/O.

Answer

Use a cluster with n1-highmem-32 instances and 1000 cores.

Question 16

A retail company needs to generate product recommendations for millions of users every few hours. The model is a small scikit-learn model. Which prediction method should be used to minimize infrastructure cost while meeting the latency requirements?

Accepted Answer

Use a Vertex AI batch prediction job that reads from BigQuery and writes results back to BigQuery or Cloud Storage.. Batch prediction is the most cost-effective approach for generating recommendations for millions of users every few hours. Vertex AI batch prediction jobs process large datasets in parallel without maintaining always-on infrastructure, and they can read from BigQuery and write results directly to BigQuery or Cloud Storage, minimizing compute costs while meeting the latency requirement of 'every few hours' (not real-time).

Answer

Use Cloud Run to host the model and invoke it for each user request.

Answer

Export the model as a container and run on Google Kubernetes Engine with cluster autoscaling.

Answer

Deploy the model to a Vertex AI endpoint with a single replica for online predictions.

Question 17

A company runs batch jobs on Dataproc. They need to ensure that if a job fails, it automatically retries with exponential backoff. What is the recommended approach?

Accepted Answer

Use Dataproc Workflow Templates with the maxAttempts parameter set to 3.. Dataproc Workflow Templates natively support automatic retries with exponential backoff via the `maxAttempts` parameter. This allows you to define a workflow that, upon failure, will retry the job with increasing delays between attempts, meeting the requirement without custom scripting or external services.

Answer

Schedule a cron job to check job status and restart manually.

Answer

Use a Cloud Function triggered by Stackdriver alerts to restart the job.

Answer

Create a Cloud Composer DAG that monitors job status and retries on failure.

Question 18

A company uses Cloud Spanner for a global e-commerce platform. They have a table of orders and a table of order items. To optimize performance for queries that join these tables on order_id, which Spanner schema design feature should they use?

Accepted Answer

Use interleaved tables with order_items as a child table of orders. Interleaved tables store child rows physically with parent rows, reducing join latency. Secondary indexes are for filtering. Partitioned tables not in Spanner. Denormalization could help but interleaved tables are the designed approach.

Answer

Use Cloud Bigtable instead

Answer

Create a secondary index on order_id

Answer

Denormalize the order items into the orders table using repeated fields

Question 19

A streaming pipeline ingests events from Pub/Sub, enriches them via a slow REST API call, and writes the result to BigQuery. The API has a limit of 10 requests per second per client. The pipeline processes 1000 messages per second. Which approach minimizes latency while respecting API limits?

Accepted Answer

Use a global window with a trigger that fires every second, and inside the DoFn limit concurrent API calls to 10.. Using a global window with a trigger every second groups 1000 messages into a batch, and then throttling concurrent API calls to 10 within the DoFn (e.g., using a fixed-size thread pool) respects the API limit while minimizing latency by processing messages in parallel up to the limit. Option B is wrong because fanning out to multiple API instances doesn't help if the limit is per client; the total requests per second across all instances would still exceed the client limit. Option C is wrong because Dataflow Flex Templates are used to run parameterized pipelines, not to solve throttling issues. Option D is wrong because assigning a random key and using a sliding window distributes messages across workers, but without explicit throttling, the API limit could still be exceeded.

Answer

Fan out the stream to multiple REST API instances using Pub/Sub topic splitting.

Answer

Use a Dataflow Flex Template to run multiple pipelines, each processing a subset of messages.

Answer

Assign each message a random key and use a sliding window of 10 seconds; the API call will be distributed across workers.

Question 20

The exhibit shows a Spark job submitted to Dataproc that fails with an out-of-memory error. Which change should be made to the submission command to resolve the issue?

Accepted Answer

Add --properties spark.executor.memory=8g to the command.. The out-of-memory error indicates that the Spark executors do not have enough memory to process the data. Adding `--properties spark.executor.memory=8g` increases the memory allocated to each executor, directly addressing the root cause. This property overrides the default executor memory (typically 1g or 4g depending on the cluster configuration) and is the standard way to tune executor memory in Spark on Dataproc.

Answer

Use a different Spark example class.

Answer

Increase the number of worker nodes in the cluster.

Answer

Add --driver-memory 8g to the command.