Scenario practice questions

You are responsible for monitoring a production ML model on Vertex AI. The model predicts loan approval probability. The business team reports that the model's predictions are becoming less accurate over the last week. You check the model's monitoring dashboard and see that the prediction distribution has changed significantly. What is the most likely issue?

Trap 1: The model is suffering from overfitting to the training data.

Overfitting would cause poor generalization from the start, not a gradual decline after deployment.

Trap 2: There is a bug in the model's preprocessing code.

A code bug would cause consistent errors, not gradual accuracy decline.

Trap 3: There is data drift in the input features.

Data drift would change input distribution, but the question says prediction distribution changed; it could be concept drift.

A
The model is suffering from overfitting to the training data.
Why wrong: Overfitting would cause poor generalization from the start, not a gradual decline after deployment.
B
There is a bug in the model's preprocessing code.
Why wrong: A code bug would cause consistent errors, not gradual accuracy decline.
C
There is data drift in the input features.
Why wrong: Data drift would change input distribution, but the question says prediction distribution changed; it could be concept drift.
D
The model is experiencing concept drift.
Concept drift means the underlying relationship between features and target has changed, causing prediction distribution to shift and accuracy to drop.

Question 2easymultiple choice

A company is designing a streaming data pipeline to process real-time clickstream events. They need to aggregate events by session window with a 5-minute gap and enable exactly-once processing semantics. Which Google Cloud service should they use?

Trap 1: Cloud Pub/Sub with Cloud Functions

Cloud Functions are stateless and do not natively support session windowing.

Trap 2: Cloud Dataproc with Spark Streaming

Spark Streaming provides at-least-once semantics by default; exactly-once requires additional configuration.

Trap 3: Cloud Bigtable with Dataflow templates

Bigtable is a NoSQL database, not designed for stream windowing; templates are limited in flexibility.

A
Cloud Pub/Sub with Cloud Functions
Why wrong: Cloud Functions are stateless and do not natively support session windowing.
B
Cloud Dataflow with Apache Beam
Dataflow with Beam natively supports session windows and exactly-once processing via its processing guarantees.
C
Cloud Dataproc with Spark Streaming
Why wrong: Spark Streaming provides at-least-once semantics by default; exactly-once requires additional configuration.
D
Cloud Bigtable with Dataflow templates
Why wrong: Bigtable is a NoSQL database, not designed for stream windowing; templates are limited in flexibility.

Question 3hardmultiple choice

A company's Dataflow pipeline uses the PubSubIO source to read messages and writes to BigQuery via the BigQueryIO sink. The pipeline is running in Streaming mode with exactly-once semantics enabled. Occasionally, duplicate rows appear in BigQuery. What is the most likely reason?

Trap 1: The pipeline is using the WriteResult method with WRITE_APPEND in…

In streaming mode, BigQueryIO uses streaming inserts by default, not batch.

Trap 2: The pipeline is experiencing the 'dataflow streaming log…

This is not a known bug; Dataflow handles duplicates internally.

Trap 3: The PubSubIO source is configured with a dead-letter queue and…

Dead-letter queues prevent redelivery; if configured correctly, duplicates should not occur.

A
The user-provided record ID for deduplication in BigQuery's streaming inserts is not being set for all messages, leading to duplicate rows.
BigQueryIO uses insertId for deduplication; if it's missing or inconsistent, duplicates can occur.
B
The pipeline is using the WriteResult method with WRITE_APPEND in batch mode, which can cause duplicates if retries happen.
Why wrong: In streaming mode, BigQueryIO uses streaming inserts by default, not batch.
C
The pipeline is experiencing the 'dataflow streaming log processing' bug, causing duplicate logs to be written.
Why wrong: This is not a known bug; Dataflow handles duplicates internally.
D
The PubSubIO source is configured with a dead-letter queue and messages are being redelivered without proper deduplication.
Why wrong: Dead-letter queues prevent redelivery; if configured correctly, duplicates should not occur.

Read the full NAT/PAT explanation →

Question 4hardmultiple choice

A data pipeline uses Cloud Pub/Sub to ingest events, then a Dataflow job writes to Cloud Storage in Avro format. The Dataflow job uses Global windows with a 10-minute trigger. The data is later loaded into BigQuery. They notice duplicate rows in BigQuery because the trigger produced multiple panes. What should the Dataflow pipeline change to eliminate duplicates?

Trap 1: Use a sharded output to Cloud Storage with unique filenames

Sharded output can avoid overwrites but does not prevent multiple panes from producing duplicate rows that are later loaded.

Trap 2: Write to a staging table and use a MERGE statement in BigQuery

This is a post-processing step, not a pipeline change, and requires additional scheduling.

Trap 3: Use a session window instead of global window

Session windows group events by activity, but do not eliminate duplicates from multiple trigger panes.

A
Enable exactly-once sink to BigQuery via Dataflow
Dataflow's exactly-once sink to BigQuery uses record IDs to deduplicate, preventing duplicates caused by trigger panes.
B
Use a sharded output to Cloud Storage with unique filenames
Why wrong: Sharded output can avoid overwrites but does not prevent multiple panes from producing duplicate rows that are later loaded.
C
Write to a staging table and use a MERGE statement in BigQuery
Why wrong: This is a post-processing step, not a pipeline change, and requires additional scheduling.
D
Use a session window instead of global window
Why wrong: Session windows group events by activity, but do not eliminate duplicates from multiple trigger panes.

Question 5mediummultiple choice

What is the most likely cause of data duplication after this command?

Network Topology

Trap 1: The Pub/Sub source is not exactly-once.

Pub/Sub exactly-once would prevent duplicates from source, but duplication is likely from sink side.

Trap 2: The pipeline uses at-least-once semantics.

At-least-once can cause duplicates but the pipeline was working before, so the change likely caused it.

Trap 3: The snapshot was taken before scaling.

Snapshots are used for recovery, not for causing duplicates.

A
The Pub/Sub source is not exactly-once.
Why wrong: Pub/Sub exactly-once would prevent duplicates from source, but duplication is likely from sink side.
B
The pipeline uses at-least-once semantics.
Why wrong: At-least-once can cause duplicates but the pipeline was working before, so the change likely caused it.
C
The snapshot was taken before scaling.
Why wrong: Snapshots are used for recovery, not for causing duplicates.
D
The BigQuery sink is not idempotent.
If the sink is not idempotent, duplicate data can be written when workers are re-added or when job state is replayed.

Question 6mediummultiple choice

You have deployed a classification model on Vertex AI Endpoints. The model's training data had a balanced class distribution, but over time, the production data has shifted such that one class appears 90% of the time. The model's overall accuracy remains high, but the recall for the minority class has dropped significantly. What is the best approach to detect and address this issue?

Trap 1: Retrain the model daily on the entire historical dataset

Retraining on old data can dilute recent shift; better to use sliding window or weighted approach.

Trap 2: Increase the number of replicas on the endpoint to reduce latency

Scaling replicas improves throughput/latency but does not affect model accuracy or drift.

Trap 3: Adjust the decision threshold to improve minority class recall

Adjusting threshold helps recall but does not address the underlying data drift; performance may degrade further.

A
Retrain the model daily on the entire historical dataset
Why wrong: Retraining on old data can dilute recent shift; better to use sliding window or weighted approach.
B
Set up Vertex AI Model Monitoring to detect skew and drift, and retrain using a sliding window of recent data
Model Monitoring detects skew/drift; retraining on recent data adapts to new distribution.
C
Increase the number of replicas on the endpoint to reduce latency
Why wrong: Scaling replicas improves throughput/latency but does not affect model accuracy or drift.
D
Adjust the decision threshold to improve minority class recall
Why wrong: Adjusting threshold helps recall but does not address the underlying data drift; performance may degrade further.