PDE Practice Test 42 — 15 Questions

Question 1

A startup needs a fully managed, serverless Spark service to run occasional data processing jobs without managing clusters. They want to pay only for the resources used during job execution. Which Google Cloud service should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Dataproc Serverless. Dataproc Serverless provides a serverless Spark environment where you pay per job execution. Cloud Data Fusion is for visual ETL. Dataproc is managed but not serverless. Dataflow is serverless for Beam, not Spark.

Answer

Dataflow

Answer

Cloud Data Fusion

Answer

Dataproc

Question 2

You need to schedule a Dataproc Spark job to run at 2 AM every day, and upon completion, trigger a BigQuery load job. Which Cloud Composer operator should you use to run the Spark job?

Accepted Answer

DataprocSubmitJobOperator. The DataprocSubmitJobOperator is specifically designed to submit a job (e.g., a Spark job) to an existing Dataproc cluster. In this scenario, you need to run a Spark job on a scheduled basis, and Cloud Composer (Airflow) provides this operator to submit the job to Dataproc. After the Spark job completes, you can chain a BigQuery load operator to trigger the load, matching the requirement exactly.

Answer

DataflowPythonOperator

Answer

BigQueryOperator

Answer

DataprocClusterCreateOperator

Question 3

Which BigQuery function can be used to retrieve the value of a column from the previous row within a partition, ordered by a timestamp?

Accepted Answer

LAG(). LAG() is a window function that returns the value of a column from a row that is a specified number of rows before the current row within the partition. LEAD() retrieves from a following row.

Answer

FIRST_VALUE()

Answer

ROW_NUMBER()

Answer

LEAD()

Question 4

A team needs to store transactional data for an e-commerce application that requires ACID transactions, automatic backups, and point-in-time recovery. The expected workload is under 10,000 QPS. Which database should they choose?

Accepted Answer

Cloud SQL. Cloud SQL is the correct choice because it provides fully managed relational databases (MySQL, PostgreSQL, SQL Server) with built-in ACID transaction support, automated backups, and point-in-time recovery (PITR) via binary logs or write-ahead logs. The workload of under 10,000 QPS is well within Cloud SQL's performance envelope, making it a cost-effective and operationally simple solution for transactional e-commerce data.

Answer

Cloud Bigtable

Answer

Firestore

Answer

Cloud Spanner

Question 5

A team wants to ingest streaming data from millions of IoT devices and store historical data in BigQuery for analysis. They need near real-time analytics on the most recent data, with sub-second latency. Which architecture should they use?

Accepted Answer

Use Pub/Sub, then a Dataflow pipeline that filters and transforms data, writing to Cloud Bigtable for real-time queries and to Cloud Storage for periodic BigQuery loads.. It uses Cloud Bigtable for sub-second latency on recent data, which is ideal for near real-time analytics on streaming IoT data. Dataflow provides the necessary stream processing, filtering, and transformation before writing to Bigtable for low-latency queries and to Cloud Storage for periodic batch loads into BigQuery for historical analysis. This architecture decouples real-time and historical paths, meeting both latency and storage requirements.

Answer

Use Pub/Sub to receive data, then stream directly into BigQuery using the streaming API, and use standard SQL queries for real-time analytics.

Answer

Use Pub/Sub to ingest data into a Dataproc Spark Streaming job that writes to both Bigtable and BigQuery.

Answer

Use Cloud SQL to store the latest data and periodically move historical data to BigQuery via cron jobs.

Question 6

Your team wants to share a BigQuery dataset with another project while ensuring that users from that project can only query specific tables. Which BigQuery feature should you use?

Accepted Answer

Create an authorised view in your dataset and share the view with the other project. Authorised views allow you to share query results with users in other projects without giving them direct access to the underlying tables.

Answer

Use a materialised view and share the underlying table

Answer

Grant the BigQuery Data Viewer role to the other project's service account

Answer

Export the table to Cloud Storage and share the bucket

Question 7

A data engineering team is designing a streaming pipeline using Cloud Dataflow. They need to join two unbounded PCollections based on a common key. The join must handle late data up to 10 minutes. Which THREE components should they use?

Accepted Answer

CoGroupByKey transform. CoGroupByKey joins two PCollections by key. Window into fixed windows of appropriate duration. Allowed lateness and triggers handle late data.

Answer

Use a global window with triggers

Answer

Flatten transform

Question 8

A data scientist wants to test a new model version on a small percentage of traffic before full rollout. Which Vertex AI feature allows this?

Accepted Answer

Endpoint traffic splitting. Vertex AI Endpoint traffic splitting allows you to route a specified percentage of inference requests to different model versions deployed on the same endpoint. This enables gradual rollout by directing a small fraction of traffic (e.g., 5%) to the new model while the rest goes to the current version, without needing separate endpoints or manual routing logic.

Answer

A/B testing

Answer

Model monitoring

Answer

Model versioning with canary deployments

Question 9

Which THREE factors should be considered when designing a Vertex AI Pipeline for continuous training?

Accepted Answer

Cost of training and infrastructure. Cost of training and infrastructure (A) is correct because Vertex AI Pipelines incur compute costs for each pipeline run, including training, data processing, and orchestration. Continuous training amplifies these costs, so you must consider budget constraints, resource optimization (e.g., using preemptible VMs), and cost monitoring to avoid unexpected bills.

Answer

Debugging tools like Cloud Debugger

Answer

Number of model versions to keep

Question 10

A mobile app needs an offline-first NoSQL database that syncs data across devices when connectivity is available. Which Google Cloud database meets these requirements?

Accepted Answer

Firestore. Firestore is a NoSQL, serverless, offline-first database that automatically syncs data across devices when connectivity is restored. It provides built-in offline persistence and real-time synchronization, making it ideal for mobile apps that need to work offline and sync later.

Answer

Memorystore

Answer

Cloud SQL

Answer

Bigtable

Question 11

A data pipeline processes JSON files from Cloud Storage, transforms them using Apache Beam, and writes the output to BigQuery. Some records are malformed and cause the pipeline to fail. How should the engineer handle these errors to ensure the pipeline continues processing while preserving the malformed records for analysis?

Accepted Answer

Catch exceptions in a DoFn and write the malformed records to a separate Cloud Storage bucket using a FileIO sink.. It allows the pipeline to continue processing by catching exceptions within a DoFn and writing malformed records to a separate Cloud Storage bucket using a FileIO sink. This preserves the malformed records for later analysis without blocking the main data flow, which is a standard pattern in Apache Beam for handling dead-letter records. The approach ensures fault tolerance while maintaining data integrity for debugging.

Answer

Set the pipeline to retry malformed records indefinitely until they succeed.

Answer

Use a side input to send malformed records to a dead letter queue in Pub/Sub for later reprocessing.

Answer

Log the malformed records to Stackdriver and skip them in the pipeline.

Question 12

A company uses Vertex AI to serve a model. They notice that some predictions are incorrect due to data drift. What is the best way to detect and retrain the model automatically?

Accepted Answer

Use Vertex AI Model Monitoring with alerts and retraining pipeline. Vertex AI Model Monitoring is specifically designed to detect data drift and feature skew in production models. It can be configured to send alerts and trigger an automated retraining pipeline via Cloud Functions or Vertex AI Pipelines, enabling continuous model improvement without manual intervention. This directly addresses the need for automatic detection and retraining in response to data drift.

Answer

Store predictions in BigQuery and run scheduled queries

Answer

Create a Cloud Monitoring dashboard

Answer

Set up Cloud Logging metrics to monitor predictions

Question 13

A data engineer is designing a streaming pipeline with Cloud Pub/Sub and Cloud Dataflow. They need to guarantee at-least-once delivery and handle occasional duplicates. Which TWO configurations should they implement?

Accepted Answer

Use idempotent sinks. Idempotent sinks (e.g., BigQuery with insertId, Cloud Storage with object generation numbers) allow the pipeline to safely process duplicate records without causing data corruption or double-counting. In a streaming pipeline with at-least-once semantics, duplicates are inevitable, and idempotent sinks ensure that repeated writes produce the same result as a single write, maintaining data consistency.

Answer

Use global windows with triggers

Answer

Use fixed windows

Answer

Enable Dataflow Streaming Engine

Question 14

Your team has implemented a CI/CD pipeline using Cloud Composer (Apache Airflow) to retrain a model every day. The pipeline reads new data from BigQuery, trains a model using Vertex AI Training, evaluates it, and if the accuracy improves, deploys it to a Vertex AI Endpoint. For the past week, the pipeline has been running successfully but no new model has been deployed because the evaluation accuracy never exceeds the previous model's accuracy. The training data volume has been consistent. You suspect that the model is not learning from the new data. What should you do?

Accepted Answer

Examine the training data for any data quality issues such as missing values or label leakage.

Answer

Deploy the new model anyway and run an A/B test in production to see if it performs better online.

Answer

Increase the training budget or number of training steps to allow the model to converge better.

Answer

Change the evaluation metric to a different one that may show improvement, such as F1 score instead of accuracy.

Question 15

A financial services company operates a real-time fraud detection pipeline using Apache Beam running on Google Cloud Dataflow. The pipeline reads transactions from Pub/Sub, enriches them with customer data from Bigtable, runs a machine learning model with side inputs from a Redis cluster, and writes results to BigQuery for downstream reporting. The data must be processed with exactly-once semantics to avoid duplicate fraud alerts or missing transactions. The pipeline currently uses a global window with 5-minute accumulation, but the team is experiencing high latency and occasional duplicates when the model side input is updated (triggered every 15 minutes via a WatchTransform). Additionally, the pipeline has a dead letter queue that outputs failed records to a separate Pub/Sub topic, but these records are never reprocessed. The team needs to ensure high reliability and data quality. Which course of action should the team take to improve solution quality?

Accepted Answer

Implement sliding windows of 5 minutes with a 2-minute allowed lateness, use side inputs with periodic refreshes using the .withUpdateFrequency transformation, and set up a Cloud Function to automatically replay dead letter records back to the main Pub/Sub topic after fixing the issue.. Sliding windows with 2-minute allowed lateness handle late-arriving data without causing duplicates, and the .withUpdateFrequency transformation refreshes the side input every 15 minutes without triggering reprocessing, reducing latency. Replaying dead letter records via a Cloud Function ensures data completeness. Option A is incorrect because fixed windows with session gaps do not address side input latency and may lose late events. Option B is incorrect because batch processing is unsuitable for real-time fraud detection and introduces significant latency. Option D is incorrect because custom triggers with early firings can cause duplicates due to side input updates, and the Read transform does not efficiently handle periodic model refreshes.

Answer

Use fixed windows with a 10-minute duration and session gap of 2 minutes, disable side input caching, and log all dead letter records to Cloud Storage for manual inspection.

Answer

Switch to a batch processing approach that runs every minute using Cloud Composer, with data loaded from Pub/Sub into BigQuery and then processed with Dataproc to run the model.

Answer

Keep the global window but use a custom trigger with early firings every 30 seconds and a late-firing threshold of 1 minute, and configure the side input to be broadcast every 5 minutes using a Read transform.