PDE Practice Test 49 — 15 Questions

Question 1

A BigQuery table contains streaming data from Cloud Pub/Sub. The table is partitioned by ingestion time. A user runs a query that accesses data from the last 5 minutes and gets correct results. After 90 minutes, the user runs the same query again but notices that some rows are missing. What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The query is using time travel to a snapshot before the streaming buffer was committed. BigQuery's streaming buffer provides low-latency access to recently ingested data, but this data is not immediately committed to managed storage. After the streaming buffer is flushed (typically within 90 minutes), the data becomes available in the table's base storage. If the user runs a query using time travel (e.g., `FOR SYSTEM_TIME AS OF`) to a snapshot taken before the buffer was committed, the query will only see data that was in managed storage at that snapshot time, missing rows that were still in the streaming buffer at that point.

Answer

The query is using cached results that exclude recent data

Answer

The schema of the table was modified after the initial query

Answer

The table has a partition expiration of 30 days

Question 2

Drag and drop the steps to create a Cloud Composer environment for Apache Airflow into the correct order.

Accepted Answer

Enable Cloud Composer API → Create Cloud Storage bucket → Create Cloud Composer environment → Upload DAGs. Cloud Composer provides a managed Airflow environment for orchestrating workflows.

Answer

Create Cloud Storage bucket → Enable Cloud Composer API → Create Cloud Composer environment → Upload DAGs

Answer

Upload DAGs → Create Cloud Composer environment → Enable Cloud Composer API → Create Cloud Storage bucket

Answer

Create Cloud Composer environment → Enable Cloud Composer API → Create Cloud Storage bucket → Upload DAGs

Question 3

A team is using BigQuery to analyze petabyte-scale data. They notice that queries are slow and expensive due to full table scans. They have already partitioned by date. What additional optimization should they implement?

Accepted Answer

Cluster by frequently filtered columns. Clustering by frequently filtered columns (option B) organizes data within each partition based on the sort order of those columns. This allows BigQuery to prune blocks during query execution, significantly reducing the amount of data scanned and improving both performance and cost. Since the table is already partitioned by date, clustering adds a secondary ordering that targets the most common filter predicates, avoiding full table scans within each partition.

Answer

Use materialized views

Answer

Convert to native tables

Answer

Use query caching

Question 4

Your Vertex AI model deployed on an endpoint is experiencing high tail latency during online predictions. The model uses a large embedding layer, and the input size varies. You have enabled automatic scaling with a minimum of 2 replicas and maximum of 10. What is the most likely cause of the latency spikes and the best first step to diagnose?

Accepted Answer

The endpoint's target CPU utilization might be set too low, causing rapid scale-down and cold starts. Check Cloud Logging for scaling events.. High tail latency with variable input sizes and a large embedding layer often points to cold starts from aggressive scaling. When the target CPU utilization is set too low, the endpoint scales down quickly during lulls, and a subsequent burst of requests forces new replicas to spin up, causing latency spikes. Checking Cloud Logging for scaling events is the best first step because it directly reveals whether the endpoint is scaling down and then experiencing cold starts.

Answer

The model's SavedModel is too large due to the embedding layer; reduce embedding dimensions to lower latency.

Answer

The model uses a custom prediction routine that is not optimized; use tf.function to improve performance.

Answer

Enable model monitoring for online prediction and add a buffer to the endpoint's machine type.

Question 5

A company is designing a data processing pipeline for real-time sensor data. They want to ensure low latency and exactly-once processing semantics. Which two Google services should they combine to achieve this? (Choose 2)

Accepted Answer

Cloud Pub/Sub with exactly-once delivery. Cloud Pub/Sub with exactly-once delivery (Option C) ensures that each message is delivered to subscribers exactly once, preventing duplicates in the pipeline. Cloud Dataflow with exactly-once processing mode (Option D) provides end-to-end exactly-once semantics by leveraging consistent snapshots and idempotent sinks, which is critical for real-time sensor data pipelines requiring low latency and accuracy.

Answer

Cloud Dataproc with Spark Streaming

Answer

Cloud Functions with Cloud Pub/Sub triggers

Answer

Cloud IoT Core with device gateways

Question 6

Which TWO approaches are recommended for handling late-arriving data in a streaming Dataflow pipeline?

Accepted Answer

Use side inputs to provide default values for late data.. Side inputs in Apache Beam (the programming model underlying Dataflow) allow you to provide default values or supplementary data to handle late-arriving elements gracefully. When a late element arrives after the window has been emitted, a side input can supply a fallback value, ensuring the pipeline can still process the data without discarding it. This approach is recommended for handling late data in streaming pipelines where completeness is not critical.

Answer

Use fixed windows with a duration of 1 second to minimize lateness.

Answer

Set the trigger to fire only at the end of the window.

Answer

Use a filter transform to drop late-arriving elements.

Question 7

A data pipeline ingests streaming data from Pub/Sub into BigQuery via Dataflow. Recently, the pipeline has been failing with 'deadline exceeded' errors. What is the most likely cause?

Accepted Answer

The Pub/Sub subscription's acknowledgement deadline is too short for the processing time.. 'deadline exceeded' errors in a Dataflow pipeline reading from Pub/Sub indicate that the subscriber is taking longer to process messages than the acknowledgement deadline allows. When the deadline expires, Pub/Sub redelivers the message, causing duplicate processing and eventual pipeline failure. This is a common issue when processing time exceeds the default 10-second acknowledgement deadline.

Answer

The BigQuery streaming quota is exceeded.

Answer

Dataflow workers are underutilized due to batch size settings.

Answer

Dataflow autoscaling is disabled.

Question 8

A company wants to stream real-time clickstream data from a website into BigQuery for near-real-time analytics. They expect peaks of 10,000 events per second. Which combination of services is most suitable for ingestion?

Accepted Answer

Pub/Sub → Dataflow → BigQuery (Storage Write API). Pub/Sub is designed for high-throughput event ingestion, Dataflow provides real-time stream processing, and the BigQuery Storage Write API offers exactly-once semantics and high throughput. Therefore, option C is the most suitable. Option A uses Cloud Functions, which are not designed for high-throughput streaming (10,000 events per second would likely exceed typical limits). Option B sends data directly from the web to Dataflow without a buffer, which could lead to data loss during spikes; Pub/Sub provides a durable buffer. Option D uses legacy streaming inserts, which are deprecated and have lower throughput and no exactly-once guarantees.

Answer

Cloud Storage → Cloud Functions → BigQuery

Answer

Direct Web → Dataflow → BigQuery

Answer

Pub/Sub → Dataflow → BigQuery (legacy streaming inserts)

Question 9

A data pipeline ingests sensor data from IoT devices via Cloud Pub/Sub, processes it with Cloud Dataflow, and writes to BigQuery. The pipeline is failing with high latency and data loss. Which troubleshooting step should be taken first?

Accepted Answer

Check Stackdriver logging for error messages.. Stackdriver (now Cloud Logging) is the first place to investigate when a Dataflow pipeline experiences high latency and data loss. Dataflow automatically logs errors, worker failures, and system messages to Cloud Logging, which can reveal root causes such as insufficient resources, stuck steps, or Pub/Sub subscription issues. Checking logs first avoids premature scaling or configuration changes that may not address the actual problem.

Answer

Disable exactly-once processing in Dataflow.

Answer

Increase the number of Dataflow workers.

Answer

Switch to BigQuery streaming inserts.

Question 10

A data engineer is responsible for a batch ETL pipeline that runs daily using Cloud Composer and Dataproc. The pipeline extracts data from Cloud SQL, transforms it with Spark, and loads to BigQuery. Last night, the pipeline failed because the Spark job ran out of memory. The team needs a solution that prevents future failures without manual intervention. Options: A. Enable Dataproc autoscaling and configure memory-based scaling. B. Use Cloud Functions to retry the job. C. Use a larger machine type for Dataproc. D. Split the Spark job into multiple stages.

Accepted Answer

Enable Dataproc autoscaling and configure memory-based scaling. Enabling Dataproc autoscaling with memory‑based scaling allows the cluster to automatically add worker nodes when memory pressure increases, preventing Spark job failures due to out‑of‑memory errors. This dynamic, automated approach addresses the root cause without manual intervention. Static changes like a larger machine type may not accommodate future workload growth, while retrying or manually splitting the job does not solve the underlying resource insufficiency.

Answer

Use Cloud Functions to retry the job

Answer

Use a larger machine type for Dataproc

Answer

Split the Spark job into multiple stages

Question 11

A company uses BigQuery to run reporting queries on a table that is partitioned by date and clustered by customer_id. Queries filtering by customer_id and a date range are performing poorly. What is the most likely cause?

Accepted Answer

The date range filter is too wide, causing scans of many partitions. When a table is partitioned by date and clustered by customer_id, queries that filter on both columns can still perform poorly if the date range filter is too wide, causing BigQuery to scan many partitions. Even with clustering, scanning a large number of partitions negates the benefit of clustering, as clustering only reduces the data scanned within each partition. The query optimizer must read all partitions that fall within the date range, and if that range is broad, the scan overhead dominates.

Answer

The project lacks sufficient BigQuery slot capacity

Answer

The table is too large for BigQuery

Answer

Clustering column order should be date first, then customer_id

Question 12

A company is migrating their on-premises data warehouse to BigQuery. They have a mix of batch and streaming ingestion. The data team wants to optimize query costs. Which THREE practices should they adopt?

Accepted Answer

Use materialized views for frequently executed aggregations.. Partitioning by date reduces bytes scanned. Using clustered tables improves performance for filter/join queries. Using materialized views can precompute aggregations and reduce scans. Flat-rate pricing is about reservation management, not cost optimization per query. Limiting slots is not a cost optimization; it may cause throttling.

Answer

Switch to flat-rate pricing to cap slot usage.

Answer

Limit the number of concurrent queries by setting a maximum slot capacity.

Question 13

Which TWO actions can reduce the cost of running a Dataproc cluster for a nightly batch job?

Accepted Answer

Use preemptible VMs for worker nodes.. Preemptible VMs (Option C) are significantly cheaper than standard VMs because Compute Engine can terminate them at any time, making them ideal for fault-tolerant, stateless batch jobs like nightly data processing on Dataproc. Deleting the cluster after the job completes (Option E) eliminates ongoing compute costs for idle resources, which is a best practice for ephemeral workloads.

Answer

Increase the number of worker nodes for faster processing.

Answer

Use high-memory machine types for master node.

Answer

Attach local SSDs to all nodes.

Question 14

A Dataflow pipeline with multiple steps uses a side input from a slowly changing reference table stored in BigQuery. The side input is updated every hour. To avoid reprocessing the entire pipeline on each update, which approach should you use?

Accepted Answer

Use a side input with a custom 'AsIterable' and a 'Repeatable' trigger that refreshes the side input every hour. The side input should be read periodically using a side input pattern with a Repeatable trigger. This refreshes the side input without restarting the pipeline.

Answer

Use a side input with a periodic refresh via a DoFn that reads BigQuery on each element

Answer

Use the side input with a default trigger and 'withAllowedLateness'

Answer

Use a global window with a trigger that fires every hour

Question 15

You need to react to changes in a GCS bucket (e.g., new object creation) and trigger a Cloud Run service to process the new file. Which Google Cloud service should you use to route the event?

Accepted Answer

Eventarc. Eventarc is the correct choice because it is purpose-built to route events from Google Cloud sources (like Cloud Storage) to Cloud Run. It directly supports Cloud Storage audit logs and Pub/Sub event triggers, allowing you to react to object creation events without custom middleware. Eventarc handles the event routing, filtering, and delivery to your Cloud Run service automatically.

Answer

Pub/Sub directly with a Cloud Run subscription

Answer

Cloud Tasks

Answer

Cloud Scheduler