PDE Practice Test 16 — 15 Questions

Question 1

A company is designing a streaming data pipeline to process real-time clickstream events. They need to aggregate events by session window with a 5-minute gap and enable exactly-once processing semantics. Which Google Cloud service should they use?

Accepted Answer

Cloud Dataflow with Apache Beam. Cloud Dataflow with Apache Beam is the correct choice because it provides native support for session windows with a 5-minute gap duration and exactly-once processing semantics via its sink and source integrations. Dataflow's Beam SDK allows you to define session windows using `Window.into(Sessions.withGapDuration(Duration.standardMinutes(5)))`, and its checkpointing and idempotent writes ensure exactly-once delivery even in failure scenarios.

Answer

Cloud Pub/Sub with Cloud Functions

Answer

Cloud Dataproc with Spark Streaming

Answer

Cloud Bigtable with Dataflow templates

Question 2

A company processes financial transactions using Cloud Dataflow. They need to ensure that late-arriving data is handled correctly for fraud detection. The pipeline uses event time processing. Which approach should they use to handle late data?

Accepted Answer

Fixed windows with allowed lateness. Option C is correct because fixed windows with allowed lateness are the standard approach in Cloud Dataflow (Apache Beam) for handling late-arriving data in event-time processing. By specifying an allowed lateness duration, the pipeline retains the window state for that period, allowing late events to be correctly assigned to their original window and triggering recomputation of results. This ensures fraud detection pipelines can account for delayed transactions without missing or misordering data.

Answer

Sliding windows with early firing

Answer

Session windows with gap duration

Answer

Global windows with triggers

Question 3

A startup wants to build a data lake on Google Cloud using Cloud Storage. They need to store raw data in its original format for future analysis. Which storage class should they use to optimize for cost given that data will be accessed occasionally after the first month?

Accepted Answer

Nearline storage class. Nearline storage class is the optimal choice because it offers low-cost storage for data accessed less than once a month, with a 30-day minimum storage duration. Since the data is accessed occasionally after the first month, Nearline provides significant cost savings over Standard while still offering low-latency access (milliseconds) suitable for analytics. Coldline and Archive have lower storage costs but impose higher retrieval fees and minimum storage durations (90 and 365 days respectively), making them more expensive for data that is accessed occasionally within the first year.

Answer

Coldline storage class

Answer

Standard storage class

Answer

Archive storage class

Question 4

A company wants to ingest IoT sensor data from thousands of devices into BigQuery for near-real-time analytics. The data volume is approximately 10 GB per hour. Which combination of Google Cloud services should they use for a cost-effective and scalable solution?

Accepted Answer

Pub/Sub → Dataflow → BigQuery. Pub/Sub provides a scalable, managed ingestion layer for high-volume IoT data, decoupling producers from consumers. Dataflow (Apache Beam) processes the streaming data in near-real-time with exactly-once semantics and auto-scaling, writing directly to BigQuery for analytics. This combination minimizes operational overhead and cost by avoiding intermediate storage and manual scaling.

Answer

Cloud IoT Core → Cloud Functions → BigQuery

Answer

Cloud IoT Core → Cloud Dataproc → BigQuery

Answer

Cloud IoT Core → Cloud Storage → BigQuery load jobs

Question 5

A gaming company uses Avro schemas for its streaming event data. They anticipate adding new optional fields to events over time. They need to ensure backward compatibility so that existing pipelines continue to work. Which strategy should they adopt?

Accepted Answer

Use Avro with a schema registry that enforces backward-compatible changes. Option A is correct because Avro, combined with a schema registry, allows schema evolution with backward compatibility. The registry enforces rules such as adding optional fields with defaults, ensuring that consumers using older schemas can still deserialize new data without breaking. This directly addresses the requirement for existing pipelines to continue working as new optional fields are added.

Answer

Use JSON instead of Avro and ignore unknown fields

Answer

Use Protocol Buffers with breaking changes

Answer

Use FlatBuffers for performance

Question 6

A data team uses Cloud Dataproc to run nightly Spark jobs. The job volume has increased, and the cluster is often underutilized during the day. They want to reduce costs while ensuring jobs can scale when needed. Which strategy should they adopt?

Accepted Answer

Use a cluster with a small number of primary workers and a large pool of preemptible workers, and enable autoscaling.. Option C is correct because it combines a small number of primary (non-preemptible) workers for reliability with a large pool of preemptible workers for cost-effective scaling, and enables autoscaling to dynamically adjust the cluster size based on workload. This minimizes cost during idle periods (preemptible instances are ~80% cheaper) while ensuring jobs can scale up quickly when needed, as autoscaling adds preemptible workers automatically. Preemptible workers are ideal for fault-tolerant Spark jobs that can handle node preemptions.

Answer

Use preemptible workers for both primary and secondary nodes to minimize cost.

Answer

Manually scale the cluster up before nightly jobs and down after.

Answer

Use custom machine types with local SSDs for primary workers to improve I/O.

Question 7

A financial services company needs to process high-frequency trading data with strict ordering guarantees. They use Pub/Sub with ordering keys and Dataflow. The pipeline occasionally produces out-of-order results. What is the most likely cause?

Accepted Answer

The window trigger allows late data to be included after the main output. Option D is correct because Dataflow's default window trigger behavior allows late data to arrive after the main pane is emitted. When using Pub/Sub with ordering keys, late-arriving events (e.g., due to network delays or retries) can be assigned to the correct window but emitted in a separate pane, causing the final output to appear out-of-order relative to the event time. This is a known behavior when combining event-time windows with late data handling.

Answer

Dataflow does not preserve order when using multiple workers

Answer

Dataflow uses at-least-once processing, which can reorder events

Answer

Pub/Sub does not guarantee message ordering

Question 8

A company is designing a data processing pipeline for real-time sensor data. They want to ensure low latency and exactly-once processing semantics. Which two Google services should they combine to achieve this? (Choose 2)

Accepted Answer

Cloud Pub/Sub with exactly-once delivery. Cloud Pub/Sub with exactly-once delivery (Option C) ensures that each message is delivered to subscribers exactly once, preventing duplicates in the pipeline. Cloud Dataflow with exactly-once processing mode (Option D) provides end-to-end exactly-once semantics by leveraging consistent snapshots and idempotent sinks, which is critical for real-time sensor data pipelines requiring low latency and accuracy.

Answer

Cloud Dataproc with Spark Streaming

Answer

Cloud Functions with Cloud Pub/Sub triggers

Answer

Cloud IoT Core with device gateways

Question 9

A team has set up a push subscription to an HTTPS endpoint. They notice that messages are not being acknowledged and are resent every 10 seconds. What is the most likely issue?

Accepted Answer

The push endpoint is returning HTTP 500. In Google Cloud Pub/Sub push subscriptions, the subscriber must acknowledge messages by returning an HTTP 200 status code. If the endpoint returns HTTP 500, Pub/Sub interprets this as a failure and will retry delivery with exponential backoff, but the default minimum retry interval is 10 seconds. This matches the observed behavior of messages being resent every 10 seconds without acknowledgment.

Answer

The push endpoint is returning HTTP 200 but taking too long to process

Answer

The push endpoint is returning HTTP 200 with 'ack' in the body

Answer

The push endpoint is returning HTTP 400

Question 10

An e-commerce company processes real-time clickstream data using Pub/Sub and Dataflow. They want to ensure that if a Dataflow worker fails, the pipeline can resume processing from the point of failure without data loss. Which feature should they enable?

Accepted Answer

Snapshot-based recovery. Snapshot-based recovery (Option C) is the correct feature because Dataflow snapshots capture the entire pipeline state, including the current position in each Pub/Sub subscription and the state of all transforms. If a worker fails, the pipeline can be resumed from the exact snapshot point, ensuring no data loss and exactly-once processing semantics for the recovered data.

Answer

At-least-once delivery mode

Answer

Exactly-once processing mode

Answer

Streaming engine

Question 11

A data warehouse team uses Cloud BigQuery for analytics. They want to optimize query performance and reduce costs. Which three actions should they take? (Choose 3)

Accepted Answer

Use partitioned tables on time columns. Option A is correct because partitioning tables on time columns (e.g., DATE, TIMESTAMP) in BigQuery allows the query engine to perform partition pruning, scanning only the relevant partitions instead of the entire table. This directly reduces the amount of data read, lowering query costs and improving performance by limiting I/O to the necessary time range.

Answer

Use automatic reclustering

Answer

Use BI Engine for all queries

Question 12

A payment processing company needs to detect fraudulent transactions in real time. The system must have sub-second latency for high-value transactions and use a machine learning model. Which two components should be part of the architecture? (Choose TWO.)

Accepted Answer

Bigtable to store user profiles and transaction history for fast lookups. Bigtable is a fully managed, scalable NoSQL database that provides consistent sub-10ms latency for high-throughput read/write operations, making it ideal for real-time lookups of user profiles and transaction history in fraud detection. Its ability to handle large volumes of data with low latency supports the sub-second requirement for high-value transactions.

Answer

Cloud Storage for transaction logs

Answer

Cloud SQL to store reference data

Answer

Cloud Functions for long-running batch model training

Question 13

A company uses Cloud Storage to store IoT sensor data in JSON format. The data is ingested using a Cloud Function triggered by Cloud Storage events. They notice that when many files are uploaded simultaneously, some files are not processed and the Cloud Function logs show 'function execution timeout'. What is the most likely cause and solution?

Accepted Answer

The Cloud Function's timeout is too short; increase timeout beyond 540 seconds.. The Cloud Function logs explicitly show 'function execution timeout', which indicates the function is exceeding its configured maximum runtime. The default Cloud Functions timeout is 60 seconds, and the maximum is 540 seconds (9 minutes). When many files are uploaded simultaneously, each function invocation may take longer due to increased processing load, causing timeouts. Increasing the timeout to the maximum of 540 seconds gives the function more time to complete processing, directly addressing the logged error.

Answer

The Cloud Function is not idempotent; implement idempotency.

Answer

The Cloud Storage event notification is unreliable; switch to Pub/Sub notifications.

Answer

The Cloud Function has too few instances; increase max instances.

Question 14

A data analyst frequently queries a BigQuery table that contains an array of structs representing product purchases. The query below runs slowly:

SELECT customer_id, COUNT(purchase) as total_purchases
FROM sales, UNNEST(purchases) as purchase
GROUP BY customer_id

What change would most improve query performance?

Accepted Answer

Create a materialized view that pre-aggregates by customer_id and purchase count. The query runs slowly because it must unnest the `purchases` array for every row and then aggregate. A materialized view pre-aggregates the data by `customer_id` and purchase count, avoiding repeated full scans and unnesting. This is the most impactful optimization because it eliminates the compute cost of UNNEST and GROUP BY at query time.

Answer

Partition the table by transaction date

Answer

Use a subquery to filter purchases first

Answer

Cluster the table by purchases.product_id

Question 15

A manufacturing company wants to detect anomalies in sensor data from thousands of IoT devices in real time. The data is streaming into Pub/Sub. The best solution should use a machine learning model served from AI Platform that scores sensor readings aggregated over 5-minute windows. Which pipeline design meets these requirements?

Accepted Answer

Use Cloud Dataflow with sliding windows to aggregate sensor readings every 5 minutes, then call a trained model hosted on AI Platform Prediction for each window. Option C is correct because Cloud Dataflow's sliding windows natively handle the 5-minute aggregation requirement for streaming data, and its ability to call external services via a DoFn allows integration with AI Platform Prediction for real-time model scoring. This design aligns with the need for low-latency, scalable processing of Pub/Sub streams without managing infrastructure.

Answer

Use Cloud Dataproc with Spark Streaming to aggregate data, and use a Spark ML model embedded in the pipeline

Answer

Use BigQuery streaming inserts and run scheduled queries that call the ML model

Answer

Use Cloud Functions triggered by Pub/Sub to process each sensor reading individually