PDE Practice Test 15 — 15 Questions

Question 1

Refer to the exhibit. A BigQuery dataset has the IAM policy shown above. An analyst is trying to run a SELECT query on a table in this dataset but receives an 'Access Denied' error. What is the most likely reason?

Accepted Answer

The analyst only has the roles/bigquery.metadataviewer role, which does not allow reading table data.. The roles/bigquery.metadataviewer role grants permissions to view table and dataset metadata (e.g., table names, schemas) but does not include the bigquery.tables.getData permission required to read table rows. Therefore, when the analyst runs a SELECT query, BigQuery denies access because the role lacks the data-reading privilege. This is the most likely reason for the 'Access Denied' error.

Answer

The analyst does not have permission to list datasets in the project.

Answer

The table is in a different region than the dataset, and the analyst's query is not cross-region compatible.

Answer

The analyst has not been granted the 'bigquery.jobs.create' permission to run queries.

Question 2

A Dataflow pipeline is processing a high-volume streaming data stream. The job is lagging behind by 30 minutes, and the Dataflow monitoring UI shows high system latency with low CPU utilization. Which action should be taken to improve throughput?

Accepted Answer

Enable Dataflow Shuffle. Option C is correct because high system latency with low CPU utilization indicates a bottleneck in data shuffling, not in processing capacity. Enabling Dataflow Shuffle offloads the shuffle operation to Google-managed resources, reducing disk I/O and network overhead, which directly improves throughput in streaming pipelines.

Answer

Enable Streaming Engine

Answer

Increase the number of workers

Answer

Disable hot key detection

Question 3

You are operating a streaming data pipeline that uses Cloud Pub/Sub and Dataflow. The data source sometimes emits events that are delayed by several minutes due to network issues. Your pipeline must produce accurate aggregations (e.g., counts per minute) even for late data, but you also need to avoid waiting for a long time before emitting results. Which approach should you use?

Accepted Answer

Use event-time processing with allowed lateness and a trigger that fires early to provide speculative results.. Option B is correct because it uses event-time processing to handle late data via allowed lateness, combined with early triggers to emit speculative results before the window closes. This balances accuracy for delayed events with low latency for downstream consumers, which is a common requirement in streaming pipelines using Cloud Pub/Sub and Dataflow.

Answer

Use processing-time windows and ignore the event timestamps entirely.

Answer

Use global windows and hold all data for 24 hours before processing to ensure completeness.

Answer

Use event-time processing and discard any data that arrives after the window ends.

Question 4

Which TWO options can help reduce costs for a Dataflow batch pipeline that processes 100 GB of data daily from Cloud Storage? (Choose 2)

Accepted Answer

Use FlexRS (Flexible Resource Scheduling). FlexRS (Flexible Resource Scheduling) allows you to run batch workloads on a discounted, flexible schedule. It reduces costs by offering lower prices in exchange for the job being able to wait up to 6 hours for resources to become available. This is ideal for a daily 100 GB batch pipeline that can tolerate some scheduling delay.

Answer

Use Dataflow Prime (now Dataflow Runner v2)

Answer

Use high-memory machine types

Answer

Use Streaming Engine

Question 5

Which THREE practices are recommended when designing a Cloud Data Fusion pipeline to ensure efficient execution and monitoring? (Choose three.)

Accepted Answer

Limit the memory and disk usage per stage to avoid Dataproc node resource exhaustion.. Option B is correct because Cloud Data Fusion pipelines run on Dataproc clusters, and limiting memory and disk usage per stage prevents resource exhaustion on worker nodes. This ensures that no single stage consumes all available resources, which could cause the pipeline to fail or degrade performance. Proper resource limits help maintain stable execution and avoid out-of-memory errors.

Answer

Manually partition input files to control parallelism.

Answer

Schedule pipeline runs using Cloud Scheduler and Pub/Sub triggers to avoid manual starts.

Question 6

Your company is building a data processing system that ingests sensor data from millions of devices, processes it in near real-time to detect anomalies, and stores raw and processed data for long-term analytics. The system must meet a 99.9% uptime SLA and minimize data loss. Which THREE design choices are best? (Choose three.)

Accepted Answer

Use Cloud Pub/Sub as the ingestion layer with a dead-letter topic to capture unprocessed messages.. Cloud Pub/Sub with a dead-letter topic ensures that messages that cannot be processed are captured and not lost, directly supporting the requirement to minimize data loss. The dead-letter topic allows for later reprocessing or analysis of failed messages, which is critical for meeting a 99.9% uptime SLA by preventing message backlogs from blocking the ingestion pipeline.

Answer

Store raw data in Cloud Bigtable and processed data in Cloud Storage.

Answer

Use a global Cloud Load Balancer in front of the Dataflow workers.

Question 7

Your Dataflow streaming pipeline is reading from Cloud Pub/Sub and writing to BigQuery. Users report occasional data duplication in the BigQuery table. You verify the pipeline uses exactly-once processing and idempotent writes. The Dataflow monitoring shows no errors, but the pipeline has occasional worker restarts. What is the most likely cause of the duplicates?

Accepted Answer

The pipeline does not set the insertId parameter in the BigQuery streaming output.. Option D is correct because BigQuery's streaming API uses the `insertId` parameter to deduplicate records within the streaming buffer. Without a unique `insertId`, BigQuery cannot detect and discard duplicate inserts that may occur when Dataflow retries a write after a worker restart. Even with exactly-once processing in the pipeline, the BigQuery streaming endpoint itself is at-least-once, so the `insertId` is essential for deduplication.

Answer

The pipeline is using a global window with an early trigger, causing late data to be reprocessed.

Answer

The Pub/Sub subscription is configured with at-least-once delivery, causing duplicate messages.

Answer

The BigQuery table has a time-based partitioning column that is not aligned with the event timestamp.

Question 8

A company is designing a streaming pipeline using Dataflow to process real-time clickstream data. The pipeline reads from Pub/Sub, performs user sessionization using Apache Beam's Session window, and writes to BigQuery. The team notices that the pipeline's lag is growing and the worker utilization is low. What is the most likely cause and recommended fix?

Accepted Answer

The session window gap duration is too large, causing excessive state per key; reduce the gap duration.. D is correct because a large session window gap duration causes Dataflow to maintain excessive state per key (user session), leading to high memory pressure and slow processing. This results in growing pipeline lag despite low worker utilization, as workers spend more time managing state than processing data. Reducing the gap duration limits the state size and improves throughput.

Answer

Too many workers are created; reduce the number of workers.

Answer

The pipeline is not using autoscaling; enable autoscaling.

Answer

Insufficient disk space per worker; increase the boot disk size.

Question 9

A company uses Cloud Composer (Airflow) to orchestrate a data pipeline. One DAG has many tasks that run in parallel and dependencies that span multiple days. Recently, the DAG started failing with 'DagRun already exists' errors. What is the most likely cause?

Accepted Answer

The DAG has max_active_runs_per_dag set to a low number, causing overlapping runs to be rejected.. The 'DagRun already exists' error occurs when Airflow attempts to create a new DAG run for a logical date that already has an active or completed run, and the DAG's concurrency settings prevent overlapping runs. Setting max_active_runs_per_dag to a low number (e.g., 1) restricts the number of concurrent runs, so if a previous run hasn't finished or been cleared, a new run for the same or overlapping schedule interval is rejected with this error. This is the most likely cause given the DAG has dependencies spanning multiple days, which can cause runs to overlap if not properly configured.

Answer

The DAG has a large number of tasks, overwhelming the Airflow scheduler.

Answer

The DAG's schedule interval is too short, causing task instances to be created with duplicate run IDs.

Answer

The DAG has a depends_on_past set to True, causing upstream failures to block new runs.

Question 10

A retail company is building a recommendation engine that requires processing customer clickstream data in near real-time. The data is ingested via Pub/Sub, and must be joined with a lookup table of product details (updated daily) before being used for model inference. Which design pattern should they use?

Accepted Answer

Use a Dataflow pipeline that reads from Pub/Sub and uses a side input from a regularly refreshed PCollection loaded from Cloud Storage.. Option B is correct because Dataflow can read streaming data from Pub/Sub and use a side input from a regularly refreshed PCollection loaded from Cloud Storage. This pattern allows the product lookup table (updated daily) to be periodically reloaded into the pipeline as a side input, enabling efficient, low-latency enrichment of each event without per-event external calls or batch delays.

Answer

Enrich the stream by querying BigQuery for each event using a Cloud Function.

Answer

Store product details in Cloud Memorystore (Redis) and have the streaming application look up each event.

Answer

Write events to BigQuery and use scheduled queries to join with the product table in batch.

Question 11

A company is building a real-time streaming pipeline to ingest clickstream events from web servers, enrich them with user profile data from Cloud Bigtable, and aggregate metrics into BigQuery. The expected throughput is 10,000 events per second with occasional spikes up to 50,000. The data must be processed with low latency (seconds) and exactly-once semantics. Which Google Cloud service should be the core processing engine?

Accepted Answer

Cloud Dataflow (Apache Beam runner). Cloud Dataflow, as a managed Apache Beam runner, is the correct choice because it provides exactly-once processing semantics, low-latency streaming (sub-second to seconds), and autoscaling to handle throughput spikes from 10,000 to 50,000 events per second. Its unified batch and streaming model allows you to enrich clickstream events with user profile data from Cloud Bigtable via side inputs or asynchronous lookups, and write aggregated metrics to BigQuery with exactly-once guarantees using the Beam BigQuery I/O connector.

Answer

Cloud Pub/Sub with Cloud Functions

Answer

Cloud Dataproc with Apache Spark Streaming

Answer

Cloud Data Fusion

Question 12

A company needs to process sensitive data in BigQuery with column-level security. They want to allow analysts to see aggregated data but not individual records. What approach?

Accepted Answer

Use authorized views with aggregation functions. Option C is correct because authorized views in BigQuery allow you to define SQL queries that aggregate data (e.g., using SUM, COUNT, AVG) and expose only the aggregated results to analysts, while hiding individual records. This approach enforces column-level security by granting access to the view rather than the underlying table, ensuring analysts cannot query the raw data directly. It meets the requirement of seeing aggregated data without seeing individual records, leveraging BigQuery's native authorization and SQL capabilities.

Answer

Use table-level access controls

Answer

Use column-level access controls with masking

Answer

Use Cloud Data Loss Prevention to de-identify data

Question 13

Which TWO are valid approaches to handle late-arriving data in a Cloud Dataflow streaming pipeline?

Accepted Answer

Set allowed lateness on the window. Option B is correct because setting allowed lateness on a window in Cloud Dataflow allows the pipeline to wait for late-arriving data within a specified duration after the watermark passes the window end. This is a standard mechanism to handle out-of-order or delayed events without discarding them, ensuring completeness of windowed aggregations.

Answer

Change to processing time windows instead of event time windows

Answer

Discard any events that arrive after the window closes

Answer

Use a trigger that fires every second

Question 14

A company uses Dataflow to process streaming data from Pub/Sub. They notice increased processing latency. What is the most likely cause?

Accepted Answer

Insufficient workers. In Dataflow, processing latency increases most commonly due to insufficient workers, as the streaming pipeline cannot keep up with the incoming data rate when the number of Compute Engine instances is too low. This causes backpressure from Pub/Sub, leading to growing unacknowledged messages and higher end-to-end latency. Autoscaling may be delayed or limited by max worker count settings, making manual or configuration-based worker scaling the primary corrective action.

Answer

Pub/Sub subscription issue

Answer

Too many shards

Answer

Wrong machine type

Question 15

A data pipeline uses Cloud Pub/Sub to ingest events and Cloud Functions to transform and write to BigQuery. The system is experiencing data loss during Pub/Sub subscription outages. Which design change improves reliability?

Accepted Answer

Use Dataflow with at-least-once delivery and checkpointing. Dataflow with at-least-once delivery and checkpointing ensures that messages are not lost during Pub/Sub subscription outages because Dataflow tracks processing progress via checkpoints and can replay unacknowledged messages from the last checkpoint. This decouples the processing from the subscription's transient failures, providing fault-tolerant, exactly-once or at-least-once semantics depending on the sink.

Answer

Use a pull subscription with a custom app that polls frequently

Answer

Use long ack deadlines to keep messages in the subscription

Answer

Increase the timeout in Cloud Functions

PDE Practice Test 15 — 15 Questions

Exhibit

PDE Practice Test 15 — 15 Questions

Exhibit