PDE Practice Test 16 — 15 Questions

Question 1

A streaming Dataflow pipeline needs to be updated without draining the existing pipeline. Which update strategy should be used?

Accepted Answer

Replace the job with a new job using the same pipeline name. Dataflow Streaming Engine allows updates without draining by using the same pipeline name and enabling Streaming Engine.

Answer

Drain the pipeline first, then start a new one

Answer

Use a different pipeline name and cancel the old one

Answer

Stop the job, update the code, and restart

Question 2

What is the most likely cause of data duplication after this command?

Accepted Answer

The BigQuery sink is not idempotent.. BigQuery sinks in Dataflow are not idempotent by default; if the pipeline retries writes (e.g., due to worker failures or checkpoint issues), duplicate rows can be inserted into the BigQuery table. This is a known limitation: BigQuery does not support deduplication at the sink level unless you implement custom deduplication logic or use a staging table with merge operations. The command likely triggered a retry scenario, and the non-idempotent sink caused the duplication.

Answer

The Pub/Sub source is not exactly-once.

Answer

The pipeline uses at-least-once semantics.

Answer

The snapshot was taken before scaling.

Question 3

You want to monitor the latency of messages in a Pub/Sub subscription. Which Cloud Monitoring metric should you use to see the age of the oldest unacknowledged message?

Accepted Answer

pubsub.googleapis.com/subscription/oldest_unacked_message_age. The metric 'subscription/oldest_unacked_message_age' measures the age (in seconds) of the oldest unacknowledged message in a subscription. This helps track processing lag. The other metrics measure different aspects: num_undelivered_messages is count, not age; topic metrics are irrelevant for subscription lag; publish_latency is about publishing, not consumption.

Answer

pubsub.googleapis.com/subscription/num_undelivered_messages

Answer

pubsub.googleapis.com/topic/send_request_count

Answer

pubsub.googleapis.com/topic/publish_latency

Question 4

A company needs to process streaming sensor data and run both real-time analytics and batch reanalysis on historical data. They want to minimize infrastructure management. Which architecture and service combination is MOST suitable?

Accepted Answer

Kappa architecture with Pub/Sub and Dataflow for both real-time and batch processing. Kappa architecture processes all data as a stream, avoiding separate batch/speed layers. Pub/Sub ingests streaming data, and Dataflow with Apache Beam can handle both real-time and batch (replay) pipelines, minimizing infrastructure management.

Answer

Lambda architecture with Pub/Sub for streaming and Cloud Storage for batch, processed by Dataflow

Answer

Batch processing only with Dataflow and Cloud Storage, ignoring real-time needs

Answer

Kappa architecture with Pub/Sub Lite and Dataflow Serverless

Question 5

An organization uses BigQuery on-demand pricing. To control costs, they want to estimate the bytes processed by a query before running it. Which command or method should they use?

Accepted Answer

Use the bq query --dry_run command. BigQuery dry run estimates bytes processed without executing the query. It can be done via CLI with --dry_run flag or in the console.

Answer

Use bq ls to list table sizes

Answer

Use BigQuery reservations to get cost estimate

Answer

Use INFORMATION_SCHEMA.JOBS_BY_PROJECT to view past costs

Question 6

You have a BigQuery table that is used by multiple teams. To save costs, you want to provide a consistent view of the data as of a specific point in time without creating full copies. Which BigQuery feature should you use?

Accepted Answer

Table snapshot. BigQuery table snapshots provide a point-in-time copy of a table that incurs only storage costs for the snapshot (no additional slot usage). They are read-only and can be used to share data without duplicating the base table.

Answer

Authorized views

Answer

Materialized views

Answer

Table clone

Question 7

A company wants to use BigQuery materialized views to accelerate queries on a table that is updated every hour. Which statement about materialized views is true?

Accepted Answer

Materialized views are automatically updated when the base table changes.. BigQuery materialized views are automatically refreshed when base tables are changed.

Answer

Materialized views cannot be clustered.

Answer

Materialized views must be manually refreshed by the user.

Answer

Materialized views can only be created on ingestion-time partitioned tables.

Question 8

You need to choose a messaging service for a real-time streaming application that requires low cost and can tolerate occasional message loss. Which service is MOST suitable?

Accepted Answer

Pub/Sub Lite. Pub/Sub Lite offers a lower-cost option compared to Pub/Sub, with reduced reliability (e.g., at-least-once delivery but not exactly-once). It is designed for cost-sensitive streaming workloads where occasional loss is acceptable.

Answer

Cloud Scheduler

Answer

Pub/Sub

Answer

Cloud Tasks

Question 9

You are building a data pipeline that ingests data from on-premises into Cloud Storage, then processes it with Dataproc, and finally loads into BigQuery. You need to schedule the pipeline to run daily. The pipeline must handle occasional failures gracefully. Which THREE Google Cloud services should you use together to achieve this? (Choose 3)

Accepted Answer

Cloud Storage. Cloud Composer orchestrates the whole pipeline. Cloud Storage is the staging area. Dataproc processes data. BigQuery is the destination. Dataflow is not needed. Pub/Sub is for messaging, not scheduling.

Answer

Dataflow

Answer

Pub/Sub

Question 10

A company uses Dataproc Serverless for Spark batch jobs. They notice that some jobs are failing due to out-of-memory (OOM) errors. Which configuration parameter should they adjust to allocate more memory per executor?

Accepted Answer

Set spark.executor.memory to a higher value, e.g., 8g. In Dataproc Serverless, Spark properties can be set via --properties. The spark.executor.memory property controls the memory per executor. Increasing it can resolve OOM errors.

Answer

Use a custom image with more memory

Answer

Set spark.driver.memory to a higher value

Answer

Increase the number of workers by setting --num-workers

Question 11

A data engineer needs to run an existing Spark job on Google Cloud with minimal code changes. The job requires Hive metastore access. Which Dataproc feature should they use to provide a managed Hive metastore?

Accepted Answer

Dataproc Metastore. Dataproc Metastore provides a fully managed Hive metastore that integrates with Dataproc clusters, allowing existing Spark jobs to use it without code changes.

Answer

Cloud SQL for MySQL

Answer

BigQuery as a Hive metastore

Answer

Dataproc on GKE

Question 12

An organization is using BigQuery for analytics. They have a table that is 500 GB and is frequently queried by 'date' and 'region'. They want to optimize query performance and reduce costs. Which TWO actions should they take?

Accepted Answer

Cluster the table by region. Partitioning by date enables partition pruning to scan only relevant partitions. Clustering by region further reduces data scanned within partitions.

Answer

Use an authorized view

Answer

Use a wildcard table

Answer

Use materialized views

Question 13

A company runs Apache Spark jobs on Dataproc. They want to reduce costs by using preemptible instances for worker nodes. The jobs are fault-tolerant and can handle occasional node loss. However, the cluster must remain available for interactive querying during business hours. Which Dataproc cluster configuration meets these requirements?

Accepted Answer

Use a standard cluster with preemptible instances as secondary workers. Dataproc clusters support multiple node types. Primary workers run the NodeManager and DataNode daemons; using preemptible instances for them is risky because they are critical for HDFS and YARN. However, secondary workers (preemptible workers) are designed for stateless processing and can be lost without affecting cluster availability. Preemptible instances cannot be used for master nodes.

Answer

Use a single-node cluster that automatically scales with preemptible instances

Answer

Use standard cluster with master and worker nodes as preemptible instances

Answer

Use a high-availability cluster with preemptible instances for primary workers

Question 14

A company uses Pub/Sub to ingest events from multiple sources. They need to ensure that messages from a specific source are processed in order (per source partition). They also need to deduplicate messages. Which TWO features should they use?

Accepted Answer

Enable exactly-once delivery on the subscription. Pub/Sub ordering keys ensure messages with the same ordering key are delivered in order. Message deduplication is achieved via exactly-once delivery (Pub/Sub now supports this). Dead letter topics are for undeliverable messages. Schemas enforce structure, not ordering.

Answer

Set a message schema to enforce ordering

Answer

Use a dead letter topic to handle out-of-order messages

Answer

Use a pull subscription with a large ack deadline

Question 15

A Dataflow pipeline using Apache Beam processes unbounded data from Pub/Sub. The pipeline uses fixed windows of 1 minute and a trigger that fires early every 30 seconds and at watermark. The team observes that the output pane for window [10:00:00, 10:01:00) contains events with timestamps from 10:00:15 and 10:00:45, but also an event with timestamp 10:02:00. What is the most likely cause?

Accepted Answer

Allowed lateness is set to more than 1 minute, so late data is still included in its original window. Late data can arrive after the watermark has passed, and with allowed lateness, it can be included in the original window. The event with timestamp 10:02:00 is late data that arrived after the watermark for window [10:00:00, 10:01:00), but within the allowed lateness period. It is not a trigger issue because the trigger fires correctly; the event is simply late.

Answer

The trigger is firing too early, causing the window to close prematurely

Answer

The watermark is incorrectly estimated, allowing late data to be included

Answer

The window duration is actually 2 minutes due to a misconfiguration