PDE Practice Test 37 — 15 Questions

Question 1

A data engineer is building a Dataflow pipeline that reads from BigQuery, transforms data using Apache Beam, and writes results to Cloud Storage in Avro format. They need to ensure the pipeline can be easily redeployed with different parameters without modifying code. Which deployment method should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Dataflow Flex Templates. Dataflow Flex Templates allow you to package a pipeline as a Docker image and pass runtime parameters, enabling parameterized deployments without code changes.

Answer

Direct deployment using the gcloud command with parameters

Answer

Dataflow Classic Templates

Answer

Deploy as a Cloud Function triggered by Cloud Scheduler

Question 2

A data engineer is designing a batch data pipeline that reads Avro files from Cloud Storage, transforms data using Apache Beam, and writes to BigQuery. The pipeline must handle daily runs and backfills. Which runner should they use?

Accepted Answer

DataflowRunner. DataflowRunner is the correct choice because it is the fully managed service runner for Apache Beam on Google Cloud, optimized for batch and streaming pipelines. It automatically handles scaling, resource management, and exactly-once processing semantics, which are essential for reliable daily runs and backfills with Avro files from Cloud Storage and BigQuery sinks.

Answer

FlinkRunner

Answer

SparkRunner

Answer

DirectRunner

Question 3

Which TWO are best practices for managing a Cloud Dataflow pipeline in production?

Accepted Answer

Set up Cloud Monitoring alerts based on Dataflow job metrics. Cloud Monitoring alerts on Dataflow job metrics (e.g., system lag, watermark delay, or element count) enable proactive detection of pipeline health issues such as backpressure or stuck workers. This is a best practice for production pipelines to ensure reliability and timely intervention.

Answer

Always use batch mode for streaming data to reduce cost

Answer

Disable autoscaling to keep compute costs predictable

Answer

Restart the pipeline when code changes are needed

Question 4

A data engineer uses Cloud Composer to orchestrate a daily batch pipeline. A downstream task should only start after an upstream BigQuery load job finishes successfully and a specific file appears in Cloud Storage. Which combination of operators should the engineer use in the Airflow DAG?

Accepted Answer

BigQueryInsertJobOperator and GCSObjectExistenceSensor with upstream dependency. The BigQueryInsertJobOperator (or BigQueryOperator) handles the load job, and the GoogleCloudStorageObjectExistenceSensor (or GCSObjectExistenceSensor) waits for the file. Task dependencies link them.

Answer

BigQueryInsertJobOperator with wait_for_downstream=True

Answer

DataflowPythonOperator and GCSObjectExistenceSensor

Answer

BigQueryOperator and FileSensor with downstream dependency

Question 5

A financial services company uses Cloud Pub/Sub with ordering keys to process transactions in order. Some messages are failing processing and getting stuck. The team wants to ensure that if a message fails, it can be reprocessed later without blocking subsequent messages. What should they implement?

Accepted Answer

Configure a dead letter topic and handle the failed message separately. A dead letter topic (DLT) allows failed messages to be moved aside after exhausting retry attempts, so they do not block the processing of subsequent ordered messages. In Cloud Pub/Sub, ordering keys require messages with the same key to be delivered in order; if a message fails and is not acknowledged, it blocks all later messages with the same key. By configuring a dead letter topic, the failed message is automatically forwarded to the DLT after a maximum of 5 delivery attempts (default), and the original subscription can continue processing the next messages in order. The team can then reprocess the failed message from the DLT separately, without affecting the order of other messages.

Answer

Create multiple subscriptions for the same topic

Answer

Use a pull subscription with flow control settings

Answer

Increase the acknowledgment deadline to 600 seconds

Question 6

An organization uses Cloud Dataproc to run Spark jobs that process sensitive data. They need to ensure data is encrypted at rest and that only specific service accounts can access the data on cluster disks. What should they do?

Accepted Answer

Use customer-managed encryption keys (CMEK) for the cluster's persistent disks and assign a dedicated service account to the cluster with minimal IAM roles.. Using customer-managed encryption keys (CMEK) allows the organization to control and manage the encryption keys for persistent disks attached to the Dataproc cluster, ensuring data at rest is encrypted. Assigning a dedicated service account with minimal IAM roles ensures that only that service account can access the data on the cluster disks, following the principle of least privilege.

Answer

Rely on the default encryption at rest and use VPC Service Controls to limit data exfiltration.

Answer

Use customer-supplied encryption keys (CSEK) and write a startup script to mount encrypted disks.

Answer

Enable encryption at rest using Google-managed encryption keys and grant all users the Dataproc Editor role.

Question 7

A global fintech company needs a database that can serve transactional (OLTP) and analytical (OLAP) workloads with strong consistency. They require high availability and PostgreSQL compatibility. Which TWO Google Cloud databases meet these requirements? (Choose 2 correct options)

Accepted Answer

Cloud Spanner. Cloud Spanner is correct because it provides a globally distributed, strongly consistent relational database service that supports both OLTP and OLAP workloads via PostgreSQL-compatible interfaces (including the PostgreSQL dialect). It offers high availability through synchronous replication across zones and regions, and its TrueTime-based atomic clocks ensure external consistency for transactions. AlloyDB is also correct because it is a fully PostgreSQL-compatible database service that delivers high availability and strong consistency. It is designed to handle both transactional and analytical workloads using a combined row-store and columnar engine, enabling high-performance OLTP and real-time analytics within a single database. Both services meet the fintech company's requirements for PostgreSQL compatibility, strong consistency, and high availability.

Answer

Cloud SQL

Answer

Cloud Bigtable

Answer

BigQuery

Question 8

A retail company processes real-time clickstream data using Cloud Pub/Sub and Dataflow. The pipeline aggregates events by user session and writes to Bigtable for low-latency queries. However, users report that session data is sometimes missing or duplicated. What is the most likely cause?

Accepted Answer

Pub/Sub provides at-least-once delivery, and Dataflow does not deduplicate by default.. D is correct because Pub/Sub offers at-least-once delivery, meaning the same message may be delivered multiple times. Dataflow does not automatically deduplicate messages unless explicitly configured (e.g., using idempotent sinks or custom deduplication logic). Without deduplication, the same session event can be processed more than once, leading to duplicate session data in Bigtable.

Answer

Session windowing is configured with too short a gap duration.

Answer

Bigtable schema design causes row key collisions.

Answer

Dataflow's default behavior discards late-arriving data.

Question 9

Refer to the exhibit. An auditor sees the following output from `gcloud ai models list`. What can they conclude about versioning?

Accepted Answer

The model has two versions with v2 being the latest. The `gcloud ai models list` output shows two model versions (v1 and v2) under the same model resource. The default traffic split or the listed order indicates v2 is the latest version. This directly confirms that the model has two versions, with v2 being the latest, making option B correct.

Answer

The model is deployed on a single endpoint

Answer

Only the latest version is available

Answer

The model is automatically scaled

Question 10

A Dataflow streaming job is processing data from Pub/Sub and writing to BigQuery. The job is stuck with the message 'No progress has been made' for several minutes. Which TWO actions should the team take to troubleshoot and resolve the issue? (Choose TWO.)

Accepted Answer

Examine the worker logs in Cloud Logging for any error messages or exceptions.. Examining worker logs in Cloud Logging helps identify the root cause of the stuck pipeline, such as OOM errors, serialization failures, or worker crashes. Option E is also correct because enabling Dataflow Streaming Engine moves state to the backend, reducing worker load and overcoming stuck progress issues caused by resource constraints. The combination of inspecting logs and offloading state allows effective troubleshooting and resolution.

Answer

Set the updateCompatibility flag to true and restart the pipeline.

Answer

Increase the persistent disk size for all workers to reduce I/O contention.

Answer

Force stop the pipeline and update it with a new version using the --update flag.

Question 11

A company processes IoT sensor data in near real-time. They ingest data via Cloud Pub/Sub, then a Dataflow streaming pipeline writes to Bigtable for low-latency queries. Recently, they observed increased Pub/Sub message backlog during traffic spikes. What is the most effective scaling strategy?

Accepted Answer

Increase Dataflow worker count and adjust autoscaling configuration. The increased Pub/Sub backlog during traffic spikes indicates that the Dataflow pipeline is unable to consume messages as fast as they are being published. Increasing the Dataflow worker count and adjusting autoscaling configuration allows the pipeline to scale horizontally, processing more messages per second and reducing the backlog. Pub/Sub itself is designed to handle high throughput, so the bottleneck is the consumer (Dataflow), not the ingestion layer.

Answer

Increase Pub/Sub subscription throughput by increasing the number of partitions

Answer

Use a Cloud Scheduler to throttle Pub/Sub publishing

Answer

Add a Cloud Function to pre-process messages before they are consumed by Dataflow

Question 12

You are designing a Dataflow pipeline that joins two unbounded PCollections from different sources. Which transform should you use?

Accepted Answer

CoGroupByKey. CoGroupByKey performs a key-based join of multiple PCollections. It can handle unbounded streams with appropriate windowing.

Answer

ParDo

Answer

Flatten

Answer

GroupByKey

Question 13

A company needs to stream real-time user click events from a web application to BigQuery for analysis. Which Google Cloud architecture is most suitable?

Accepted Answer

App Engine -> Pub/Sub -> Dataflow -> BigQuery. It provides a fully managed, scalable, and decoupled architecture for ingesting real-time click events. Pub/Sub acts as a durable, asynchronous message buffer that can handle high-throughput streams, Dataflow (Apache Beam) processes the events in near real-time with exactly-once semantics, and BigQuery serves as the analytics warehouse. This pattern is the recommended Google Cloud approach for streaming analytics, as it decouples producers from consumers and supports auto-scaling.

Answer

Cloud Scheduler -> BigQuery

Answer

Compute Engine -> Cloud Storage -> BigQuery

Answer

Cloud Functions -> BigQuery

Question 14

A data engineer wants to store archived log files in Cloud Storage with a retention policy that prevents deletion for 5 years. Which feature should they use?

Accepted Answer

Retention Policy on the bucket set to 5 years. A retention policy on a Cloud Storage bucket enforces a minimum retention period for all objects in the bucket, preventing deletion or overwrite until the policy duration has elapsed. Setting it to 5 years ensures that archived log files cannot be deleted before that time, meeting the data engineer's requirement exactly. This is a bucket-level, immutable setting that applies to all objects, unlike object-level holds or lifecycle rules.

Answer

Object Lifecycle rule with Delete action after 5 years

Answer

Object Hold (temporal)

Answer

Versioning enabled

Question 15

A company is designing a data pipeline that ingests real-time events from IoT devices and must handle late-arriving data (up to 1 hour late) while minimizing duplicate processing. They plan to use Dataflow with Pub/Sub. Which combination of windowing and trigger settings should they use?

Accepted Answer

Session windows with a 10-minute gap duration and allowed lateness of 1 hour. Session windows naturally group events based on a gap duration, so late events within the gap extend the window. Setting the allowed lateness to 1 hour ensures that late events are still included in the correct session. Using withAllowedLateness(1 hour) allows the watermark to advance and the session to finalize after the gap, but late data within 1 hour will trigger a pane update.

Answer

Sliding windows of 10 minutes with allowed lateness of 30 minutes and accumulating panes

Answer

Global window with allowed lateness of 1 hour and accumulating trigger every 5 minutes

Answer

Fixed windows of 1 hour with allowed lateness of 1 hour and no accumulation