PDE Practice Test 2 — 30 Questions

Question 1

Drag and drop the steps to configure a VPC network with private Google access for on-premises connectivity using Cloud VPN into the correct order.

Question 2

Match each data pipeline term to its definition.

Question 3

Match each data lifecycle stage to its description.

Question 4

A company runs a batch ETL pipeline on Cloud Dataproc. During peak hours, the job takes longer than expected. The pipeline reads from Cloud Storage, transforms data, and writes to BigQuery. What is the most cost-effective way to improve performance without redesigning the pipeline?

Accepted Answer

Add a secondary worker group using preemptible VMs and increase the number of workers.. Adding a secondary worker group with preemptible VMs is the most cost-effective way to improve performance because it allows you to scale out the cluster horizontally with compute instances that are significantly cheaper (up to 80% discount) than regular VMs. This directly addresses the bottleneck of processing capacity during peak hours without requiring any pipeline redesign, as Cloud Dataproc can automatically distribute work across additional workers.

Answer

Enable local SSDs on all worker nodes.

Answer

Increase the master node's machine type to n1-highmem-32.

Answer

Use Cloud Composer to schedule the job with a higher priority.

Question 5

Match each data storage term to its characteristic.

Question 6

Match each data encryption concept to its description.

Question 7

A financial services firm processes sensitive transactions using Cloud Dataflow. The pipeline reads from Pub/Sub, performs stateful processing (e.g., fraud detection), and writes to Cloud Spanner. Compliance requires exactly-once processing semantics. Which configuration ensures exactly-once processing?

Accepted Answer

Set Dataflow pipeline to exactly-once mode and design Spanner writes to be idempotent.. Option C is correct because exactly-once processing in a Dataflow pipeline requires the pipeline itself to be set to exactly-once mode (which uses consistent snapshots and transactional sinks) and the output writes to Spanner to be idempotent. This combination ensures that even if a record is reprocessed due to failures, the final state in Spanner remains consistent, satisfying compliance requirements.

Answer

Configure Pub/Sub to use exactly-once delivery mode.

Answer

Use Pub/Sub with at-least-once delivery and Dataflow with at-least-once processing mode.

Answer

Enable Dataflow's streaming engine and use Spanner's built-in retry logic.

Question 8

A healthcare company streams patient monitoring data to Cloud Pub/Sub. A Dataflow pipeline reads the stream, enriches with patient records from BigQuery, and writes to Bigtable for real-time queries. The BigQuery lookup is slow and causes pipeline lag. What is the best approach to improve performance?

Accepted Answer

Use a side input from a BigQuery query with a global window and periodic refresh.. Option D is correct because using a side input from BigQuery with a global window and periodic refresh allows the Dataflow pipeline to cache the patient records in memory across all workers, avoiding per-element slow lookups. This pattern leverages Beam's side input semantics to broadcast a relatively static lookup table, significantly reducing latency compared to synchronous BigQuery queries for each incoming event.

Answer

Increase the number of Dataflow workers and use vertical scaling.

Answer

Use BigQuery's streaming read API in the pipeline.

Answer

Pre-join the data in a batch pipeline and load into Bigtable.

Question 9

A media company ingests video files from partners via a REST API. Files are stored in Cloud Storage, and metadata is written to Firestore. A Cloud Function is triggered on object finalize to transcode video using Transcoder API. Sometimes, the function fails because the file is still being uploaded when triggered. How should this be fixed?

Accepted Answer

Use Cloud Pub/Sub notifications for Cloud Storage and trigger the function from the subscription.. Option D is correct because Cloud Storage object finalize notifications are sent only after the entire file has been written and committed. By using Pub/Sub notifications for Cloud Storage and triggering the Cloud Function from the subscription, you decouple the trigger from the upload process, ensuring the function only runs when the file is fully available. This eliminates the race condition where the function is triggered before the upload completes.

Answer

Implement a Cloud Composer workflow to poll for file existence.

Answer

Require partners to use resumable uploads.

Answer

Increase the Cloud Functions timeout to allow time for the upload to finish.

Question 10

A company uses Cloud Composer to orchestrate data pipelines. One DAG fails intermittently with the error: 'Task received SIGTERM signal.' The task runs a long-running Dataproc job. What is the most likely cause?

Accepted Answer

The Airflow task timed out due to the default execution timeout.. The default Airflow task execution timeout is 28 days in Cloud Composer, but individual tasks can have a shorter `execution_timeout` set in the DAG definition. When a long-running Dataproc job exceeds this timeout, Airflow sends a SIGTERM signal to the task to kill it, resulting in the observed error. This is the most likely cause because the error message directly indicates a forced termination by the Airflow scheduler, not an infrastructure or code failure.

Answer

The Dataproc cluster was preempted by Google Cloud.

Answer

The Dataproc job failed due to an error in the code.

Answer

The Cloud Composer environment ran out of disk space.

Question 11

A team uses Vertex AI AutoML Tables to train a model. They need to deploy the model for real-time predictions with high availability. Which deployment configuration should they use?

Accepted Answer

Deploy to a Vertex AI Endpoint with multiple replicas and auto-scaling. For real-time predictions with high availability, you need a deployment that can handle traffic spikes and failover. Deploying to a Vertex AI Endpoint with multiple replicas and auto-scaling ensures that the model is served from multiple instances, providing redundancy and the ability to scale up or down based on demand. This configuration meets the high-availability requirement by distributing load and automatically recovering from instance failures.

Answer

Export as a Cloud Function

Answer

Deploy to a Vertex AI Endpoint with 1 replica

Answer

Use a Vertex AI Batch Prediction job

Question 12

Drag and drop the steps to create a Cloud Function triggered by Cloud Storage events into the correct order.

Question 13

A company runs a batch processing job on Dataproc that uses Apache Spark to process 500 GB of data daily. The job completes successfully but takes 4 hours. The team wants to reduce the runtime to under 2 hours without increasing cost. What should they do?

Accepted Answer

Use preemptible VMs for worker nodes and increase the number of workers.. Preemptible VMs cost significantly less than standard VMs (about 60-80% discount). By using preemptible VMs for worker nodes, you can increase the number of workers (and thus parallelism) without increasing cost. This directly reduces runtime by distributing the 500 GB workload across more executors, while the cost savings from preemptible VMs offset the additional nodes.

Answer

Increase the master node's machine type to n2-standard-8.

Answer

Increase the machine type of worker nodes to n2-highmem-8.

Answer

Migrate the job to Dataflow with autoscaling enabled.

Question 14

A Dataflow pipeline reads log files from Cloud Storage, parses them into LogEvent objects, and writes to BigQuery. The pipeline fails with the above errors. What is the most likely cause?

Accepted Answer

The LogEvent class does not have a no-argument constructor.. Apache Beam's SDK requires that custom types used as PCollection elements (like LogEvent) have a no-argument constructor so that the framework can deserialize objects during distributed processing, especially when using the Dataflow runner. Without it, the pipeline fails at runtime with a serialization error because Beam's default coder (e.g., SerializableCoder) cannot reconstruct the object.

Answer

The pipeline is missing required import statements for LogEvent.

Answer

The BigQuery table schema does not match the LogEvent fields.

Answer

The log files are not in the expected format, causing parsing failures.

Question 15

Which TWO statements are correct about designing a data pipeline using Cloud Dataflow for processing unbounded data?

Accepted Answer

Watermarks are used to measure the progress of event time.. Watermarks in Cloud Dataflow measure the progress of event time, indicating when all data up to a certain timestamp is expected to have arrived. This allows the pipeline to handle late-arriving data and determine when to close windows for unbounded data streams.

Answer

Triggers can only emit results at the end of a window.

Answer

Dataflow guarantees exactly-once processing for streaming pipelines.

Answer

Fixed windows are always based on processing time.

Question 16

A retail company processes real-time clickstream data using Cloud Pub/Sub and Dataflow. The pipeline aggregates events by user session and writes to Bigtable for low-latency queries. However, users report that session data is sometimes missing or duplicated. What is the most likely cause?

Accepted Answer

Pub/Sub provides at-least-once delivery, and Dataflow does not deduplicate by default.. D is correct because Pub/Sub offers at-least-once delivery, meaning the same message may be delivered multiple times. Dataflow does not automatically deduplicate messages unless explicitly configured (e.g., using idempotent sinks or custom deduplication logic). Without deduplication, the same session event can be processed more than once, leading to duplicate session data in Bigtable.

Answer

Session windowing is configured with too short a gap duration.

Answer

Bigtable schema design causes row key collisions.

Answer

Dataflow's default behavior discards late-arriving data.

Question 17

Which THREE considerations are important when designing a data lake on Google Cloud using Cloud Storage?

Accepted Answer

Choose the appropriate storage class based on access patterns.. Option C is correct because selecting the appropriate storage class (e.g., Standard, Nearline, Coldline, Archive) based on data access patterns directly optimizes cost and performance in Cloud Storage. For a data lake, where data may be accessed frequently initially and rarely later, matching the storage class to the access pattern avoids paying premium rates for infrequently accessed data.

Answer

Use Cloud Storage's eventual consistency model for cost savings.

Answer

Define a schema when writing data to enforce data quality.

Question 18

A company uses Cloud Dataproc to run Spark ML jobs. The jobs are memory-intensive and often fail with OutOfMemory errors. Which action would most effectively reduce memory pressure without changing the Spark code?

Accepted Answer

Increase the number of worker nodes and reduce the number of cores per worker.. Increasing the number of worker nodes while reducing the number of cores per worker reduces memory pressure by distributing the workload across more JVMs, each with a smaller heap. This lowers the per-executor memory requirement and reduces the risk of OutOfMemory errors without modifying Spark code. In Cloud Dataproc, this approach directly addresses memory contention by giving each executor fewer tasks to process concurrently.

Answer

Increase the master node's memory.

Answer

Increase the number of Spark partitions.

Answer

Use preemptible VMs for workers.

Question 19

Which THREE best practices should be followed when designing a Dataflow pipeline for real-time data processing?

Accepted Answer

Set up monitoring alerts for system lag and data freshness.. Option A is correct because monitoring alerts for system lag and data freshness are essential for maintaining operational visibility in real-time Dataflow pipelines. System lag (the time between data ingestion and processing) and data freshness (how current the processed output is) directly impact the pipeline's ability to meet latency SLAs. Without these alerts, issues like worker backpressure or Pub/Sub subscription backlog can go unnoticed, leading to stale or lost data.

Answer

Use static side inputs that are loaded once at pipeline start.

Answer

Use global windows with early triggers for low latency.

Question 20

A data engineer needs to process a large dataset (500 TB) stored in Cloud Storage using Dataproc. The processing job requires reading the entire dataset and writing results back to Cloud Storage. The job is expected to run for 6 hours. Which configuration minimizes cost?

Accepted Answer

Use a cluster with a mix of standard and preemptible VMs.. Option C is correct because preemptible VMs cost about 80% less than standard VMs, and mixing them with standard VMs provides fault tolerance for the job's 6-hour duration. Since the job reads and writes to Cloud Storage (not local HDFS), local SSDs are unnecessary, and a single-node cluster would lack the parallelism needed to process 500 TB efficiently within 6 hours. Using a mix of standard (for critical master/worker nodes) and preemptible VMs (for worker nodes) minimizes cost while ensuring job completion.

Answer

Use a single-node cluster with standard VMs.

Answer

Use a cluster with local SSDs for faster I/O.

Answer

Use a cluster with n1-highmem-32 instances and 1000 cores.