PDE Practice Exam 3 — 20 Questions

Question 1

A company wants to implement a data lake on Google Cloud to store raw sensor data (unstructured binary files) and allow data scientists to run SQL queries on processed data. They expect to store terabytes of data and have different access patterns. Which combination of GCP services best meets these requirements?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Cloud Storage for raw data and BigQuery for processed data. Cloud Storage is the ideal service for storing raw, unstructured binary sensor data at petabyte scale, offering low-cost, durable object storage with multiple access tiers. BigQuery is a serverless, highly scalable data warehouse that allows data scientists to run SQL queries on processed data, with features like columnar storage and automatic optimization for analytical workloads. This combination directly addresses the need for raw storage and SQL-based analytics on processed data.

Answer

Bigtable for raw data and Cloud Spanner for processed data

Answer

Cloud Storage for both raw and processed data

Answer

Cloud SQL for raw data and Cloud Dataproc for processing

Question 2

You are monitoring a Dataproc cluster and notice that the cluster utilisation is high, but jobs are running slowly. The cluster uses preemptible workers for cost savings. What is the most likely cause of the performance degradation?

Accepted Answer

The preemptible workers are being preempted frequently, causing task retries and slowdowns.. Preemptible (spot) workers in Dataproc can be reclaimed by Compute Engine at any time, and when they are preempted mid-task, the task must be retried on remaining workers. Frequent preemption causes repeated task retries, stragglers, and overall job slowdown even though the cluster appears highly utilized. This is the classic symptom of relying heavily on preemptible workers.

Answer

The primary workers are using standard disks instead of SSDs.

Answer

The cluster is under-provisioned; increase the number of preemptible workers.

Answer

The cluster is using an older image version; upgrade to the latest.

Question 3

Drag and drop the steps to create a Cloud Bigtable instance and table using the CLI into the correct order.

Accepted Answer

Create Cloud Bigtable instance → Create table inside the instance → Add column families to the table. Bigtable instances contain clusters; tables are created within instances and must have column families.

Answer

Create table → Create Cloud Bigtable instance → Add column families to the table

Answer

Create Cloud Bigtable instance → Add column families → Create table inside the instance

Answer

Add column families → Create Cloud Bigtable instance → Create table inside the instance

Question 4

A company uses Looker Studio to create dashboards from BigQuery data. They notice that dashboard queries take several seconds to load. They want to improve performance without changing the underlying data or creating materialized views. Which option should they use?

Accepted Answer

Enable BigQuery BI Engine for the project. BigQuery BI Engine is an in-memory analysis service that accelerates queries from Looker Studio (and other BI tools) by caching data in memory, significantly reducing latency. Replicating data to Cloud SQL would add complexity and may not handle the volume. Using Looker instead of Looker Studio doesn't inherently speed up queries. Increasing BigQuery slots would help but is more expensive and not as targeted for BI tools.

Answer

Increase the number of BigQuery slots

Answer

Switch to Looker instead of Looker Studio

Answer

Replicate the data to Cloud SQL for faster queries

Question 5

A Dataflow pipeline reads log files from Cloud Storage, parses them into LogEvent objects, and writes to BigQuery. The pipeline fails with the above errors. What is the most likely cause?

Accepted Answer

The LogEvent class does not have a no-argument constructor.. Apache Beam's SDK requires that custom types used as PCollection elements (like LogEvent) have a no-argument constructor so that the framework can deserialize objects during distributed processing, especially when using the Dataflow runner. Without it, the pipeline fails at runtime with a serialization error because Beam's default coder (e.g., SerializableCoder) cannot reconstruct the object.

Answer

The pipeline is missing required import statements for LogEvent.

Answer

The BigQuery table schema does not match the LogEvent fields.

Answer

The log files are not in the expected format, causing parsing failures.

Question 6

A company uses Cloud Functions to process events from Cloud Storage. They notice that occasionally functions are not triggered. What should they check first to ensure solution quality?

Accepted Answer

Verify that the Cloud Storage bucket has notifications configured for the correct event type.. The most common reason for Cloud Functions not being triggered by Cloud Storage events is that the bucket's notification configuration is missing or misconfigured. Cloud Functions relies on Pub/Sub notifications from the bucket to invoke the function; if the notification is not set for the correct event type (e.g., `OBJECT_FINALIZE`), the function will never be triggered. Therefore, verifying the notification configuration is the first and most direct diagnostic step.

Answer

Check the logs for function execution.

Answer

Increase the function memory allocation.

Answer

Increase the function timeout.

Question 7

A data engineer needs to query data from BigQuery and another cloud provider's storage (AWS S3) using a single SQL query. The data must not be moved or copied to GCP. Which Google Cloud service should they use?

Accepted Answer

BigQuery Omni. BigQuery Omni allows querying data across multiple clouds (AWS S3, Azure Blob Storage) using BigQuery's interface without moving data. BigQuery Omni runs compute in the other cloud's region. BigQuery Transfer Service moves data into BigQuery. Dataplex is for data management, not cross-cloud queries. Cloud Storage Transfer Service is for moving data between clouds.

Answer

Cloud Storage Transfer Service

Answer

Dataplex

Answer

BigQuery Data Transfer Service

Question 8

A company uses Dataplex to manage data lakes on Google Cloud. They want to enforce data quality rules on a BigQuery table, such as ensuring that a 'email' column is not null and matches a regex pattern. Which Dataplex feature should they use?

Accepted Answer

Dataplex Data Quality. Dataplex Data Quality is the feature specifically designed to define and enforce data quality rules on BigQuery tables, including checks for null values and regex pattern matching. It allows you to create data quality rules that can be scheduled or run on-demand, and it provides results that can be used for monitoring and alerting. The other options are Dataplex components for metadata management, lake organization, and lineage tracking, not for enforcing data quality rules.

Answer

Dataplex Universal Catalog

Answer

Dataplex Lake

Answer

Dataplex Data Lineage

Question 9

A company uses Cloud Dataproc to run nightly Spark ETL jobs that process about 500 GB of data each night. The jobs currently take 4 hours to complete. The company wants to reduce the runtime to under 2 hours to meet a new SLA. The cluster is configured with 10 worker nodes (n1-standard-4) and 1 master node (n1-standard-4). The jobs are CPU-bound and use only default settings. The cluster is deleted after each job and recreated. The data is stored in Cloud Storage. The company is open to increasing cost but wants the most cost-effective solution to meet the SLA. Which approach should they take?

Accepted Answer

Increase the number of worker nodes to 20 and use preemptible VMs for half of them.. Adding more worker nodes (from 10 to 20) directly increases parallelism for CPU-bound Spark jobs, and using preemptible VMs for half of them reduces cost while still meeting the SLA. Since the job is CPU-bound and uses default settings, scaling horizontally with a mix of standard and preemptible VMs is the most cost-effective way to halve runtime, as Spark can efficiently distribute the workload across more cores.

Answer

Use a regional Cloud Storage bucket to improve read throughput.

Answer

Replace worker nodes with n1-highmem-16 instances to increase memory.

Answer

Change machine type to n2-standard-8 for all nodes.

Question 10

You have a BigQuery table 'events' with a TIMESTAMP column 'event_time'. You need to compute, for each event, the difference in seconds from the previous event of the same user. Which window function should you use?

Accepted Answer

LAG(event_time) OVER (PARTITION BY user_id ORDER BY event_time). LAG(event_time) OVER (PARTITION BY user_id ORDER BY event_time) returns the event_time value from the previous row within the same user_id partition, ordered chronologically. Subtracting that returned timestamp from the current row's event_time (e.g., TIMESTAMP_DIFF(event_time, LAG(...), SECOND)) yields the seconds elapsed since the user's prior event. This is the canonical pattern for gap/delta calculations in BigQuery analytic functions.

Answer

FIRST_VALUE(event_time) OVER (PARTITION BY user_id ORDER BY event_time)

Answer

LEAD(event_time) OVER (PARTITION BY user_id ORDER BY event_time)

Answer

ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY event_time)

Question 11

An e-commerce company runs a daily batch pipeline that processes clickstream data from Cloud Storage using Cloud Dataproc with Spark. The pipeline includes a join between a large fact table and a small dimension table. The dimension table is stored in Cloud Storage as a CSV file. The join is slow due to shuffling. The data engineer considers broadcasting the dimension table. However, the dimension table is updated daily and the pipeline reads the latest version. What is the best approach to implement this optimization?

Accepted Answer

Use DataFrame.join with broadcast hint on the dimension DataFrame. Broadcasting the small dimension table using the broadcast hint (e.g., `broadcast(dimensionDF)`) forces Spark to replicate the dimension data to all executor nodes, eliminating the need for a shuffle during the join. This is ideal when the dimension table is small enough to fit in executor memory, and since the pipeline reads the latest CSV daily, the broadcast will automatically use the updated data without additional code changes.

Answer

Read the fact table and dimension table into separate DataFrames and use standard join

Answer

Read the dimension table as an RDD and collect as a map, then use map-side join

Answer

Increase the spark.sql.autoBroadcastJoinThreshold to a large value

Question 12

A company runs a real-time anomaly detection system on Google Cloud. Streaming data from IoT devices is ingested via Pub/Sub, processed by Dataflow (Apache Beam), and results are written to Bigtable for low-latency serving. Recently, the system has been experiencing increased latency and occasional data loss. The Dataflow pipeline shows high system lag and backlog in Pub/Sub. The Bigtable cluster has 3 nodes and is reporting high CPU utilization (over 90%). The team suspects the issue is with the pipeline configuration. They have already verified that there are no errors in the pipeline code and no network issues. Which action should they take to resolve the issue?

Accepted Answer

Increase the number of Bigtable nodes to handle the write throughput.. The high CPU utilization on Bigtable (over 90%) indicates that the cluster is saturated and cannot keep up with the write throughput from Dataflow. This causes backpressure in the pipeline, leading to increased system lag and backlog in Pub/Sub, and eventually data loss when Pub/Sub messages expire. Increasing the number of Bigtable nodes directly addresses the bottleneck by distributing the write load and reducing CPU pressure, which allows the pipeline to drain the backlog and reduce latency.

Answer

Change the Dataflow worker machine type to n2-standard-8.

Answer

Decrease the batch size in the Dataflow pipeline to reduce latency.

Answer

Increase the number of Dataflow workers to process messages faster.

Question 13

A data engineer needs to ingest on-premises Oracle CDC data into BigQuery in near real-time with minimal operational overhead. Which service should they use?

Accepted Answer

Datastream. Datastream is purpose-built for streaming change data capture (CDC) from Oracle and other sources into BigQuery with near-real-time latency and minimal operational overhead. It handles schema propagation, checkpointing, and automatic retries, eliminating the need to manage custom ingestion pipelines.

Answer

Pub/Sub + Dataflow

Answer

Storage Transfer Service

Answer

Transfer Appliance

Question 14

An organization needs to restrict access to BigQuery and Cloud Storage so that data can only be accessed from within a specific VPC network and cannot be exfiltrated. Which Google Cloud feature should they use?

Accepted Answer

VPC Service Controls. VPC Service Controls (option B) is the correct choice because it creates a security perimeter around Google Cloud services like BigQuery and Cloud Storage, preventing data exfiltration even from within a VPC. It enforces context-aware access based on the VPC network, ensuring data can only be accessed from authorized VPC sources and blocking unauthorized transfers outside the perimeter.

Answer

Private Service Access

Answer

VPC firewall rules

Answer

IAM conditions

Question 15

A company wants to version its ML models and track lineage from training data to deployed model. Which Google Cloud service should they use?

Accepted Answer

Vertex AI ML Metadata. Vertex AI ML Metadata is the Google Cloud service designed to track ML artifacts, executions, and contexts, providing model versioning and lineage from training data through to deployed models. It is the native metadata store for Vertex AI pipelines and experiments.

Answer

Cloud Storage with object versioning

Answer

Data Catalog

Answer

Artifact Registry

Question 16

A data engineer needs to schedule recurring nightly loads from Amazon S3 to Google Cloud Storage. The data is in CSV format and the volume is approximately 500 GB per night. Which Google Cloud service should they use?

Accepted Answer

Storage Transfer Service. The Storage Transfer Service is designed for online data transfers from external cloud providers like Amazon S3 to Google Cloud Storage. It supports scheduling recurring nightly transfers, handles large volumes (500 GB/night), and automatically retries failed transfers, making it the correct choice for this use case.

Answer

Transfer Appliance

Answer

BigQuery Data Transfer Service

Answer

Datastream

Question 17

A company uses Cloud Dataproc for large-scale Spark jobs. They notice that some jobs are failing due to insufficient memory on the worker nodes. They want to improve memory management without over-provisioning. Which three configurations should they apply? (Choose 3)

Accepted Answer

Set spark.executor.memory to a value that fits within the node memory. Setting spark.executor.memory to a value that fits within the node memory ensures that each executor does not exceed the available RAM on a worker node, preventing out-of-memory (OOM) errors. This configuration directly controls the heap size allocated to each executor, and when combined with spark.executor.cores and spark.executor.instances, it allows precise memory budgeting per node. Over-provisioning is avoided by calculating the maximum safe executor memory as (node memory - OS overhead - HDFS cache) / number of executors per node.

Answer

Use local SSDs for temporary storage

Answer

Use preemptible worker nodes for volatile tasks

Question 18

You are designing a row key for Cloud Bigtable to store user activity logs. Each log entry has a timestamp (millisecond precision) and a user ID. There will be millions of writes per second from many users. To avoid hotspotting, which row key design is BEST?

Accepted Answer

hash(userID)#userID#timestamp_millis. Best because it uses a hash of the user ID as the row key prefix, which distributes writes across all Bigtable nodes and avoids hotspotting. Appending the user ID and timestamp ensures uniqueness and supports efficient queries for a specific user's logs. This design prevents the sequential timestamp from creating a single hot node, which is critical for handling millions of writes per second.

Answer

timestamp_millis#hash(userID)

Answer

timestamp_millis#userID

Answer

userID#timestamp_millis

Question 19

A data platform team uses Cloud Composer 2 to orchestrate a DAG that runs a Dataproc Serverless batch job producing a partitioned BigQuery table. The DAG passes the output location to a downstream task that runs a dbt model. The team wants failures in the dbt task to automatically trigger a retry of only that task, and they want the DAG to expose the Dataproc job ID in the Airflow UI for troubleshooting. Which approach BEST satisfies both requirements?

Accepted Answer

Use the DataprocSubmitJobOperator and pass the returned job reference to XCom, then set retries on the dbt task and read the XCom value in its template fields.. DataprocSubmitJobOperator returns a job reference that Airflow pushes to XCom, which makes the job ID available for downstream tasks and visible in the Airflow UI. Configuring retries specifically on the dbt task ensures that a transient failure there triggers only that task to rerun, leaving the already-successful Dataproc submission intact. This combination precisely meets both the observability and granular retry goals.

Answer

Use the DataprocSubmitJobOperator and configure the DAG's default_args with retries, which applies the retry policy to all tasks including the dbt task.

Answer

Use a single PythonOperator that submits the Dataproc job via the API, waits for completion, and then invokes dbt, with retries set on the PythonOperator.

Answer

Use the DataprocSubmitJobOperator with retries set on it, and add a separate downstream PythonOperator that runs dbt without its own retry configuration.

Question 20

A company is using Pub/Sub to ingest clickstream events. They need to ensure that events are delivered to a subscriber at least once, but duplicates can be tolerated. They also need to filter events by type before processing. Which subscription configuration should be used?

Accepted Answer

Pull subscription with a filter on event type attribute. A pull subscription with a filter on the event type attribute satisfies both requirements: Pub/Sub's at-least-once delivery is the default for pull subscriptions, so duplicates are tolerated, and message filtering via subscription filter expressions (on attributes like event type) prevents unwanted events from being delivered. This is the only option that combines at-least-once semantics with server-side filtering.

Answer

Pull subscription with exactly-once delivery enabled

Answer

Push subscription with no filter

Answer

Push subscription with a dead letter topic