PDE Practice Test 24 — 15 Questions

Question 1

A data engineer is designing a Bigtable row key for a time-series application that records temperature sensor readings every second. To avoid hotspotting, they want to distribute writes across all nodes. Which row key design is best?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

[hash of sensor_id]#[timestamp]. Hotspotting occurs when sequential keys hit a single tablet server. A reversed timestamp or hashed prefix distributes writes. Pre-pending a hash ensures even distribution. Timestamp alone causes hotspotting. Sensor ID + timestamp can still be sequential if sensor ID is low cardinality.

Answer

[timestamp reversed]#[sensor_id]

Answer

[sensor_id]#[timestamp]

Answer

[timestamp]#[sensor_id]

Question 2

A data scientist wants to import a pre-trained TensorFlow model into BigQuery ML for batch predictions. The model is stored in a Cloud Storage bucket. Which statement is correct?

Accepted Answer

Use CREATE MODEL with model_type='tensorflow' and model_path='gs://bucket/model'.. BigQuery ML supports importing TensorFlow models via CREATE MODEL with model_type='tensorflow' and model_path pointing to the SavedModel directory in Cloud Storage.

Answer

Use CREATE MODEL with model_type='imported_tensorflow' and model_path='gs://bucket/model'.

Answer

First upload the model to Vertex AI Model Registry, then reference it in BigQuery ML.

Answer

Use the ML.IMPORT_MODEL function to load the model into BigQuery.

Question 3

A company uses Cloud Composer for pipeline orchestration. They need to define task dependencies where Task B and Task C can run in parallel after Task A, and Task D must run after both B and C complete. How should they define the DAG?

Accepted Answer

A >> [B, C] >> D. Using bitshift operators: A >> [B, C] >> D sets B and C after A, and D after both B and C complete.

Answer

A >> B; B >> D; A >> C; C >> D

Answer

A >> B >> C >> D

Answer

A.set_downstream(B); B.set_upstream(C); C.set_downstream(D)

Question 4

A data team needs to share a BigQuery dataset with another business unit. They want to provide a point-in-time snapshot of the data without incurring additional storage costs for the copy. Which BigQuery feature should they use?

Accepted Answer

BigQuery table clones. Clones use the same underlying storage as the source table; snapshots also share storage but are immutable. Both are cost-effective. For regular updates, clones are more flexible.

Answer

BigQuery table snapshots

Answer

BigQuery authorized views

Answer

BigQuery export to Cloud Storage

Question 5

A data team wants to use Approximate Aggregation Functions in BigQuery to get faster query results. Which two functions can they use? (Choose 2)

Accepted Answer

APPROX_QUANTILES. BigQuery provides APPROX_COUNT_DISTINCT for approximate distinct counts and APPROX_QUANTILES for approximate quantiles. Other approximate functions include APPROX_TOP_COUNT and APPROX_TOP_SUM.

Answer

APPROX_SUM

Answer

APPROX_AVG

Answer

APPROX_MEDIAN

Question 6

A data engineer is designing a Dataflow pipeline that reads from a Kafka topic (using Pub/Sub for Kafka) and writes to BigQuery. The data schema may change over time, with new fields appearing. The engineer wants to handle schema drift automatically without failing the pipeline. Which approach should the engineer use?

Accepted Answer

Store the entire record as a single JSON string column in BigQuery and parse it later.. Storing the entire record as a single JSON string column in BigQuery allows the pipeline to accept any schema changes without requiring schema modifications at write time. This approach decouples the ingestion from schema evolution, enabling the data to be parsed later using BigQuery's JSON functions (e.g., JSON_EXTRACT) or by loading into a separate schema-on-read layer. It avoids pipeline failures caused by mismatched fields or type changes.

Answer

Use a Dataflow side input that reads the latest schema from a file and updates the BigQuery schema accordingly.

Answer

Define a UDF in Dataflow that dynamically adjusts the output schema.

Answer

Configure the Dataflow pipeline to use BigQuery's schema autodetect option for each insert.

Question 7

You are implementing a data pipeline that reads from Cloud Storage (parquet files), transforms data with Cloud Dataflow, and writes to BigQuery. The pipeline runs on a batch schedule every hour. You notice that the Dataflow job takes 10 minutes, but the overall pipeline latency is 15 minutes due to file availability and scheduling. The business requires latency under 5 minutes. Which change should you make?

Accepted Answer

Switch to streaming pipeline with .watchForNewFiles() and process files as they arrive. The root cause of the latency is file availability and scheduling delay, not the processing time. Switching to a streaming pipeline with `.watchForNewFiles()` (or the equivalent `FileIO.match().continuously()`) allows Dataflow to process files as soon as they arrive in Cloud Storage, eliminating the batch scheduling wait and reducing overall latency to near the processing time.

Answer

Batch the hourly data into a single larger hourly run

Answer

Use a larger machine type for the Dataflow workers

Answer

Increase the number of workers and use smaller input files

Question 8

Based on the exhibit, what is the most likely cause of duplicate rows despite using the same event_id as insertId?

Accepted Answer

BigQuery's streaming buffer deduplication is best-effort and may not catch duplicates within a short time window.. BigQuery's streaming buffer uses best-effort deduplication based on the `insertId` field. When multiple rows are inserted with the same `event_id` mapped to `insertId` within a short time window (typically up to a few minutes), the deduplication mechanism may fail to remove all duplicates, especially under high throughput or network retries. This is a documented limitation of BigQuery streaming, not a guarantee of exactly-once semantics.

Answer

The Dataflow pipeline is retrying inserts due to network errors, and the same event_id is not being used in retries.

Answer

The pipeline is writing more than 100,000 rows per second, exceeding BigQuery's streaming quota.

Answer

The table is partitioned by timestamp, so BigQuery cannot deduplicate across partitions.

Question 9

You are using Looker to model data from BigQuery. You have a dimension that should be filtered by a user attribute (e.g., user's region). Which LookML concept allows you to apply dynamic row-level security based on user attributes?

Accepted Answer

Access filter. Access filters in LookML allow dynamic filtering based on user attributes from the authentication system. Required access grants are for permissions, not dynamic filtering.

Answer

Custom field

Answer

Derived table

Answer

Required access grant

Question 10

During a Vertex AI training pipeline, the training job fails with an error: 'Out of memory: Killed process'. The model is a large deep learning model using TensorFlow. Which THREE steps should the team take to resolve this issue?

Accepted Answer

Enable memory growth configuration in TensorFlow. TensorFlow by default allocates all available GPU memory, which can cause out-of-memory (OOM) errors when other processes or the system itself need memory. Enabling memory growth with `tf.config.experimental.set_memory_growth` allows TensorFlow to allocate memory incrementally, reducing the risk of OOM kills. This is a direct mitigation for the 'Killed process' error caused by memory exhaustion.

Answer

Change to a distributed training strategy

Answer

Switch the training from GPU to TPU accelerator

Question 11

A financial services firm uses Cloud Pub/Sub to ingest real-time market data. The data is processed by a Cloud Dataflow streaming pipeline that aggregates trades per symbol and writes to BigQuery. The pipeline currently uses a single global window with a trigger that fires every minute. The firm now needs to support late data up to 5 minutes and also wants to reduce the number of writes to BigQuery to avoid hitting the table limit of 1,500 inserts per second. The current pipeline writes every minute, which is acceptable for inserts per second, but after adding late data handling, the number of writes doubles. How can you redesign the pipeline to handle late data while keeping write volume low?

Accepted Answer

Use session windows with a gap duration of 5 minutes and a count-based trigger that fires after accumulating 1000 elements. Session windows naturally group events into bursts of activity separated by a gap duration (5 minutes), which reduces the number of writes by accumulating many trades per symbol before emitting a pane. Adding a count-based trigger that fires after 1000 elements further limits write frequency, keeping the insert rate well below BigQuery's 1,500 per second limit while still allowing late data up to the gap duration. This design handles late data implicitly within the session gap and avoids the write amplification seen with fixed windows and frequent triggers.

Answer

Use fixed windows of 5 minutes with allowed lateness 5 minutes and trigger every 30 seconds

Answer

Increase the global window duration to 10 minutes and keep the same trigger

Answer

Discard all late data and keep the current windowing

Question 12

A company uses Cloud Dataproc for ephemeral clusters to run batch jobs. They want to ensure job reliability and data quality. Which two configuration options should they use? (Choose two.)

Accepted Answer

Use initialization actions for cluster setup.. Initialization actions allow you to install dependencies, configure software, or validate data sources on every cluster node before jobs run. This ensures consistent cluster setup across ephemeral clusters, directly supporting job reliability and data quality by preventing environment mismatches or missing libraries.

Answer

Enable preemptible VMs for cost savings.

Answer

Enable idle timeout to automatically delete clusters.

Answer

Use custom machine types for better performance.

Question 13

A company wants to use BigQuery to query data stored in Cloud Storage as Parquet files without loading the data into BigQuery storage. Which feature should they use?

Accepted Answer

BigQuery external tables. BigQuery external tables allow querying data stored in Cloud Storage (including Parquet files) directly without loading it into BigQuery storage. This is achieved by defining a table schema that references the external data source, enabling BigQuery to read the Parquet files on-the-fly using its federated query engine.

Answer

BigQuery ingestion from Cloud Storage using load jobs

Answer

Cloud Storage FUSE to mount bucket and query

Answer

BigQuery federated queries with Cloud Storage

Question 14

A healthcare startup is deploying a natural language processing (NLP) model for extracting medical entities from clinical notes. The model is a fine-tuned BERT model served on Vertex AI Prediction using a custom container. The team observes that prediction latency is around 500ms per request, but they need to handle up to 100 requests per second (QPS) with end-to-end latency under 200ms. The model currently runs on n1-standard-4 machines (4 vCPU, 15 GB memory). During load testing, CPU utilization reaches 90% and memory usage is 12 GB. The team is considering options to meet the requirements. Which action should they take?

Accepted Answer

Use a machine type with a GPU, such as n1-standard-4 with a NVIDIA Tesla T4 accelerator, and optimize the model with TensorRT.. The bottleneck is CPU-bound inference (90% CPU utilization) with memory well within limits (12 GB of 15 GB). Adding a GPU (NVIDIA Tesla T4) and optimizing with TensorRT reduces per-request latency via hardware acceleration and graph optimizations, enabling sub-200ms inference at 100 QPS. This directly addresses the latency requirement without changing the machine family or scaling strategy.

Answer

Switch to n1-highmem-4 machines to provide more memory for the model.

Answer

Deploy the model using TensorFlow Serving with CPU-only nodes and increase the number of replicas.

Answer

Move the model to Cloud Run with automatic scaling to handle the QPS.

Question 15

Which THREE actions reduce the cost of a Cloud Composer environment?

Accepted Answer

Delete old and unused DAG files to reduce scheduler load. Deleting old and unused DAG files reduces the number of DAGs the scheduler must parse and evaluate. The Cloud Composer scheduler scans the DAG folder every 30 seconds by default; fewer DAG files mean lower CPU and memory consumption, directly reducing the cost of the environment's compute resources.

Answer

Use standard network tier instead of premium

Answer

Increase the number of schedulers for higher throughput