PDE Practice Test 40 — 15 Questions

Question 1

A data engineer needs to design a batch pipeline that processes daily log files from Cloud Storage and writes aggregated results to BigQuery. Which service is most appropriate for this ETL job?

Accepted Answer

Dataproc with PySpark. Dataproc with PySpark is the most appropriate choice because it provides a managed Spark/Hadoop environment that can efficiently process large daily log files stored in Cloud Storage using distributed computing. PySpark's native integration with BigQuery via the Spark BigQuery connector allows direct writing of aggregated results, making it ideal for batch ETL workloads that require complex transformations and high throughput.

Answer

Cloud Pub/Sub with Cloud Functions

Answer

Cloud Composer

Answer

Cloud Data Fusion

Question 2

A data engineer is building a feature store for ML models using Vertex AI Feature Store. The features are computed daily from BigQuery and need to be available for both online predictions (low latency) and offline training. Which two actions must the engineer take? (Choose TWO)

Accepted Answer

Create an entity type in the feature store. Vertex AI Feature Store requires creating an entity type and a featurestore (which serves online and offline). Features are ingested via Dataflow or batch jobs.

Answer

Deploy a custom TensorFlow model on Vertex AI

Answer

Use Cloud SQL to store feature metadata

Answer

Create a BigQuery table for offline serving

Question 3

A data engineer needs to implement data lineage tracking for a BigQuery data warehouse. They want to automatically capture column-level lineage from ETL jobs run by Dataform and from manual SQL queries executed in the BigQuery console. Which approach meets these requirements?

Accepted Answer

Use BigQuery's built-in column-level lineage, which automatically tracks lineage for all queries, and query it using INFORMATION_SCHEMA.JOBS. BigQuery's column-level lineage is automatically captured for all queries (including Dataform and console) and can be queried via the INFORMATION_SCHEMA.JOBS_BY_USER views. Dataplex lineage requires integration but BigQuery lineage is native.

Answer

Enable Dataplex Universal Catalog and use the Dataplex Lineage API to capture lineage from both sources

Answer

Use the BigQuery Lineage API provided by Data Catalog to register lineage manually

Answer

Export Dataform execution logs to Cloud Logging and use a custom script to extract lineage

Question 4

An organization wants to ingest on-premises Oracle database changes into BigQuery for real-time analytics with minimal latency. The Oracle database is version 19c and has a high transactional volume. Which Google Cloud service should they use?

Accepted Answer

Datastream. Datastream is the correct service because it is purpose-built for real-time, serverless change data capture (CDC) from Oracle databases (including 19c) to BigQuery. It uses LogMiner to read redo logs with minimal overhead, supporting high transactional volumes and sub-second latency without requiring custom code or manual schema management.

Answer

Pub/Sub

Answer

Dataflow

Answer

Storage Transfer Service

Question 5

A data engineer needs to restrict access to BigQuery datasets such that only data from approved VPC networks can query them. They also need to audit data access. Which two security controls should they implement? (Choose two.)

Accepted Answer

Cloud Audit Logs. Cloud Audit Logs (option A) is correct because it provides a record of all administrative and data-access operations on BigQuery datasets, enabling the data engineer to audit who accessed what data and from which network. This satisfies the requirement to audit data access by capturing detailed logs of API calls, including the identity of the caller and the source IP address.

Answer

Customer-managed encryption keys (CMEK)

Answer

Data Loss Prevention (DLP)

Answer

IAM roles

Question 6

A Cloud Build pipeline is set up to train a model on Vertex AI. The build fails with the error: 'ERROR: (gcloud.ai-platform.jobs.submit.training) NOT_FOUND: The parent project does not exist.' The project ID and the service account are correctly configured. What is the most likely cause?

Accepted Answer

The Cloud Build service account does not have the aiplatform.jobs.create permission on the project.. The error 'NOT_FOUND: The parent project does not exist' indicates that the Cloud Build service account lacks the necessary IAM permission to submit a training job to Vertex AI. Even though the project ID and service account are correctly configured, the Cloud Build service account must have the 'aiplatform.jobs.create' permission (or the 'Vertex AI User' role) on the project. Without this, the API call fails because the service account is not authorized to access the project resource.

Answer

The region specified for the training job does not exist.

Answer

The training job requires a GPU, which is not available in the specified region.

Answer

The training package is not uploaded to Cloud Storage before the pipeline runs.

Question 7

A data engineering team needs to ingest streaming data from an existing Kafka cluster (on-premises) into Google Cloud for real-time analytics. They want to minimize changes to the existing Kafka setup and avoid long-term operational overhead. Which TWO approaches should they consider?

Accepted Answer

Deploy a Kafka Connect cluster on Google Cloud with the Pub/Sub sink connector. Using Kafka Connect with the Pub/Sub connector or setting up a Dataproc cluster running Kafka with mirroring are two ways to bridge on-premises Kafka to GCP with minimal changes.

Answer

Use Storage Transfer Service to copy Kafka logs from on-premises to GCS

Answer

Use Datastream to capture changes from Kafka

Answer

Replace the on-premises Kafka cluster with Google Cloud Pub/Sub

Question 8

You need to ingest streaming data from a custom application into BigQuery with exactly-once semantics and low latency. The data volume is up to 10 MB/s. Which TWO services should you combine?

Accepted Answer

Pub/Sub. Pub/Sub provides reliable, low-latency message ingestion, and Dataflow can read from Pub/Sub and write to BigQuery using the Storage Write API, which supports exactly-once semantics. The Storage Write API with committed mode ensures exactly-once delivery.

Answer

Cloud Functions

Answer

BigQuery legacy streaming inserts

Answer

Datastream

Question 9

A company needs to stream data from a MySQL database to BigQuery with a latency under 10 seconds. They also need to handle schema changes automatically. Which TWO services should they combine?

Accepted Answer

BigQuery. Datastream captures CDC and can write to BigQuery directly. Pub/Sub is not needed if Datastream writes directly.

Answer

Dataflow

Answer

Pub/Sub

Answer

Cloud SQL

Question 10

You are designing a streaming pipeline that needs to handle sudden spikes in traffic without losing data. The pipeline uses Pub/Sub and Dataflow. Which configuration ensures data is not lost if Dataflow falls behind?

Accepted Answer

Use Pub/Sub with a pull subscription and set the message retention duration to 7 days. Pub/Sub stores messages for up to 7 days, allowing Dataflow to catch up. Dataflow uses checkpointing to track progress. This combination ensures no data loss.

Answer

Use Cloud Pub/Sub Lite with a smaller retention period

Answer

Use Pub/Sub with a push subscription and increase the acknowledgment deadline

Answer

Use Pub/Sub with exactly-once delivery and Dataflow with at-least-once processing

Question 11

A financial services company uses Cloud Bigtable to store trade data. They are experiencing hot-spotting on a single node, causing high latency. The row key format is [trade_id]#[timestamp]. Which row key design change would BEST distribute writes across tablets?

Accepted Answer

Use a hashed prefix of the trade_id, e.g., [hash(trade_id)]#[trade_id]#[timestamp]. Adding a hashed prefix of the trade_id ensures that writes are evenly distributed across all Bigtable tablets. Bigtable partitions data by row key lexicographic order; without a hash, sequential trade IDs or timestamps cause all recent writes to land on a single tablet, creating a hotspot. The hash spreads the write load uniformly, regardless of the underlying key pattern.

Answer

Use a single row key of [timestamp]

Answer

Increase the number of Bigtable nodes to 20

Answer

Change row key to [timestamp]#[trade_id]

Question 12

A company uses Cloud Pub/Sub for event ingestion. They want to ensure that if a subscriber fails to process a message after 5 attempts, the message is sent to a dead letter topic for analysis. Which TWO configurations are needed?

Accepted Answer

Set max delivery attempts to 5 on the subscription.. Dead letter topics require setting max delivery attempts and specifying a dead letter topic.

Answer

Set the subscription's ack deadline to 600 seconds.

Answer

Enable message ordering on the subscription.

Answer

Set the subscription type to push.

Question 13

A company wants to build a real-time dashboard for monitoring application logs. The logs are ingested via Pub/Sub and must be processed with low latency (sub-second). You need to enrich the logs with user metadata from Cloud SQL and store the results in BigQuery for analysis. Which TWO services should be used for the stream processing? (Choose two.)

Accepted Answer

Dataflow. Dataflow can read from Pub/Sub, enrich with side inputs from Cloud SQL, and write to BigQuery. Pub/Sub is the ingestion point. The pipeline uses Dataflow for stream processing.

Answer

Dataproc

Answer

Cloud Data Fusion

Answer

Cloud Functions

Question 14

An application requires a globally distributed, strongly consistent database with 99.999% availability SLA. The workload is OLTP with high throughput across continents. Which service fits best?

Accepted Answer

Cloud Spanner. Cloud Spanner is the only service that provides globally distributed, strongly consistent (external consistency via TrueTime) OLTP with 99.999% availability SLA. It supports high-throughput ACID transactions across continents using synchronous replication and atomic clocks, meeting all stated requirements.

Answer

Cloud SQL with cross-region replicas

Answer

Firestore

Answer

Cloud Bigtable

Question 15

A company uses Workflows to orchestrate a multi-step data pipeline. One step calls an HTTP endpoint that may take up to 10 minutes, but the default Workflows timeout is too short. They also need to handle transient errors with retries. Which TWO configurations should they apply? (Choose 2)

Accepted Answer

Set a step timeout of 600 seconds for the HTTP call step. To extend the timeout, set a step timeout of 600 seconds (10 minutes). To handle transient errors, use a retry policy with appropriate conditions. Setting the entire workflow timeout to 10 minutes is not necessary if individual step timeouts are set. The default retry policy does not cover all transient errors. Adding a dead letter queue is for event-driven patterns, not Workflows.

Answer

Configure a dead letter queue for failed steps

Answer

Use the default retry policy on the step

Answer

Set the workflow execution timeout to 600 seconds