PDE Practice Test 16 — 15 Questions

Question 1

A company is building a real-time streaming pipeline using Pub/Sub and Dataflow to process clickstream data. The pipeline writes aggregated metrics to BigQuery every 10 seconds using a fixed window. During peak traffic, some windows produce duplicate rows in BigQuery. What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Dataflow is retrying BigQuery streaming inserts after a timeout, and the retries succeed even though the original insert succeeded.. Dataflow uses at-least-once semantics for streaming inserts into BigQuery. When a streaming insert times out, Dataflow retries the insert, and if the original insert actually succeeded but the acknowledgment was lost, the retry produces a duplicate row. This is a known behavior of BigQuery streaming inserts with retry logic.

Answer

The pipeline uses default triggers instead of after-watermark triggers.

Answer

The fixed window duration is too short, causing overlapping windows.

Answer

The pipeline is using too many Dataflow workers, causing load balancing issues.

Question 2

A team wants to use Cloud Pub/Sub Lite for a high-throughput, low-cost messaging system. They need exactly-once delivery to subscribers. What should they know about Pub/Sub Lite's delivery guarantees?

Accepted Answer

Pub/Sub Lite provides at-least-once delivery, same as standard Pub/Sub.. Pub/Sub Lite offers at-least-once delivery like standard Pub/Sub; exactly-once is not guaranteed.

Answer

Pub/Sub Lite provides exactly-once delivery when using push subscriptions.

Answer

Pub/Sub Lite provides exactly-once delivery when using pull subscriptions.

Answer

Pub/Sub Lite supports exactly-once delivery by default.

Question 3

A company is building a data lake on Cloud Storage for log analysis. Log files (CSV) arrive every 5 minutes from multiple sources. The files should be ingested into BigQuery for reporting within 15 minutes. Which approach best meets the requirements with minimal operational overhead?

Accepted Answer

Set up a Cloud Storage notification to trigger a Cloud Function that loads each file into BigQuery using the BigQuery API.. Cloud Storage notifications trigger a Cloud Function on each file upload, which then loads the file into BigQuery via the BigQuery API. This provides near-real-time ingestion (within seconds of file arrival) with minimal operational overhead, as there are no servers to manage and no scheduling needed. The 5-minute file arrival and 15-minute SLA are easily met without complex infrastructure.

Answer

Schedule a daily batch load from Cloud Storage to BigQuery using the BigQuery Data Transfer Service.

Answer

Use Dataflow to read from Pub/Sub (ingested from Cloud Storage) and write to BigQuery.

Answer

Use BigQuery federated queries to query the CSV files directly from Cloud Storage.

Question 4

An organization needs to trigger a Cloud Run service whenever a new file is uploaded to a specific Cloud Storage bucket. Which service should they use to set up this event-driven architecture?

Accepted Answer

Eventarc with a trigger for Cloud Storage events. Eventarc is the recommended Google Cloud service for routing events from Cloud Storage (and 90+ other sources) to Cloud Run, Cloud Functions, or GKE. It provides a native Cloud Storage trigger type, handles authentication, retries, and dead-lettering, and integrates directly with Cloud Run's event delivery model. This is the canonical event-driven pattern for GCS-to-Cloud-Run.

Answer

Pub/Sub notifications on the bucket with a push subscription to Cloud Run

Answer

Cloud Scheduler calling Cloud Run on a schedule

Answer

Cloud Functions with a GCS trigger

Question 5

Refer to the exhibit. A Cloud Build step fails when pushing a Docker image to Artifact Registry. What is the missing IAM role for the Cloud Build service account?

Accepted Answer

roles/artifactregistry.writer. The Cloud Build service account needs the `roles/artifactregistry.writer` role to push Docker images to Artifact Registry. This role grants the necessary permissions to upload artifacts, including images, to the registry. Without it, the build step fails with an authorization error.

Answer

roles/containerregistry.admin

Answer

roles/storage.objectCreator

Answer

roles/cloudbuild.builds.editor

Question 6

A team needs to migrate an existing on-premises Hadoop Hive workload to Google Cloud. They want to minimize code changes and use a managed service for transient clusters. Which service should they choose?

Accepted Answer

Cloud Dataproc. Cloud Dataproc is the correct choice because it is a managed Spark and Hadoop service that supports Hive workloads natively, allowing you to run existing Hive scripts with minimal changes. It also supports transient clusters, which can be automatically scaled up and down, aligning with the requirement for transient clusters.

Answer

Cloud Dataflow

Answer

Cloud Dataprep

Answer

BigQuery

Question 7

A data pipeline reads thousands of JSON files from Cloud Storage, processes them with Cloud Dataflow, and writes to BigQuery. The pipeline sometimes fails because of malformed JSON records. Which three steps should the data engineering team take to improve pipeline reliability? (Choose THREE.)

Accepted Answer

Integrate Cloud Pub/Sub as an intermediary to buffer and allow message retry. Integrating Cloud Pub/Sub as an intermediary decouples the ingestion of JSON files from the Dataflow pipeline. Pub/Sub provides at-least-once delivery and automatic retries for messages that are not acknowledged, which buffers against transient failures and malformed records. This allows the pipeline to pull messages at its own pace and retry processing without losing data.

Answer

Use a try-catch block in the pipeline to retry processing failed records

Answer

Create a Cloud Monitoring alert on pipeline failures

Question 8

A company wants to implement a data lake on Google Cloud. They need to store raw, structured data in open formats and allow querying directly from BigQuery without loading. Which TWO services or features should they use? (Choose 2)

Accepted Answer

Cloud Storage (GCS). Cloud Storage (GCS) [CORRECT] is the right choice because a data lake on Google Cloud stores raw, structured data as objects in open formats (e.g., Parquet, Avro, ORC, JSON) in buckets, providing durable, scalable, and cost-effective storage. BigLake [CORRECT] is also correct because it is a storage engine that lets BigQuery query data directly in Cloud Storage (and other sources) without loading it, using external tables with fine-grained access control and metadata caching. Together, GCS holds the raw files and BigLake exposes them to BigQuery for direct querying, which matches the requirement exactly. Dataproc (B) is a managed Spark/Hadoop service for processing, not the storage or direct-query layer, and Dataflow (C) is a managed Apache Beam pipeline service for stream/batch processing, so neither fulfills the storage-plus-query requirement. Cloud SQL (D) is a managed relational database for transactional workloads, not a data lake storage layer, and it does not provide the open-format object storage or BigQuery external querying described.

Answer

Dataproc

Answer

Dataflow

Answer

Cloud SQL

Question 9

A data engineer needs to design a data pipeline that ingests streaming data from Cloud Pub/Sub, performs real-time aggregations, and loads the results into BigQuery for dashboarding. Which Google Cloud service should they use for the streaming aggregation step?

Accepted Answer

Dataflow. Dataflow is a fully managed service for stream and batch processing that integrates with Pub/Sub and BigQuery. It supports exactly-once processing and low-latency streaming.

Answer

Cloud Functions

Answer

Cloud Dataproc

Answer

Cloud Composer

Question 10

You want to quickly estimate the number of distinct visitors to your website from a large BigQuery table. Which function provides an approximate count with low latency?

Accepted Answer

APPROX_COUNT_DISTINCT. APPROX_COUNT_DISTINCT is a built-in BigQuery function that returns an approximate count of distinct values using a HyperLogLog++ sketch. It is designed for large-scale data, providing fast, low-latency results with a small error rate (typically <1%). This makes it ideal for quickly estimating distinct visitors without the overhead of exact counting.

Answer

HyperLogLog++

Answer

COUNT(DISTINCT)

Answer

APPROX_QUANTILES

Question 11

A data engineer runs this Dataflow template to load CSV files from Cloud Storage into BigQuery. The job fails with a 'File pattern not matching any files' error. What is the most likely cause?

Accepted Answer

The CSV files are stored in a subdirectory that is not matched by the pattern. The error 'File pattern not matching any files' indicates that the file pattern specified in the Dataflow template does not resolve to any existing objects in Cloud Storage. If the CSV files are stored in a subdirectory (e.g., gs://bucket/subdir/*.csv) but the pattern only references the root (e.g., gs://bucket/*.csv), no files will be matched. This is the most likely cause because the pattern must explicitly include the subdirectory path.

Answer

The bucket name is incorrectly spelled

Answer

The template has a bug

Answer

The output table does not exist

Question 12

A company needs to stream real-time user click events from a web application to BigQuery for analysis. Which Google Cloud architecture is most suitable?

Accepted Answer

App Engine -> Pub/Sub -> Dataflow -> BigQuery. It provides a fully managed, scalable, and decoupled architecture for ingesting real-time click events. Pub/Sub acts as a durable, asynchronous message buffer that can handle high-throughput streams, Dataflow (Apache Beam) processes the events in near real-time with exactly-once semantics, and BigQuery serves as the analytics warehouse. This pattern is the recommended Google Cloud approach for streaming analytics, as it decouples producers from consumers and supports auto-scaling.

Answer

Cloud Scheduler -> BigQuery

Answer

Compute Engine -> Cloud Storage -> BigQuery

Answer

Cloud Functions -> BigQuery

Question 13

A company wants to use dbt (data build tool) to transform data in BigQuery. They have a Cloud Storage bucket containing raw CSV files that are loaded daily into BigQuery via an external table. Which dbt feature should they use to modularize the transformation logic and handle dependencies between models?

Accepted Answer

dbt models with ref(). C is correct because dbt models with the `ref()` function allow you to modularize SQL transformation logic and automatically handle dependencies between models. When you use `ref('model_name')`, dbt builds a dependency graph, ensuring models are executed in the correct order based on their references. This is essential for transforming raw data from an external table into a structured, analytics-ready dataset in BigQuery.

Answer

dbt tests

Answer

dbt snapshots

Answer

dbt seeds

Question 14

You are designing a Dataflow pipeline to process streaming data. The pipeline may encounter malformed records. You need to handle these errors without failing the entire pipeline and store the bad records for later analysis. What is the best practice?

Accepted Answer

Use a dead letter sink to write malformed records to a separate Pub/Sub topic or GCS location.. A dead letter sink writes malformed records to a separate Pub/Sub topic or GCS location so they can be analyzed later without failing the pipeline, which is the recommended Apache Beam pattern for handling bad records. This preserves the main pipeline's throughput and keeps the bad data available for debugging or reprocessing.

Answer

Catch the exception and log it, then continue processing.

Answer

Write all records to BigQuery using the Storage Write API and handle errors in the write operation.

Answer

Raise an exception in the DoFn to stop the pipeline for manual intervention.

Question 15

Drag and drop the steps to set up Cloud IAP (Identity-Aware Proxy) for an App Engine app into the correct order.

Accepted Answer

Enable the Identity-Aware Proxy API, configure the OAuth consent screen, create an OAuth 2.0 client ID, enable IAP for the App Engine application, then add members to the IAP-secured Web App User role.. IAP verifies identity and authorization before allowing access to the application.

Answer

Enable IAP for the App Engine application, enable the Identity-Aware Proxy API, configure the OAuth consent screen, create an OAuth 2.0 client ID, then add members to the IAP-secured Web App User role.

Answer

Configure the OAuth consent screen, create an OAuth 2.0 client ID, enable IAP for the App Engine application, enable the Identity-Aware Proxy API, then add members to the IAP-secured Web App User role.

Answer

Enable the Identity-Aware Proxy API, configure the OAuth consent screen, enable IAP for the App Engine application, create an OAuth 2.0 client ID, then add members to the IAP-secured Web App User role.