Courseiva

Google Professional Data Engineer (PDE) — Questions 451–525

890 questions total · 12pages · All types, answers revealed

Take a mock examExam hub
Page 6

Page 7 of 12

Page 8
451
Multi-Selectmedium

A data warehouse in BigQuery is experiencing performance issues. Which THREE techniques can improve performance without moving data to a different storage system?

Select 3 answers
A.Partition by date
B.Cluster by common filter columns
C.Use streaming buffer
D.Use BigQuery slots
E.Use materialized views
AnswersA, B, E

Partitioning limits scans to relevant partitions.

Why this answer

Partitioning by date in BigQuery allows the query engine to prune entire partitions that do not match the query's date filter, significantly reducing the amount of data scanned and improving performance. This technique works without moving data to a different storage system because it is a metadata-level reorganization of the existing table.

Exam trap

Google Cloud often tests the misconception that streaming buffer (Option C) is a performance optimization, when in fact it is designed for near-real-time ingestion and can degrade query performance due to the small, unoptimized files it creates.

Full explanation →
452
MCQeasy

Which Dataflow feature allows you to package a pipeline into a reusable template that can be deployed with different parameters at runtime?

A.Cloud Dataproc
B.Classic Templates
C.Dataflow SQL
D.Flex Templates
AnswerD

Flex Templates use Docker containers and support complex runtime parameters.

Why this answer

Dataflow Flex Templates allow you to containerize a pipeline and provide runtime parameters, enabling reusability across different environments or jobs.

Full explanation →
453
MCQmedium

A data engineer needs to design a streaming pipeline that ingests events from multiple sources, enriches them with a lookup table stored in BigQuery (updated every hour), and writes the results to a BigQuery table for real-time dashboards. The pipeline must handle late-arriving data up to 1 hour. Which Dataflow feature should be configured to manage late data?

A.A custom watermark estimation function
B.Using side inputs with a periodic refresh
C.A trigger that fires on every late element
D.Allowed lateness on the window
AnswerD

Setting allowed lateness (e.g., 1 hour) tells the pipeline to keep the window open for late data within that duration. This is the correct way to handle late-arriving events.

Why this answer

Allowed lateness in Apache Beam specifies the maximum delay of late data to be included in the window. Watermark estimation does not control late data acceptance; triggers specify when output is emitted, not how late data is handled. Side inputs are for enrichment, not late data management.

Full explanation →
454
MCQmedium

A machine learning pipeline uses Vertex AI Pipelines. One component fails intermittently due to resource constraints. What is the best way to handle this?

A.Use retry policies in the component specification
B.Deploy the pipeline on a larger cluster
C.Increase the pipeline timeout
D.Use a different orchestrator
AnswerA

Retry policies handle intermittent failures by automatically retrying the component.

Why this answer

Vertex AI Pipelines supports retry policies at the component level via the `retry` field in the component specification (YAML or Python). This allows the pipeline to automatically re-execute a failed component when the failure is transient (e.g., resource exhaustion), without manual intervention. Retry policies are the standard mechanism for handling intermittent failures in a serverless orchestration environment like Vertex AI Pipelines.

Exam trap

Google Cloud often tests the misconception that scaling up infrastructure (Option B) is the primary fix for intermittent failures, when in fact retry policies are the correct, cost-efficient solution for transient resource constraints in a managed pipeline service.

How to eliminate wrong answers

Option B is wrong because deploying the pipeline on a larger cluster does not address the intermittent nature of the failure; it only increases resource capacity, which may not be cost-effective and does not handle transient resource spikes. Option C is wrong because increasing the pipeline timeout does not resolve resource constraints; it only gives the component more time to run, which will still fail if resources are insufficient. Option D is wrong because using a different orchestrator (e.g., Kubeflow Pipelines, Argo) does not inherently fix resource constraints; the issue is with the component's resource allocation, not the orchestration engine itself.

Full explanation →
455
Multi-Selectmedium

You are configuring Dataplex data quality rules for a BigQuery table. Which three types of rules can be defined using Dataplex's SQL-based rule engine? (Choose three.)

Select 3 answers
A.Row-level rules (e.g., condition must be true for every row)
B.Set-level rules (e.g., intersection or difference between two tables)
C.Pattern matching rules (e.g., regex on column values)
D.Table-level rules (e.g., row count threshold)
E.Column-level rules (e.g., uniqueness, nullness, range)
AnswersA, D, E

Row-level rules validate each row against a condition.

Why this answer

Dataplex's SQL-based rule engine supports row-level rules that enforce a condition that must be true for every row in a BigQuery table. These rules are defined using a SQL expression that is evaluated per row, and if any row fails the condition, the rule is violated. This allows you to validate data integrity at the most granular level, such as ensuring a column value is always positive.

Exam trap

Google often tests the distinction between rule categories that are natively supported versus those that require custom SQL workarounds, leading candidates to mistakenly select set-level or pattern matching rules as separate types when they are actually implemented within row-level rules.

Full explanation →
456
MCQmedium

A company wants to orchestrate a multi-step data processing workflow that includes calling a Cloud Run service, waiting for its completion, and then running a BigQuery query. The workflow should be serverless and integrate with Cloud Events. Which Google Cloud service should they use?

A.Eventarc
B.Cloud Workflows
C.Cloud Composer
D.Cloud Dataflow
AnswerB

Workflows is a serverless orchestration service that can call Cloud Run, BigQuery, and other APIs, and can be triggered by Eventarc.

Why this answer

Cloud Workflows is the correct choice because it is a serverless workflow orchestrator that can coordinate multi-step processes involving Cloud Run and BigQuery. It natively supports waiting for asynchronous operations (like Cloud Run job completion) via its 'call' and 'wait' steps, and it can trigger subsequent steps such as BigQuery queries. Additionally, Cloud Workflows can be triggered by Cloud Events, making it fully integrated with the event-driven architecture described.

Exam trap

The trap here is that candidates confuse Google Cloud's Eventarc (event routing) with workflow orchestration, assuming that routing events alone can handle sequencing and waiting, when in fact Eventarc lacks the state management and step coordination required for multi-step workflows.

How to eliminate wrong answers

Option A is wrong because Eventarc is a service for routing events from various sources to targets (like Cloud Run, Cloud Functions), but it does not provide workflow orchestration capabilities such as waiting for completion or sequencing steps. Option C is wrong because Cloud Composer is a managed Apache Airflow service that is not serverless (it requires provisioning and managing a cluster of workers) and is overkill for a simple multi-step workflow; it is designed for complex, scheduled pipelines, not lightweight event-driven orchestration. Option D is wrong because Cloud Dataflow is a stream and batch data processing service (based on Apache Beam) that focuses on transforming data pipelines, not on orchestrating heterogeneous services like Cloud Run and BigQuery queries; it lacks native workflow sequencing and event-driven triggers.

Full explanation →
457
MCQeasy

An engineer needs to create a Pub/Sub subscription that sends messages to an HTTPS endpoint. The endpoint must be able to acknowledge messages individually. Which type of subscription should they use?

A.Pull subscription
B.Push subscription
C.BigQuery subscription
D.Cloud Storage subscription
AnswerB

Push subscription sends messages to a webhook endpoint; ack is via HTTP 200.

Why this answer

Push subscriptions deliver messages to a configured HTTPS endpoint. The endpoint can acknowledge by returning a 200 status.

Full explanation →
458
MCQmedium

A retail company uses a machine learning model to predict inventory demand. The model is retrained weekly using Vertex AI Pipelines. Recently, the model's accuracy has degraded because the data distribution has shifted. Which action should you take to monitor and detect this drift automatically?

A.Enable Vertex AI Model Monitoring for the endpoint and configure alerting on feature drift
B.Set up alerts for when the model's mean absolute error exceeds a threshold on the evaluation dataset
C.Enable Cloud Logging for the prediction endpoint and search for error logs
D.Schedule a job to compare the distribution of incoming features with the training data using Cloud Dataflow
AnswerA

Model Monitoring automates drift detection.

Why this answer

Vertex AI Model Monitoring is purpose-built to automatically detect feature drift and prediction drift on deployed endpoints. By enabling it and configuring alerting on feature drift, you can proactively identify when the distribution of incoming features deviates from the training data, which directly addresses the root cause of accuracy degradation without manual intervention.

Exam trap

Google Cloud often tests the distinction between monitoring model performance metrics (like MAE) versus monitoring input data distributions (feature drift), and candidates mistakenly choose a performance-based alerting option because they think accuracy degradation is the only signal, ignoring that drift detection is the proactive mechanism to catch the root cause before accuracy drops.

How to eliminate wrong answers

Option B is wrong because setting alerts on mean absolute error (MAE) on the evaluation dataset only detects performance degradation after the fact, not the underlying data distribution shift; it also requires ground truth labels, which may not be available in real time. Option C is wrong because Cloud Logging for the prediction endpoint captures request/response logs and error messages, but it does not perform statistical drift analysis or compare feature distributions. Option D is wrong because scheduling a job with Cloud Dataflow to compare distributions is a custom, manual approach that lacks the automated, integrated monitoring and alerting capabilities of Vertex AI Model Monitoring, and it introduces unnecessary operational overhead.

Full explanation →
459
Multi-Selectmedium

Which THREE metrics should be monitored for a deployed machine learning model in production?

Select 3 answers
A.Number of replicas
B.Prediction error rate
C.Data drift detection
D.Training time
E.Prediction latency
AnswersB, C, E

Accuracy metric.

Why this answer

Prediction error rate (Option B) is a direct measure of model accuracy in production, reflecting how often the model's predictions deviate from actual outcomes. Monitoring this metric is essential for detecting model degradation, data quality issues, or concept drift that can silently reduce model performance over time.

Exam trap

Google Cloud often tests the distinction between operational metrics (like latency, error rate, drift) and development/infrastructure metrics (like training time, replica count) to see if candidates understand what is relevant for ongoing model monitoring versus model building or deployment scaling.

Full explanation →
460
MCQeasy

A team developed a microservice that writes logs to stdout. They want to centralize logs for analysis. Which GCP service should they use to automatically collect and store logs?

A.Install the Cloud Logging agent on the VM running the microservice.
B.Publish logs to a Pub/Sub topic and later store them.
C.Write logs directly to Cloud Storage.
D.Use the Cloud Logging client library (google-cloud-logging) for the microservice's language.
AnswerD

The client library automatically sends structured logs to Cloud Logging, enabling centralized analysis.

Why this answer

The Cloud Logging client library (google-cloud-logging) allows the microservice to write logs directly to Cloud Logging via the Cloud Logging API, without needing a separate agent or intermediate storage. This is the recommended approach for applications running in environments like GKE, Cloud Run, or Compute Engine when you want structured, automatically collected logs that are immediately available for analysis in Cloud Logging.

Exam trap

Google often tests the misconception that you must install an agent (Option A) to collect logs from any application, but the trap here is that modern microservices can use client libraries to send logs directly to Cloud Logging, making agents unnecessary for custom applications.

How to eliminate wrong answers

Option A is wrong because installing the Cloud Logging agent on the VM is unnecessary and adds operational overhead; the agent is designed for legacy applications that cannot be modified, but the microservice can use the client library to send logs directly via the API. Option B is wrong because publishing logs to a Pub/Sub topic adds latency and complexity (you would need a subscriber to write to Cloud Logging), and it is not an automatic collection method—it requires additional infrastructure to store and forward logs. Option C is wrong because writing logs directly to Cloud Storage is not automatic; it requires custom code to format and upload log files, and Cloud Storage is not a log analysis tool—it is object storage, so logs would not be searchable in Cloud Logging without additional processing.

Full explanation →
461
Multi-Selectmedium

A data science team has deployed a custom TensorFlow model on Vertex AI Prediction. They notice increasing prediction latency and a growing number of 503 errors during peak traffic hours. The model is served using a single regional endpoint with min replica count of 2 and max replica count of 10. Which TWO actions should the team take to address these issues?

Select 2 answers
A.Use a larger machine type (e.g., n1-highmem-8) instead of the current n1-standard-4 to improve per-replica throughput.
B.Enable autoscaling with a higher max replica count and configure a CPU utilization target of 60%.
C.Reduce the min replica count to 0 to allow the service to scale down to zero when not in use.
D.Deploy the model as a batch prediction job and move all online predictions to batch.
E.Switch to a global endpoint with automatic scaling to distribute traffic across multiple regions.
AnswersB, E

Increasing max replicas and tuning CPU utilization target helps handle peak load and reduce latency.

Why this answer

Increasing the max replica count and setting a CPU utilization target of 60% allows the Vertex AI Prediction service to scale out more aggressively during traffic spikes, reducing both latency and 503 errors. This directly addresses the bottleneck of insufficient compute capacity under peak load, as the current max of 10 replicas may be too low to handle demand.

Exam trap

The trap here is that candidates often assume upgrading machine types (Option A) is the primary fix for latency, when in reality the issue is insufficient scaling capacity under peak load, not per-replica performance.

Full explanation →
462
Multi-Selectmedium

A data engineering team needs to ingest streaming data from an existing Kafka cluster (on-premises) into Google Cloud for real-time analytics. They want to minimize changes to the existing Kafka setup and avoid long-term operational overhead. Which TWO approaches should they consider?

Select 2 answers
A.Use Storage Transfer Service to copy Kafka logs from on-premises to GCS
B.Deploy a Kafka Connect cluster on Google Cloud with the Pub/Sub sink connector
C.Use Datastream to capture changes from Kafka
D.Replace the on-premises Kafka cluster with Google Cloud Pub/Sub
E.Set up a Dataproc cluster with Kafka and use MirrorMaker to replicate data to the cloud Kafka cluster
AnswersB, E

This allows streaming from on-prem Kafka to Pub/Sub without modifying the existing Kafka setup.

Why this answer

Using Kafka Connect with the Pub/Sub connector or setting up a Dataproc cluster running Kafka with mirroring are two ways to bridge on-premises Kafka to GCP with minimal changes.

Full explanation →
463
MCQhard

A data engineer needs to implement data lineage tracking for a BigQuery data warehouse. They want to automatically capture column-level lineage from ETL jobs run by Dataform and from manual SQL queries executed in the BigQuery console. Which approach meets these requirements?

A.Enable Dataplex Universal Catalog and use the Dataplex Lineage API to capture lineage from both sources
B.Use the BigQuery Lineage API provided by Data Catalog to register lineage manually
C.Use BigQuery's built-in column-level lineage, which automatically tracks lineage for all queries, and query it using INFORMATION_SCHEMA.JOBS
D.Export Dataform execution logs to Cloud Logging and use a custom script to extract lineage
AnswerC

BigQuery automatically captures column-level lineage for all SQL jobs, including Dataform and console queries.

Why this answer

BigQuery's column-level lineage is automatically captured for all queries (including Dataform and console) and can be queried via the INFORMATION_SCHEMA.JOBS_BY_USER views. Dataplex lineage requires integration but BigQuery lineage is native.

Full explanation →
464
MCQhard

A company uses BigQuery Omni to query data stored in AWS S3. They need to join this data with data in BigQuery (GCP). The dataset in AWS is large (10 TB) and frequently updated. Which approach minimizes data movement and cost?

A.Use BigQuery Omni cross-cloud join with the query processed in AWS region
B.Create a materialized view in BigQuery that includes the AWS data
C.Copy the AWS data into BigQuery storage using a scheduled transfer, then join
D.Use a federated query from BigQuery to read the AWS data directly and join in BigQuery
AnswerA

BigQuery Omni processes the join on the AWS side, pulling only the required rows from GCP, minimizing data movement.

Why this answer

BigQuery Omni supports cross-cloud joins by running the query on the cloud where the data resides, using the cross-cloud join feature to pull only the necessary rows from the other cloud.

Full explanation →
465
Multi-Selectmedium

A company is building a data pipeline that ingests streaming data from Pub/Sub, transforms it with Dataflow, and loads it into BigQuery. They want to handle malformed messages that cannot be parsed. Which TWO actions should they implement for error handling? (Choose 2)

Select 2 answers
A.Configure the pipeline to drop malformed messages silently
B.Use a side input to filter out malformed messages
C.Use a dead letter sink to write malformed messages to Cloud Storage or Pub/Sub for later analysis
D.Raise an exception in the DoFn to fail the pipeline immediately
E.Log the error and continue processing the next message
AnswersC, E

This allows reprocessing without blocking the main pipeline.

Why this answer

A dead letter sink (e.g., writing malformed messages to Cloud Storage or a separate Pub/Sub topic) allows the pipeline to continue processing valid data while preserving the problematic records for offline inspection, retries, or debugging. This pattern is a standard best practice in streaming pipelines to avoid data loss and enable recovery without blocking the main data flow.

Exam trap

Google Cloud often tests the misconception that raising an exception (Option D) is acceptable for error handling in streaming pipelines, but the correct approach is to isolate failures using a dead letter sink (Option C) while logging errors (Option E) to maintain pipeline continuity.

Full explanation →
466
MCQhard

A company serves multiple models using Vertex AI endpoints. Each model has different latency and memory requirements. To minimize cost, the company wants to share underlying compute resources among models. Which approach should they use?

A.Deploy each model as a separate Cloud Run service and use a load balancer.
B.Use a single GKE cluster with multiple deployments and use Istio for routing.
C.Deploy all models to a single Vertex AI endpoint and configure traffic splitting.
D.Create separate endpoints for each model and use a load balancer to route traffic.
AnswerC

Vertex AI endpoints allow deploying multiple models behind one endpoint, sharing resources.

Why this answer

Vertex AI endpoints support traffic splitting, allowing you to deploy multiple models behind a single endpoint and route a percentage of traffic to each model. This enables resource sharing and cost optimization because the underlying compute infrastructure is shared among the models, unlike separate endpoints which would each require dedicated resources.

Exam trap

Google Cloud often tests the misconception that separate endpoints or services are required for different models, when in fact Vertex AI endpoints support multi-model deployment with traffic splitting to share resources and minimize cost.

How to eliminate wrong answers

Option A is wrong because deploying each model as a separate Cloud Run service and using a load balancer does not share underlying compute resources; each service runs in its own container instance, leading to higher cost and no direct model-level traffic splitting. Option B is wrong because using a single GKE cluster with multiple deployments and Istio for routing is overly complex for Vertex AI model serving, and it bypasses the managed Vertex AI endpoint capabilities that natively support traffic splitting and resource sharing. Option D is wrong because creating separate endpoints for each model and using a load balancer defeats the purpose of sharing compute resources; each endpoint would have its own dedicated resources, increasing cost and management overhead.

Full explanation →
467
Multi-Selectmedium

A retail company wants to trigger a Cloud Run service whenever a new CSV file is uploaded to a specific Cloud Storage bucket. Which THREE components are needed to set up this event-driven architecture? (Choose 3)

Select 3 answers
A.Cloud Storage bucket with notifications enabled
B.Eventarc trigger
C.Cloud Dataflow pipeline
D.Cloud Run service
E.Cloud Pub/Sub topic
AnswersA, B, D

The source of events; bucket must be configured to send notifications.

Why this answer

Cloud Storage buckets must have notifications enabled to publish events to Pub/Sub when objects are created. Without enabling notifications, the bucket cannot emit events that trigger downstream services. This is typically done by configuring the bucket to send notifications to a Pub/Sub topic for each new object upload.

Exam trap

Google often tests the misconception that you must manually create a Pub/Sub topic as a separate component, when in fact Eventarc manages it automatically, making the Pub/Sub topic an implicit part of the Eventarc trigger rather than a distinct required component.

Full explanation →
468
MCQmedium

You need to inspect a BigQuery table for sensitive data such as credit card numbers and apply masking. Which GCP service should you use to identify and de-identify the data?

A.IAM Recommender
B.Cloud KMS
C.Dataplex
D.Cloud Data Loss Prevention (DLP)
AnswerD

Cloud DLP is designed for inspection and de-identification of sensitive data.

Why this answer

Cloud DLP can automatically inspect BigQuery tables for sensitive data types (credit card numbers, etc.) and apply de-identification transforms like masking, tokenization, etc.

Full explanation →
469
MCQmedium

A media company uses Cloud Data Loss Prevention (DLP) API to inspect and de-identify sensitive data before loading into BigQuery. They want to reduce costs by sampling the data during inspection. Which configuration should they use?

A.Use the 'ROWS' limit in the inspection job.
B.Set the sample method to 'RANDOM' with a percentage.
C.Use a hybrid inspection with a BigQuery sample table.
D.Use the 'BYTES_LIMIT' parameter.
AnswerB

DLP supports random sampling to inspect a subset of data, reducing cost.

Why this answer

The Cloud DLP API supports a 'sample_method' of 'RANDOM' with a 'sampling_percentage' to inspect only a random subset of rows. This directly reduces the volume of data scanned, lowering costs while still providing statistically representative coverage for sensitive data discovery.

Exam trap

The trap here is that candidates confuse 'limiting rows/bytes' (which scans sequentially from the start) with 'random sampling' (which distributes inspection across the entire dataset), leading them to pick options A or D, which do not achieve representative cost reduction.

How to eliminate wrong answers

Option A is wrong because the 'ROWS' limit in an inspection job caps the total number of rows scanned but does not sample randomly; it stops after scanning that many rows from the start, which can miss sensitive data in later rows and does not provide representative sampling. Option C is wrong because hybrid inspection with a BigQuery sample table requires manually creating and maintaining a separate table, adding complexity and storage costs, whereas the DLP API's built-in sampling is simpler and directly integrated. Option D is wrong because 'BYTES_LIMIT' limits the total bytes scanned but, like 'ROWS', scans sequentially from the beginning and does not perform random sampling, leading to biased results and potential cost inefficiency.

Full explanation →
470
MCQmedium

A company has a batch ETL job that runs daily using Cloud Dataflow. The job reads from Cloud Storage, transforms data, and writes to BigQuery. Recently, the job started failing with 'Resources have been exhausted' errors. What is the most likely cause?

A.The Cloud Storage bucket has been deleted.
B.The project has reached its Dataflow API quota.
C.The input data volume has increased significantly.
D.The BigQuery output table schema has changed.
AnswerB

Resource exhausted error indicates quota issue.

Why this answer

The 'Resources have been exhausted' error in Cloud Dataflow typically indicates that the project has reached its Dataflow API quota, such as the maximum number of concurrent jobs or API requests per minute. This is a common issue when multiple jobs run simultaneously or when the quota is set low by default. The error is distinct from resource exhaustion in the underlying compute or storage layers.

Exam trap

Google Cloud often tests the distinction between API quota exhaustion and resource exhaustion in the underlying infrastructure (e.g., Compute Engine CPU/memory), leading candidates to incorrectly attribute the error to increased data volume or schema changes.

How to eliminate wrong answers

Option A is wrong because deleting the Cloud Storage bucket would cause a 'bucket not found' or 'object not found' error, not a 'Resources have been exhausted' error. Option C is wrong because a significant increase in input data volume would lead to autoscaling limits or worker resource exhaustion (e.g., out of memory), but the specific 'Resources have been exhausted' message is tied to API quota limits, not data volume. Option D is wrong because a schema change in BigQuery would result in a schema mismatch or insertion error, not an API quota exhaustion error.

Full explanation →
471
MCQeasy

Your team uses Vertex AI Feature Store to serve features for online predictions. A feature value changes frequently (e.g., user session clicks). Which type of feature should you use to ensure low-latency writes and reads?

A.Streaming feature
B.Batch feature
C.Feature view
D.Bigtable-backed feature
AnswerA

Streaming features are designed for low-latency, high-frequency updates and reads.

Why this answer

A is correct because streaming features in Vertex AI Feature Store are designed for low-latency writes and reads, making them ideal for frequently changing values like user session clicks. They use an online serving infrastructure (typically backed by Bigtable) that supports real-time updates and sub-millisecond retrieval, ensuring predictions are based on the latest data without batch delays.

Exam trap

The trap here is that candidates confuse 'streaming feature' with 'Bigtable-backed feature' as if they are separate options, when in fact Bigtable is the underlying technology for streaming features, not a feature type itself.

How to eliminate wrong answers

Option B is wrong because batch features are optimized for bulk ingestion and offline serving, not for low-latency online writes and reads; they introduce latency due to periodic batch jobs. Option C is wrong because a feature view is a logical grouping of features for serving, not a type of feature with specific latency guarantees; it can reference either streaming or batch features. Option D is wrong because Bigtable-backed feature is not a distinct feature type in Vertex AI Feature Store; Bigtable is the underlying storage for streaming features, but the question asks for the feature type itself, not the storage backend.

Full explanation →
472
Multi-Selecthard

A data pipeline reads thousands of JSON files from Cloud Storage, processes them with Cloud Dataflow, and writes to BigQuery. The pipeline sometimes fails because of malformed JSON records. Which three steps should the data engineering team take to improve pipeline reliability? (Choose THREE.)

Select 3 answers
A.Integrate Cloud Pub/Sub as an intermediary to buffer and allow message retry
B.Use a try-catch block in the pipeline to retry processing failed records
C.Create a Cloud Monitoring alert on pipeline failures
D.Add schema validation before processing to reject invalid JSON records
E.Implement a dead-letter queue in the Dataflow pipeline to store failed records for later analysis
AnswersA, D, E

Pub/Sub can retry delivery of messages, improving reliability.

Why this answer

Integrating Cloud Pub/Sub as an intermediary decouples the ingestion of JSON files from the Dataflow pipeline. Pub/Sub provides at-least-once delivery and automatic retries for messages that are not acknowledged, which buffers against transient failures and malformed records. This allows the pipeline to pull messages at its own pace and retry processing without losing data.

Exam trap

The trap here is that candidates often confuse reactive monitoring (Option C) with proactive reliability improvements, or they assume a simple try-catch block (Option B) is sufficient in a distributed processing framework like Dataflow, where fault tolerance requires persistent retry mechanisms and dead-letter queues.

Full explanation →
473
MCQeasy

A team needs to migrate an existing on-premises Hadoop Hive workload to Google Cloud. They want to minimize code changes and use a managed service for transient clusters. Which service should they choose?

A.Cloud Dataflow
B.Cloud Dataprep
C.Cloud Dataproc
D.BigQuery
AnswerC

Dataproc is fully compatible with Hadoop/Hive and offers ephemeral clusters with minimal code changes.

Why this answer

Cloud Dataproc is the correct choice because it is a managed Spark and Hadoop service that supports Hive workloads natively, allowing you to run existing Hive scripts with minimal changes. It also supports transient clusters, which can be automatically scaled up and down, aligning with the requirement for transient clusters.

Exam trap

The trap here is that candidates often confuse Cloud Dataflow's ability to process batch data with Hadoop compatibility, but Dataflow does not support Hive or transient Hadoop clusters, making Dataproc the only correct option for minimizing code changes.

How to eliminate wrong answers

Option A is wrong because Cloud Dataflow is a unified stream and batch data processing service based on Apache Beam, not designed for Hive workloads or transient Hadoop clusters. Option B is wrong because Cloud Dataprep is a data preparation and cleaning service (based on Trifacta) that does not run Hive or provide transient clusters. Option D is wrong because BigQuery is a serverless data warehouse that does not support Hive execution engines or transient clusters; migrating Hive to BigQuery would require significant code changes.

Full explanation →
474
Multi-Selectmedium

A data engineer is monitoring a Dataflow streaming pipeline and notices that the 'System Lag' metric is increasing. Which TWO actions should be taken to diagnose the issue?

Select 2 answers
A.Check the Dataflow monitoring UI for each stage's throughput and backlog.
B.Cancel the pipeline and restart with a larger initial worker count.
C.Increase the maximum number of workers to handle backlog.
D.Examine the worker logs for error messages or stack traces.
E.Increase the BigQuery quota for streaming inserts.
AnswersA, D

Identifies bottleneck stages.

Why this answer

The Dataflow monitoring UI provides per-stage metrics such as throughput and backlog, which directly indicate where data is accumulating. By examining these metrics, you can identify the specific stage causing the increasing system lag, enabling targeted troubleshooting without unnecessary pipeline changes.

Exam trap

Google Cloud often tests the distinction between diagnostic actions and remedial actions; the trap here is that candidates confuse scaling up workers (a fix) with diagnosing the root cause of the lag.

Full explanation →
475
MCQhard

A Dataflow streaming job is processing high-volume sensor data from thousands of IoT devices. The job uses global windows with a 10-minute processing time trigger. Recently, the job's CPU utilization is nearly 100% and it is falling behind. Which action is most likely to reduce CPU load while maintaining data freshness?

A.Increase the number of workers to distribute the load.
B.Change the trigger to event time with a 10-minute allowed lateness.
C.Replace GroupByKey with Combine.globally and use a fanout.
D.Use side inputs to broadcast a static lookup table to all workers.
AnswerC

Combine.globally with fanout reduces the number of unique keys tracked per worker, lowering CPU usage from grouping large numbers of keys.

Why this answer

Using `Combine.globally` with a fanout reduces the amount of data shuffled and merged in a single worker, lowering CPU load. In Dataflow, `GroupByKey` triggers a full shuffle and per-key aggregation, which is expensive for high-volume sensor data; `Combine.globally` with a fanout performs partial aggregation on each worker before a final merge, reducing network I/O and CPU cycles. This maintains data freshness because the 10-minute processing time trigger still fires on time, but with less per-element overhead.

Exam trap

Google Cloud often tests the misconception that scaling out workers (Option A) is the universal fix for performance issues, but the trap here is that the real bottleneck is the shuffle-heavy `GroupByKey` operation, not worker count.

How to eliminate wrong answers

Option A is wrong because increasing the number of workers distributes load but does not address the root cause—the high CPU cost of per-key grouping and shuffling in `GroupByKey`; it may temporarily reduce backlog but adds cost and can still hit scaling limits. Option B is wrong because changing to event time with allowed lateness does not reduce CPU utilization; it only changes watermark semantics and may increase state size, worsening CPU pressure. Option D is wrong because using side inputs to broadcast a static lookup table does not reduce the CPU cost of the aggregation step; it adds memory overhead and does not address the shuffle bottleneck.

Full explanation →
476
MCQmedium

You deployed a model on Vertex AI Endpoints using a custom container. The model serves predictions but the latency is higher than expected. You suspect the container is not making full use of the CPU resources. What should you do to reduce latency?

A.Modify the container to use multi-threading or increase the number of workers in the prediction server (e.g., Gunicorn workers).
B.Enable response caching on the endpoint.
C.Change the machine type to a GPU-accelerated machine.
D.Increase the number of nodes by adjusting autoscaling limits.
AnswerA

Properly configuring concurrency allows each node to process multiple requests in parallel, reducing latency under load.

Why this answer

High latency in a CPU-based custom container often stems from underutilizing available CPU cores. By increasing the number of workers (e.g., Gunicorn workers) or enabling multi-threading, you allow the prediction server to handle multiple requests concurrently, reducing queue time and improving throughput. This directly addresses the symptom of the container not making full use of CPU resources.

Exam trap

Google Cloud often tests the misconception that scaling out (adding more nodes) or upgrading hardware (GPU) is the default fix for latency, when the real issue is often software-level concurrency configuration within the container.

How to eliminate wrong answers

Option B is wrong because response caching reduces latency only for repeated identical requests, not for the general case of underutilized CPU resources; it does not improve concurrent request handling. Option C is wrong because switching to a GPU-accelerated machine would only help if the model benefits from GPU parallelism (e.g., deep learning models), but the question states the container is not making full use of CPU resources, implying the bottleneck is software configuration, not hardware type. Option D is wrong because increasing the number of nodes via autoscaling adds more instances but does not fix the per-instance CPU underutilization; it may even increase cost without addressing the root cause of inefficient request handling within each container.

Full explanation →
477
MCQmedium

A company uses Cloud Dataproc to run Spark ML training jobs. They want to persist the trained models and metadata in a Hive-compatible metastore. Which Dataproc feature should they use?

A.Cloud Hive Metastore (self-managed)
B.Cloud Bigtable
C.Dataproc Metastore
D.Cloud Data Catalog
AnswerC

Dataproc Metastore is a fully managed, Hive-compatible metastore service.

Why this answer

Dataproc Metastore provides a Hive-compatible metastore that can be used across clusters and services.

Full explanation →
478
MCQmedium

A company is using Cloud Storage to store raw logs. They want to use Cloud Data Fusion to transform and load the data into BigQuery on a daily schedule. The transformations are complex and involve joining multiple datasets. What is the most efficient way to run these pipelines?

A.Use Cloud Composer to orchestrate Dataproc jobs that run the transformations
B.Use Cloud Functions to trigger a Dataflow job that does the transformations
C.Use Cloud Data Fusion to design the pipeline and schedule it to run on a Dataproc cluster
D.Use Cloud Dataprep to design the transformation and export to BigQuery
AnswerC

Cloud Data Fusion orchestrates the execution on Dataproc, which is the expected approach.

Why this answer

Cloud Data Fusion supports scheduling pipelines and runs them on Dataproc. This is the standard way to run batch pipelines.

Full explanation →
479
MCQhard

You are a data engineer at a financial services company. You have deployed a credit risk model on Vertex AI Endpoints using a custom container with a TensorFlow SavedModel. The model expects input features as a JSON object. Recently, the model has been returning high prediction latency and occasional 503 errors. You have enabled autoscaling with minNodes=2 and maxNodes=10. The model is CPU-only and uses n1-standard-4 machines. Monitoring shows that during peak hours, CPU utilization reaches 90% and memory is at 80%. The number of prediction requests per second peaks at 100. You suspect that the model is not scaling fast enough. Which action will most effectively reduce latency and eliminate 503 errors?

A.Increase maxNodes to 20 to allow more replicas during peak
B.Change the machine type to n1-standard-4 with a GPU (e.g., NVIDIA T4) and update the custom container to use GPU
C.Set minNodes to 5 to keep more replicas warm
D.Switch to n1-highmem-4 machines to provide more memory per node
AnswerB

GPU acceleration reduces per-request latency and can handle more requests per node.

Why this answer

The high CPU utilization (90%) indicates that the model's inference is compute-bound. Offloading the computation to a GPU (NVIDIA T4) significantly accelerates TensorFlow model inference, reducing per-request latency and allowing each replica to handle more requests per second. This directly addresses the root cause of the 503 errors (requests timing out due to slow inference) and reduces the need for rapid scaling.

Exam trap

Google Cloud often tests the misconception that scaling out (increasing replicas) is always the solution to latency and 503 errors, when in fact the root cause may be per-replica performance (CPU vs. GPU) that scaling cannot fix.

How to eliminate wrong answers

Option A is wrong because increasing maxNodes to 20 does not address the fundamental bottleneck: each replica is CPU-bound at 90% utilization. More replicas would still be slow and may not scale quickly enough to handle sudden spikes, and they would increase cost without fixing latency. Option C is wrong because setting minNodes to 5 keeps more replicas warm but does not reduce the latency of each individual prediction; the replicas would still be CPU-bound, so 503 errors from slow inference would persist.

Option D is wrong because memory is only at 80%, not a bottleneck; switching to n1-highmem-4 provides more memory but does not accelerate the CPU-bound computation, so latency and 503 errors would remain.

Full explanation →
480
Multi-Selecthard

A payment processing company needs to detect fraudulent transactions in real time. The system must have sub-second latency for high-value transactions and use a machine learning model. Which two components should be part of the architecture? (Choose TWO.)

Select 2 answers
A.Cloud Storage for transaction logs
B.Bigtable to store user profiles and transaction history for fast lookups
C.Dataflow for stream processing with sliding windows
D.Cloud SQL to store reference data
E.Cloud Functions for long-running batch model training
AnswersB, C

Bigtable offers sub-millisecond latency for point lookups, essential for real-time fraud scoring.

Why this answer

Bigtable is a fully managed, scalable NoSQL database that provides consistent sub-10ms latency for high-throughput read/write operations, making it ideal for real-time lookups of user profiles and transaction history in fraud detection. Its ability to handle large volumes of data with low latency supports the sub-second requirement for high-value transactions.

Exam trap

Google Cloud often tests the distinction between storage services optimized for real-time access (Bigtable) versus batch/archive (Cloud Storage) and between stream processing (Dataflow) versus batch processing or short-lived compute (Cloud Functions).

Full explanation →
481
MCQeasy

A startup wants to build a data lake on Google Cloud using Cloud Storage. They need to store raw data in its original format for future analysis. Which storage class should they use to optimize for cost given that data will be accessed occasionally after the first month?

A.Nearline storage class
B.Coldline storage class
C.Standard storage class
D.Archive storage class
AnswerA

Optimized for data accessed less than once a month, cost-effective.

Why this answer

Nearline storage class is the optimal choice because it offers low-cost storage for data accessed less than once a month, with a 30-day minimum storage duration. Since the data is accessed occasionally after the first month, Nearline provides significant cost savings over Standard while still offering low-latency access (milliseconds) suitable for analytics. Coldline and Archive have lower storage costs but impose higher retrieval fees and minimum storage durations (90 and 365 days respectively), making them more expensive for data that is accessed occasionally within the first year.

Exam trap

Google Cloud often tests the misconception that lower storage cost always means lower total cost, ignoring the impact of retrieval fees and minimum storage duration penalties, which can make Coldline or Archive more expensive for data accessed occasionally within the first year.

How to eliminate wrong answers

Option B (Coldline) is wrong because it is designed for data accessed less than once a quarter (90-day minimum storage duration) and has higher retrieval costs, making it more expensive than Nearline for data accessed occasionally after the first month. Option C (Standard) is wrong because it is optimized for frequently accessed data (no minimum storage duration) and has the highest storage cost, which is not cost-effective for data that is only accessed occasionally. Option D (Archive) is wrong because it is intended for long-term archival data accessed less than once a year (365-day minimum storage duration) and has very high retrieval costs and latency (hours), making it unsuitable for occasional access within a year.

Full explanation →
482
MCQmedium

A data engineer is designing a batch data pipeline that reads Avro files from Cloud Storage, transforms data using Apache Beam, and writes to BigQuery. The pipeline must handle daily runs and backfills. Which runner should they use?

A.FlinkRunner
B.DataflowRunner
C.SparkRunner
D.DirectRunner
AnswerB

DataflowRunner is a fully managed service that supports batch pipelines, backfills, and direct integration with GCS and BigQuery.

Why this answer

DataflowRunner is the correct choice because it is the fully managed service runner for Apache Beam on Google Cloud, optimized for batch and streaming pipelines. It automatically handles scaling, resource management, and exactly-once processing semantics, which are essential for reliable daily runs and backfills with Avro files from Cloud Storage and BigQuery sinks.

Exam trap

The trap here is that candidates may confuse the runner with the execution engine, assuming that any distributed runner (Flink, Spark) is suitable for production, when the question specifically tests knowledge of Google Cloud-native services and the need for managed infrastructure for batch pipelines with backfills.

How to eliminate wrong answers

Option A is wrong because FlinkRunner is designed for running Beam pipelines on Apache Flink clusters, which require manual cluster management and are not natively integrated with Google Cloud services like Cloud Storage and BigQuery. Option C is wrong because SparkRunner runs Beam pipelines on Apache Spark, which is not a managed service on Google Cloud and lacks the seamless integration with Cloud Storage and BigQuery that DataflowRunner provides. Option D is wrong because DirectRunner is intended for local testing and development only, not for production workloads or handling large-scale daily runs and backfills.

Full explanation →
483
Multi-Selectmedium

Which TWO are best practices for managing a Cloud Dataflow pipeline in production?

Select 2 answers
A.Always use batch mode for streaming data to reduce cost
B.Disable autoscaling to keep compute costs predictable
C.Set up Cloud Monitoring alerts based on Dataflow job metrics
D.Use pipeline updates (update) to modify running streaming pipelines
E.Restart the pipeline when code changes are needed
AnswersC, D

Alerts help detect issues proactively.

Why this answer

Cloud Monitoring alerts on Dataflow job metrics (e.g., system lag, watermark delay, or element count) enable proactive detection of pipeline health issues such as backpressure or stuck workers. This is a best practice for production pipelines to ensure reliability and timely intervention.

Exam trap

Google Cloud often tests the misconception that disabling autoscaling or restarting pipelines is acceptable for cost control or simplicity, when in fact these actions violate production best practices for reliability and data integrity.

Full explanation →
484
MCQhard

A financial services company uses Cloud Pub/Sub with ordering keys to process transactions in order. Some messages are failing processing and getting stuck. The team wants to ensure that if a message fails, it can be reprocessed later without blocking subsequent messages. What should they implement?

A.Create multiple subscriptions for the same topic
B.Use a pull subscription with flow control settings
C.Configure a dead letter topic and handle the failed message separately
D.Increase the acknowledgment deadline to 600 seconds
AnswerC

Dead letter topics isolate failures, allowing forwarding of messages for later reprocessing.

Why this answer

A dead letter topic (DLT) allows failed messages to be moved aside after exhausting retry attempts, so they do not block the processing of subsequent ordered messages. In Cloud Pub/Sub, ordering keys require messages with the same key to be delivered in order; if a message fails and is not acknowledged, it blocks all later messages with the same key. By configuring a dead letter topic, the failed message is automatically forwarded to the DLT after a maximum of 5 delivery attempts (default), and the original subscription can continue processing the next messages in order.

The team can then reprocess the failed message from the DLT separately, without affecting the order of other messages.

Exam trap

Google Cloud often tests the misconception that increasing the acknowledgment deadline or adding flow control can resolve stuck messages with ordering keys, but the real solution is to use a dead letter topic to offload the failing message and unblock the ordered stream.

How to eliminate wrong answers

Option A is wrong because creating multiple subscriptions for the same topic does not solve the blocking issue; each subscription independently receives all messages, but within a single subscription, ordering keys still cause a failed message to block subsequent messages with the same key. Option B is wrong because pull subscriptions with flow control settings only limit the rate of message delivery and do not handle failed messages that are stuck; they do not provide a mechanism to move failed messages out of the way to unblock ordering. Option D is wrong because increasing the acknowledgment deadline to 600 seconds only gives the subscriber more time to process a message before it is redelivered, but it does not prevent a persistently failing message from blocking subsequent ordered messages indefinitely.

Full explanation →
485
MCQeasy

A data engineer needs to process data in a Dataflow pipeline that reads from a Pub/Sub topic. The pipeline must group events into 5-minute windows and compute the average value per key. Which Beam transform should they use after windowing?

A.Combine.perKey
B.ParDo
C.GroupByKey
D.CoGroupByKey
AnswerA

Combine.perKey applies a combining function (e.g., average) per key.

Why this answer

Combine.perKey with an averaging function computes per-key average. GroupByKey groups by key but requires manual combination. ParDo is for element-wise processing.

CoGroupByKey joins multiple PCollections.

Full explanation →
486
MCQhard

An organization uses Cloud Dataproc to run Spark jobs that process sensitive data. They need to ensure data is encrypted at rest and that only specific service accounts can access the data on cluster disks. What should they do?

A.Rely on the default encryption at rest and use VPC Service Controls to limit data exfiltration.
B.Use customer-supplied encryption keys (CSEK) and write a startup script to mount encrypted disks.
C.Enable encryption at rest using Google-managed encryption keys and grant all users the Dataproc Editor role.
D.Use customer-managed encryption keys (CMEK) for the cluster's persistent disks and assign a dedicated service account to the cluster with minimal IAM roles.
AnswerD

CMEK provides control over keys, and a dedicated service account restricts data access.

Why this answer

Using customer-managed encryption keys (CMEK) allows the organization to control and manage the encryption keys for persistent disks attached to the Dataproc cluster, ensuring data at rest is encrypted. Assigning a dedicated service account with minimal IAM roles ensures that only that service account can access the data on the cluster disks, following the principle of least privilege.

Exam trap

The trap here is that candidates often confuse CSEK (used for Cloud Storage) with CMEK (used for persistent disks), or assume that default encryption combined with VPC Service Controls is sufficient for granular access control to disk data.

How to eliminate wrong answers

Option A is wrong because default encryption at rest uses Google-managed keys, which does not allow the organization to control key access or restrict which service accounts can access data on cluster disks; VPC Service Controls prevent data exfiltration but do not enforce service-account-level access to disk data. Option B is wrong because customer-supplied encryption keys (CSEK) are used for encrypting data in Cloud Storage, not for persistent disks on Dataproc; mounting encrypted disks via a startup script is not a supported or recommended method for Dataproc clusters. Option C is wrong because granting all users the Dataproc Editor role would allow any user to access and modify cluster resources, violating the requirement that only specific service accounts can access data on cluster disks.

Full explanation →
487
Multi-Selecteasy

A company is deploying a machine learning model for fraud detection. The model is trained using TensorFlow and will be served on Vertex AI Prediction. The team wants to implement model monitoring to detect prediction drift. Which TWO actions should they take? (Choose 2)

Select 2 answers
A.Configure Vertex AI Model Monitoring to compare online prediction inputs against training data statistics.
B.Collect ground truth labels for all predictions to measure accuracy drift.
C.Set up a separate Cloud Monitoring alerting policy to watch for prediction errors.
D.Enable automatic model retraining in Vertex AI Model Monitoring when drift is detected.
E.Enable prediction drift monitoring to detect changes in model output distribution.
AnswersA, E

This detects feature drift, which is a common monitoring need.

Why this answer

Vertex AI Model Monitoring can be configured to compare online prediction inputs against training data statistics to detect skew, which is a form of drift. This is a standard capability of Vertex AI Model Monitoring, where you specify a baseline dataset (typically training data) and the service automatically computes statistics on incoming prediction requests to identify distribution shifts.

Exam trap

Google Cloud often tests the distinction between monitoring for drift (which focuses on input/output distributions) versus monitoring for model accuracy (which requires ground truth labels), and candidates mistakenly think collecting ground truth is a prerequisite for drift detection.

Full explanation →
488
MCQmedium

A company uses Dataflow streaming pipelines to process real-time events. They notice increasing system lag over time. Which two Cloud Monitoring metrics should be examined to diagnose the cause?

A.Pub/Sub subscription/num_undelivered_messages and Dataflow job/watermark_lag
B.Dataproc cluster/yarn_allocated_memory_percentage and Dataflow job/worker_cpu
C.Dataflow job/system_lag and Dataflow job/data_freshness
D.BigQuery query/execution_times and Dataflow job/elapsed_time
AnswerC

system_lag indicates processing delay; data_freshness shows watermark progress. Both are key for streaming lag.

Why this answer

System lag measures the time between event ingestion and processing. Data freshness shows the watermark. Worker CPU indicates compute resource issues.

Full explanation →
489
Multi-Selecthard

A global fintech company needs a database that can serve transactional (OLTP) and analytical (OLAP) workloads with strong consistency. They require high availability and PostgreSQL compatibility. Which TWO Google Cloud databases meet these requirements? (Choose 2 correct options)

Select 2 answers
A.Cloud SQL
B.Cloud Bigtable
C.Cloud Spanner
D.BigQuery
E.AlloyDB
AnswersC, E

Strong consistency, globally distributed, OLTP+analytics via SQL, PostgreSQL interface available.

Why this answer

Cloud Spanner is correct because it provides a globally distributed, strongly consistent relational database service that supports both OLTP and OLAP workloads via PostgreSQL-compatible interfaces (including the PostgreSQL dialect). It offers high availability through synchronous replication across zones and regions, and its TrueTime-based atomic clocks ensure external consistency for transactions. AlloyDB is also correct because it is a fully PostgreSQL-compatible database service that delivers high availability and strong consistency.

It is designed to handle both transactional and analytical workloads using a combined row-store and columnar engine, enabling high-performance OLTP and real-time analytics within a single database. Both services meet the fintech company's requirements for PostgreSQL compatibility, strong consistency, and high availability.

Exam trap

A common mistake is to think that Cloud SQL or BigQuery can serve both OLTP and OLAP workloads with strong consistency, but Cloud SQL lacks global scaling and OLAP performance, and BigQuery is purely analytical without transactional support.

Full explanation →
490
MCQeasy

Which Google Cloud service provides a visual interface for building ETL pipelines using a drag-and-drop design and includes pre-built transforms from a marketplace?

A.Dataproc
B.Cloud Data Fusion
C.Dataprep
D.BigQuery
AnswerB

Cloud Data Fusion is a visual ETL tool with CDAP plugins and a Hub for marketplace transforms.

Why this answer

Cloud Data Fusion offers a visual, code-free ETL tool with a rich set of plugins from the Hub. Dataprep is for data wrangling, not full ETL.

Full explanation →
491
MCQmedium

A retail company processes real-time clickstream data using Cloud Pub/Sub and Dataflow. The pipeline aggregates events by user session and writes to Bigtable for low-latency queries. However, users report that session data is sometimes missing or duplicated. What is the most likely cause?

A.Session windowing is configured with too short a gap duration.
B.Bigtable schema design causes row key collisions.
C.Dataflow's default behavior discards late-arriving data.
D.Pub/Sub provides at-least-once delivery, and Dataflow does not deduplicate by default.
AnswerD

At-least-once delivery leads to duplicates without dedup in pipeline.

Why this answer

D is correct because Pub/Sub offers at-least-once delivery, meaning the same message may be delivered multiple times. Dataflow does not automatically deduplicate messages unless explicitly configured (e.g., using idempotent sinks or custom deduplication logic). Without deduplication, the same session event can be processed more than once, leading to duplicate session data in Bigtable.

Exam trap

Google Cloud often tests the misconception that Pub/Sub provides exactly-once delivery or that Dataflow automatically deduplicates messages from Pub/Sub, when in fact Pub/Sub is at-least-once and Dataflow requires explicit deduplication for idempotent processing.

How to eliminate wrong answers

Option A is wrong because a short gap duration would cause sessions to be split prematurely, leading to missing data (events not grouped into the same session), not duplicates. Option B is wrong because row key collisions in Bigtable would cause overwrites or errors, not missing or duplicate session data; Bigtable uses lexicographic ordering and row keys are unique per write. Option C is wrong because Dataflow's default behavior for late-arriving data depends on the windowing strategy; with session windows, late data can be included if within the allowed lateness, and Dataflow does not discard late data by default—it uses a default allowed lateness of 0 seconds, which would cause late data to be dropped, but this would result in missing data, not duplicates.

Full explanation →
492
MCQeasy

A data engineer wants to store archived log files in Cloud Storage with a retention policy that prevents deletion for 5 years. Which feature should they use?

A.Object Lifecycle rule with Delete action after 5 years
B.Retention Policy on the bucket set to 5 years
C.Object Hold (temporal)
D.Versioning enabled
AnswerB

Retention policies ensure objects cannot be deleted or replaced until the retention period expires.

Why this answer

A retention policy on a Cloud Storage bucket enforces a minimum retention period for all objects in the bucket, preventing deletion or overwrite until the policy duration has elapsed. Setting it to 5 years ensures that archived log files cannot be deleted before that time, meeting the data engineer's requirement exactly. This is a bucket-level, immutable setting that applies to all objects, unlike object-level holds or lifecycle rules.

Exam trap

Google often tests the distinction between lifecycle rules that delete objects and retention policies that prevent deletion, so the trap here is assuming that a lifecycle rule with a Delete action can enforce a retention period, when in fact it does the opposite.

How to eliminate wrong answers

Option A is wrong because an Object Lifecycle rule with a Delete action after 5 years would automatically delete objects after 5 years, which is the opposite of preventing deletion; it does not enforce a retention period. Option C is wrong because an Object Hold (temporal) is a temporary hold placed on individual objects for a specific duration (e.g., days), not a bucket-wide policy for 5 years, and it is typically used for legal or compliance holds, not long-term retention. Option D is wrong because Versioning enabled preserves previous versions of objects but does not prevent deletion of the current version; it allows recovery after deletion but does not block deletion itself, so it does not enforce a retention policy.

Full explanation →
493
Multi-Selecteasy

Which TWO are benefits of using Vertex AI Endpoints for model serving?

Select 2 answers
A.Batch prediction support out of the box.
B.Integrated monitoring for prediction latency and error rates.
C.Automatic scaling based on traffic.
D.Automatic model retraining when drift is detected.
E.Built-in support for A/B testing without any additional configuration.
AnswersB, C

Vertex AI endpoints integrate with Cloud Monitoring for operational metrics.

Why this answer

Vertex AI Endpoints provide integrated monitoring for prediction latency and error rates out of the box, enabling you to track model performance and detect anomalies without additional instrumentation. This is a core operational feature that helps maintain service-level objectives (SLOs) and quickly identify degradation in production.

Exam trap

Google Cloud often tests the distinction between features that are 'built-in' versus those that require separate services or additional configuration, so candidates mistakenly assume batch prediction or automatic retraining are part of Endpoints when they are actually separate Vertex AI components.

Full explanation →
494
Multi-Selecthard

A team is deploying a complex model with multiple preprocessing steps. They want to ensure consistent preprocessing during training and serving. Which three approaches can achieve this? (Select 3)

Select 3 answers
A.Store preprocessing logic in a shared Python module
B.Use a separate preprocessing service called from the model
C.Use two separate pipelines for training and serving
D.Use Vertex AI Feature Transform Engine
E.Embed preprocessing logic in the model graph
AnswersA, D, E

A shared module ensures the same code is used in training and serving if properly versioned.

Why this answer

Storing preprocessing logic in a shared Python module ensures that the same code is used during both training and serving, eliminating drift between environments. This approach leverages version control and dependency management to guarantee consistency, which is critical for reproducibility in production ML pipelines.

Exam trap

Google Cloud often tests the misconception that a separate preprocessing service (Option B) is a good architectural pattern for consistency, when in fact it introduces a single point of failure and versioning complexity that undermines the goal of identical preprocessing.

Full explanation →
495
Multi-Selectmedium

You need to ingest streaming data from a custom application into BigQuery with exactly-once semantics and low latency. The data volume is up to 10 MB/s. Which TWO services should you combine?

Select 2 answers
A.Pub/Sub
B.Cloud Functions
C.BigQuery legacy streaming inserts
D.Dataflow with Storage Write API
E.Datastream
AnswersA, D

Pub/Sub is the recommended message ingestion service for streaming data.

Why this answer

Pub/Sub provides reliable, low-latency message ingestion, and Dataflow can read from Pub/Sub and write to BigQuery using the Storage Write API, which supports exactly-once semantics. The Storage Write API with committed mode ensures exactly-once delivery.

Full explanation →
496
Multi-Selectmedium

A data engineer is designing a BigQuery table for time-series data that will be queried frequently by time range and also by a customer_id. Which TWO design decisions will improve query performance and manage costs? (Choose two.)

Select 2 answers
A.Partition the table by day on the timestamp column
B.Cluster the table on customer_id
C.Disable automatic reclustering to save costs
D.Set partition expiration to 1 year
E.Use nested repeated fields for customer data
AnswersA, B

Enables partition pruning for time-range queries.

Why this answer

Partitioning the table by day on the timestamp column allows BigQuery to prune partitions when queries filter by a time range, scanning only the relevant partitions instead of the entire table. This directly reduces the amount of data read, improving query performance and lowering costs.

Exam trap

Google Cloud often tests the misconception that disabling automatic reclustering saves costs, but in reality it is free and essential for maintaining clustering benefits, while partition expiration is a lifecycle management feature, not a performance optimization.

Full explanation →
497
MCQhard

You are designing a Dataflow pipeline that reads from Pub/Sub, aggregates events into 10-minute windows, and writes the results to BigQuery. The pipeline must reliably handle late-arriving data (up to 1 hour) and prevent duplicate aggregations. Which combination of pipeline options should you use?

A.Use exactly-once processing by setting the pipeline's streaming engine to exactly-once and using a BigQuery sink with exactly-once semantics
B.Use at-least-once processing and rely on BigQuery's automatic deduplication
C.Use exactly-once processing and write results to a staging table, then use a scheduled merge query to combine with the main table
D.Use at-most-once processing to guarantee no duplicates, and accept data loss
AnswerA

Dataflow's exactly-once sink for BigQuery ensures no duplicates even with late data, using a combination of idempotent writes and deduplication.

Why this answer

To prevent duplicate aggregations, you need exactly-once processing. Dataflow supports exactly-once sinks (like BigQuery) when using the FILE_LOADS method or streaming inserts with exactly-once semantics. Using at-least-once with deduplication in BigQuery is not reliable.

Exactly-once semantics require idempotent writes; the recommended approach is to use the BigQuery sink with exactly-once support (by setting the trigger frequency appropriately and using a dedup key). However, the simplest way is to use the Dataflow streaming engine and the BigQuery sink with exactly-once enabled. Among the options, using the BigQuery streaming inserts with exactly-once semantics (available in Dataflow) is correct.

Full explanation →
498
MCQeasy

A data engineer needs to load 10 GB of CSV files from Amazon S3 into BigQuery on a daily basis. The files arrive in a specific S3 bucket at 3 AM UTC each day. Which service should be used to automate this transfer?

A.Cloud Storage Transfer Service
B.Dataflow with Pub/Sub
C.Transfer Appliance
D.BigQuery Data Transfer Service
AnswerD

BigQuery Data Transfer Service can schedule and automate data loads from Amazon S3 directly into BigQuery.

Why this answer

BigQuery Data Transfer Service supports scheduled transfers from Amazon S3 directly to BigQuery, making it the appropriate choice for this recurring batch load.

Full explanation →
499
Multi-Selecthard

You are building a time-series forecasting model with BigQuery ML. Which three steps should you perform to properly split the data and evaluate the model? (Choose THREE)

Select 3 answers
A.Evaluate on a holdout set that is later in time than the training set.
B.Use time-series cross-validation with expanding windows.
C.Split the data randomly into training and testing sets.
D.Use a chronological split based on a cutoff date.
E.Use k-fold cross-validation with random folds.
AnswersA, B, D

Testing on future data simulates real-world forecasting.

Why this answer

For time-series, you must maintain temporal order: split chronologically (not randomly), use a cutoff date for training/validation, and evaluate on unseen future data. Cross-validation should be time-series aware (e.g., expanding window). Random split is invalid for time-series.

Using a single train/test split may be insufficient; multiple windows are better.

Full explanation →
500
MCQeasy

You need to preprocess tabular data for training a classification model using Vertex AI. The dataset has missing values in numerical columns and categorical columns with high cardinality. Which Vertex AI service provides automated feature engineering and preprocessing as part of the pipeline?

A.Vertex AI Pipelines
B.AutoML Tables
C.Vertex AI Feature Store
D.Vertex AI Workbench
AnswerA

Vertex AI Pipelines orchestrates preprocessing steps such as imputation and encoding as part of an ML pipeline.

Why this answer

Vertex AI Pipelines allows you to build ML pipelines with components for feature engineering, including handling missing values and encoding. Vertex AI Feature Store is for serving features, not preprocessing.

Full explanation →
501
MCQhard

A company has a batch prediction job that runs daily using AI Platform Batch Prediction. The job uses a TensorFlow model and processes 10 GB of data. Recently, the job started failing with the error 'The replica worker 0 exited with a non-zero exit code: Out of memory'. Which action should the team take to resolve this without rewriting the model?

A.Increase the number of workers (parallelism) to distribute the data across more machines.
B.Use a machine type with more memory, such as n1-highmem-8.
C.Reduce the batch size parameter in the prediction job configuration.
D.Optimize the model to use less memory by pruning or quantization.
AnswerB

Directly addresses the out-of-memory error by providing more RAM per worker.

Why this answer

The error 'Out of memory' on replica worker 0 indicates that the machine type assigned to the prediction job does not have enough RAM to load the model and process the 10 GB batch. Increasing the machine type to one with more memory (e.g., n1-highmem-8) directly addresses the memory constraint without requiring any code changes. This is the most straightforward fix because AI Platform Batch Prediction allows you to specify machine types in the job configuration, and the error is purely a resource allocation issue.

Exam trap

Google Cloud often tests the distinction between scaling horizontally (adding workers) and scaling vertically (increasing machine resources), where candidates mistakenly assume parallelism solves memory issues, but the error is per-worker memory exhaustion, not throughput.

How to eliminate wrong answers

Option A is wrong because increasing the number of workers (parallelism) distributes the data across more machines but does not increase the memory per worker; each replica still has the same limited memory, so the out-of-memory error would persist on each worker. Option C is wrong because reducing the batch size parameter controls how many predictions are processed per step, which can reduce peak memory usage per request, but the error occurs during model loading or initial data processing, not during per-step prediction; the 10 GB dataset and model size still require sufficient base memory. Option D is wrong because while pruning or quantization could reduce model memory footprint, the question explicitly states 'without rewriting the model,' and these techniques require modifying the model architecture or retraining, which is a form of rewriting.

Full explanation →
502
MCQeasy

A company needs to process large files (100GB each) from Cloud Storage using Dataproc. They want to minimize job execution time. Which configuration is most appropriate?

A.Use a single-node cluster
B.Use a cluster with preemptible worker nodes and high-CPU machine types
C.Use HDFS for input data to avoid network latency
D.Use a cluster with many standard worker nodes
AnswerB

Preemptible VMs reduce cost, high-CPU machines improve speed.

Why this answer

Preemptible worker nodes are significantly cheaper than standard nodes, allowing you to scale out the cluster with many more workers for the same cost, which directly reduces job execution time for embarrassingly parallel data processing tasks. High-CPU machine types are ideal for compute-intensive Dataproc jobs like data transformation or machine learning, as they provide more vCPUs per core for parallel processing. This combination maximizes parallelism and minimizes wall-clock time for large-scale batch jobs.

Exam trap

The trap here is that candidates often assume standard worker nodes are always better for performance, ignoring the cost-benefit of preemptible nodes that allow scaling to many more workers for the same budget, which directly reduces execution time for parallelizable jobs.

How to eliminate wrong answers

Option A is wrong because a single-node cluster lacks parallelism, so processing 100GB files would be severely bottlenecked by a single machine's CPU and memory, leading to long execution times. Option C is wrong because HDFS is not used for input data from Cloud Storage; Dataproc reads directly from Cloud Storage via the gs:// connector, and using HDFS would require copying data first, adding network latency and storage overhead. Option D is wrong because using many standard worker nodes is less cost-effective than using preemptible nodes; standard nodes are more expensive, so for the same budget you can provision fewer workers, resulting in longer job execution times compared to a larger cluster of preemptible nodes.

Full explanation →
503
MCQmedium

A team is designing a data lake on Google Cloud using Cloud Storage and BigQuery. They need to ensure that sensitive data (e.g., PII) is encrypted at rest and have the ability to audit access. Which approach meets these requirements?

A.Use Customer-Managed Encryption Keys (CMEK) and enable VPC Service Controls.
B.Use Customer-Managed Encryption Keys (CMEK) and enable Cloud Audit Logs.
C.Use Default Encryption and enable Data Loss Prevention (DLP) API.
D.Use Customer-Supplied Encryption Keys (CSEK) and enable VPC Service Controls.
AnswerB

CMEK provides control over encryption keys, and Cloud Audit Logs record access to data.

Why this answer

Customer-Managed Encryption Keys (CMEK) allow the team to control and manage the encryption keys used to protect data at rest in Cloud Storage and BigQuery, while enabling Cloud Audit Logs provides the necessary audit trail for access to both the data and the keys. This combination directly satisfies the requirements for encryption at rest and auditability.

Exam trap

Google Cloud often tests the distinction between encryption key management (CMEK vs. CSEK vs. Default) and security controls (VPC Service Controls vs.

Audit Logs), leading candidates to conflate network perimeter controls with audit capabilities.

How to eliminate wrong answers

Option A is wrong because VPC Service Controls provide network-based security boundaries to prevent data exfiltration, but they do not provide audit logging of access to data or keys, which is a separate requirement. Option C is wrong because Default Encryption uses Google-managed keys, which do not give the team control over encryption keys, and the DLP API is for inspecting and classifying sensitive data, not for encryption at rest or audit logging. Option D is wrong because Customer-Supplied Encryption Keys (CSEK) require the customer to manage their own keys outside Google Cloud, which adds operational complexity and does not integrate with Cloud Audit Logs for key access auditing; VPC Service Controls again do not provide audit logging.

Full explanation →
504
MCQmedium

You are building a streaming pipeline to ingest real-time clickstream data from a website into BigQuery for immediate analysis. The data must be available in BigQuery within seconds and you need to handle late-arriving data (e.g., browser offline events) that may arrive hours later. Which approach should you use?

A.Use Pub/Sub with Dataflow, writing to BigQuery using the Storage Write API in committed mode.
B.Use Cloud Logging to capture logs and export to BigQuery via a sink.
C.Use Pub/Sub with Cloud Functions, writing each event directly via BigQuery legacy streaming inserts.
D.Use Datastream to stream clickstream data from Cloud SQL to BigQuery.
AnswerA

This provides low-latency streaming, late data handling via Dataflow's triggers, and efficient writes.

Why this answer

Pub/Sub provides a scalable, durable ingestion layer for real-time clickstream data, and Dataflow can handle late-arriving data via its built-in watermark and trigger mechanisms. The Storage Write API in committed mode ensures exactly-once semantics and low-latency writes to BigQuery, meeting the sub-second availability requirement while preserving data consistency for delayed events.

Exam trap

The trap here is that candidates assume legacy streaming inserts (Option C) are sufficient for real-time needs, but they overlook the 1-hour buffer delay and lack of late-data handling, which are explicitly tested in the PDE exam's focus on streaming pipelines with out-of-order events.

How to eliminate wrong answers

Option B is wrong because Cloud Logging is designed for log ingestion and analysis, not for high-throughput real-time clickstream pipelines; exporting logs via a sink introduces latency (typically minutes) and cannot guarantee sub-second BigQuery availability. Option C is wrong because BigQuery legacy streaming inserts have a 1-hour buffer before data is available for queries, do not support exactly-once semantics, and Cloud Functions lack the stateful processing capabilities (e.g., windowing, triggers) needed to handle late-arriving data correctly. Option D is wrong because Datastream is built for continuous replication from databases like Cloud SQL to BigQuery, not for ingesting raw clickstream events from a website; it requires an intermediary database, which adds unnecessary complexity and latency.

Full explanation →
505
MCQhard

You are migrating a large on-premises data warehouse to BigQuery. The data includes sensitive PII columns that must be masked for certain users. Which BigQuery feature can automatically redact PII in query results based on user roles?

A.IAM conditions on tables
B.Authorized views
C.Column-level security with data masking
D.Cloud DLP API
AnswerC

Data masking policy tags can automatically redact PII based on user roles.

Why this answer

BigQuery column-level security with policy tags and data masking can automatically mask sensitive data based on IAM roles. Authorized views require manual creation. Dynamic data masking is part of column-level security.

IAM conditions don't mask data.

Full explanation →
506
MCQeasy

You need to process large-scale log files (hundreds of terabytes) using Apache Spark on Google Cloud. The job runs nightly and you want to minimise costs. Which Dataproc cluster configuration is MOST cost-effective?

A.Single-node cluster
B.Standard cluster with preemptible workers for the primary worker nodes
C.Standard cluster with preemptible secondary workers
D.Standard cluster with all standard (non-preemptible) workers
AnswerC

Secondary workers can be preemptible, reducing cost. Primary workers handle coordination and must be standard.

Why this answer

Preemptible VMs are significantly cheaper than standard VMs and suitable for fault-tolerant batch jobs like nightly Spark processing. Standard mode is fine but using preemptible workers reduces cost.

Full explanation →
507
MCQmedium

You configured a model deployment monitor on your Vertex AI endpoint as shown. What will happen when the feature 'age' has a skew of 0.4?

A.An alert will be sent to admin@example.com
B.The endpoint will automatically roll back to a previous model version
C.No alert will be sent because the skew threshold is 0.2 for income
D.An alert will be sent only if both features exceed their thresholds
AnswerA

Skew 0.4 exceeds threshold 0.3 for age.

Why this answer

The monitoring configuration shows an alert threshold of 0.3 for the feature 'age', and a skew of 0.4 exceeds that threshold. Vertex AI Model Monitoring will trigger the configured alert action, which in this case is sending an email to admin@example.com. The alert is based on the specific feature's threshold, not on any other feature's threshold.

Exam trap

Google Cloud often tests the misconception that alerts require multiple features to exceed thresholds or that the system can automatically roll back models, when in reality each feature is evaluated independently and only notifications are sent.

How to eliminate wrong answers

Option B is wrong because Vertex AI Model Monitoring does not automatically roll back model deployments; it only sends alerts based on configured actions, and auto-rollback is not a supported feature in this context. Option C is wrong because the skew threshold for 'age' is 0.2, not 0.2 for 'income'; the question states the skew for 'age' is 0.4, which exceeds its own threshold, so an alert will be sent regardless of the 'income' feature's threshold. Option D is wrong because the alert is triggered per feature when its individual threshold is exceeded; there is no requirement for both features to exceed their thresholds simultaneously.

Full explanation →
508
Drag & Dropmedium

Drag and drop the steps to migrate an on-premises MySQL database to Cloud SQL using Database Migration Service into the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps
Order
1Step 1
2Step 2
3Step 3
4Step 4

Why this order

Database Migration Service enables minimal-downtime migrations using replication.

Full explanation →
509
Multi-Selecthard

A company uses Cloud Composer to orchestrate Dataproc and BigQuery jobs. They need to implement retry logic for transient failures. Which THREE features can help?

Select 3 answers
A.Dataflow pipeline retries
B.DAG retry_delay
C.BigQuery job retries
D.Cloud Composer high availability
E.Task retries and retry_delay
AnswersB, C, E

Composer can retry the entire DAG on failure with a delay.

Why this answer

Cloud Composer (Apache Airflow) allows setting `retry_delay` at the DAG level to define the time delay between task retries. This is a native Airflow feature that helps handle transient failures by automatically retrying failed tasks after a specified delay, reducing manual intervention.

Exam trap

The trap here is confusing infrastructure-level high availability (Option D) with application-level retry logic, leading candidates to select HA as a retry mechanism when it only ensures environment uptime, not task-level failure recovery.

Full explanation →
510
MCQmedium

An e-commerce company runs a daily batch pipeline that processes clickstream data from Cloud Storage using Cloud Dataproc with Spark. The pipeline includes a join between a large fact table and a small dimension table. The dimension table is stored in Cloud Storage as a CSV file. The join is slow due to shuffling. The data engineer considers broadcasting the dimension table. However, the dimension table is updated daily and the pipeline reads the latest version. What is the best approach to implement this optimization?

A.Use DataFrame.join with broadcast hint on the dimension DataFrame
B.Read the fact table and dimension table into separate DataFrames and use standard join
C.Read the dimension table as an RDD and collect as a map, then use map-side join
D.Increase the spark.sql.autoBroadcastJoinThreshold to a large value
AnswerA

Forces broadcast join regardless of table size.

Why this answer

Broadcasting the small dimension table using the broadcast hint (e.g., `broadcast(dimensionDF)`) forces Spark to replicate the dimension data to all executor nodes, eliminating the need for a shuffle during the join. This is ideal when the dimension table is small enough to fit in executor memory, and since the pipeline reads the latest CSV daily, the broadcast will automatically use the updated data without additional code changes.

Exam trap

The trap here is that candidates may think increasing `spark.sql.autoBroadcastJoinThreshold` is a safe global fix, but it can cause memory pressure and does not guarantee a broadcast join if the table size fluctuates, whereas the explicit broadcast hint provides deterministic behavior.

How to eliminate wrong answers

Option B is wrong because a standard join without any hint or optimization will trigger a full shuffle of both datasets, which is exactly the performance problem described. Option C is wrong because manually collecting the dimension table as an RDD and using a map-side join is an outdated, error-prone approach that bypasses Spark SQL's Catalyst optimizer and broadcast join optimizations; it also requires manual handling of updates and memory management. Option D is wrong because increasing `spark.sql.autoBroadcastJoinThreshold` globally may cause the dimension table to be broadcast automatically, but it does not guarantee the join uses a broadcast if the table size exceeds the threshold, and it can lead to out-of-memory errors if the threshold is set too high without considering executor memory limits.

Full explanation →
511
MCQeasy

A data engineer tries to grant a service account read access to a Cloud Storage bucket using the IAM policy above. The service account still cannot read objects. What is the most likely reason?

A.The role does not include the necessary permission
B.The condition prevents access because the request time is after 2023
C.The service account is misspelled
D.The role should be roles/storage.admin
AnswerB

The condition expression requires request.time before 2023, which is likely no longer true.

Why this answer

The IAM condition explicitly restricts access to requests made before January 1, 2023. Since the current time is after that date, the condition evaluates to false, denying the service account's read access regardless of the role binding. IAM conditions are evaluated at request time, and if the condition is not met, the permission is not granted.

Exam trap

Google Cloud often tests the subtlety that IAM conditions are evaluated at request time and can override a valid role binding, leading candidates to mistakenly focus on the role's permissions rather than the condition's effect.

How to eliminate wrong answers

Option A is wrong because roles/storage.objectViewer includes the storage.objects.get permission required to read objects, so the role does include the necessary permission. Option C is wrong because a misspelled service account would result in the role not being bound at all, but the question states the policy was applied, implying the service account name is correct. Option D is wrong because roles/storage.admin is an overly permissive role that includes many additional permissions beyond read access; the issue is not the role's permissions but the condition blocking access.

Full explanation →
512
MCQeasy

A company uses Cloud Monitoring to track application latency. They notice a spike in latency every 30 minutes. What is the best initial step to diagnose the issue?

A.Increase the number of instances to handle the load.
B.Enable Cloud Trace for all requests.
C.Check if scheduled jobs or cron tasks overlap.
D.Change the alert threshold to ignore the spikes.
AnswerC

Regularly recurring spikes suggest a scheduled job causing contention; investigating this is the most direct diagnostic step.

Why this answer

Recurring spikes at regular intervals often indicate a scheduled process (e.g., cron job, batch job) that runs every 30 minutes. Checking for overlapping scheduled jobs is the most efficient first step before scaling or other actions.

Full explanation →
513
MCQmedium

A data engineer needs to store raw sensor data in Cloud Storage and automatically transition it to a lower-cost storage class after 30 days, then delete it after 365 days. What should they configure?

A.Use Cloud Pub/Sub notifications to trigger a Cloud Function that moves objects.
B.Use gsutil rewrite command in a cron job.
C.Configure a lifecycle rule with SetStorageClass to Nearline after 30 days and Delete after 365 days.
D.Set a bucket retention policy with a retention period of 365 days.
AnswerC

Lifecycle rules can automatically transition objects to a different storage class and then delete them based on age.

Why this answer

Cloud Storage lifecycle management rules allow you to automatically transition objects to a lower-cost storage class (such as Nearline) after a specified number of days and then delete them after another period. This is the native, serverless way to manage object lifecycle without external scripts or compute resources.

Exam trap

Google Cloud often tests the distinction between lifecycle management (which automates transitions and deletions) and retention policies (which only prevent deletion/overwrites), leading candidates to confuse the two.

How to eliminate wrong answers

Option A is wrong because Cloud Pub/Sub notifications and Cloud Functions introduce unnecessary complexity and cost; lifecycle rules handle this natively without custom code. Option B is wrong because using gsutil rewrite in a cron job is a manual, error-prone approach that does not scale and incurs additional egress/operation costs; lifecycle rules are the intended automated solution. Option D is wrong because a bucket retention policy prevents deletion before the retention period ends, but it does not automatically transition objects to a lower-cost storage class; it only enforces immutability.

Full explanation →
514
Multi-Selectmedium

Which TWO statements are correct about designing a data pipeline using Cloud Dataflow for processing unbounded data?

Select 2 answers
A.Watermarks are used to measure the progress of event time.
B.Triggers can only emit results at the end of a window.
C.Dataflow guarantees exactly-once processing for streaming pipelines.
D.Cloud Pub/Sub is the recommended source for streaming pipelines.
E.Fixed windows are always based on processing time.
AnswersA, D

Watermarks track event time progress.

Why this answer

Watermarks in Cloud Dataflow measure the progress of event time, indicating when all data up to a certain timestamp is expected to have arrived. This allows the pipeline to handle late-arriving data and determine when to close windows for unbounded data streams.

Exam trap

Google Cloud often tests the misconception that triggers only fire at window boundaries, when in fact Dataflow supports early, on-time, and late firings for flexible result emission.

Full explanation →
515
MCQmedium

A Cloud Build pipeline is set up to train a model on Vertex AI. The build fails with the error: 'ERROR: (gcloud.ai-platform.jobs.submit.training) NOT_FOUND: The parent project does not exist.' The project ID and the service account are correctly configured. What is the most likely cause?

A.The region specified for the training job does not exist.
B.The training job requires a GPU, which is not available in the specified region.
C.The Cloud Build service account does not have the aiplatform.jobs.create permission on the project.
D.The training package is not uploaded to Cloud Storage before the pipeline runs.
AnswerC

Insufficient permissions can cause the project to appear as not found to the service account.

Why this answer

The error 'NOT_FOUND: The parent project does not exist' indicates that the Cloud Build service account lacks the necessary IAM permission to submit a training job to Vertex AI. Even though the project ID and service account are correctly configured, the Cloud Build service account must have the 'aiplatform.jobs.create' permission (or the 'Vertex AI User' role) on the project. Without this, the API call fails because the service account is not authorized to access the project resource.

Exam trap

Google Cloud often tests the misconception that a 'NOT_FOUND' error always means a missing resource (like a project ID or region), when in fact it can indicate an IAM permission issue where the service account is not authorized to see or use the project.

How to eliminate wrong answers

Option A is wrong because an invalid region would produce a different error, such as 'INVALID_ARGUMENT: Region not found' or 'PERMISSION_DENIED', not 'NOT_FOUND: The parent project does not exist'. Option B is wrong because GPU availability issues would result in a 'RESOURCE_EXHAUSTED' or 'ZONE_RESOURCE_POOL_EXHAUSTED' error, not a project-level not found error. Option D is wrong because a missing training package in Cloud Storage would cause a 'FILE_NOT_FOUND' or 'INVALID_ARGUMENT' error during job submission, not a project not found error.

Full explanation →
516
MCQeasy

You need to allow a data analyst to run queries on a BigQuery dataset but prevent them from modifying the data or deleting the dataset. Which IAM role should you grant?

A.roles/bigquery.dataOwner
B.roles/bigquery.dataViewer
C.roles/bigquery.jobUser
D.roles/bigquery.dataEditor
AnswerB

DataViewer grants read-only access to data and metadata, ideal for analysts.

Why this answer

BigQuery Data Viewer grants read-only access to datasets and tables. roles/bigquery.dataViewer allows running queries and viewing metadata but not modifying or deleting data.

Full explanation →
517
MCQmedium

A company has deployed a machine learning model to AI Platform Prediction. The model uses a custom container with a TensorFlow SavedModel. After deployment, the prediction latency is higher than expected. Which action is most likely to reduce latency without significantly impacting model accuracy?

A.Convert the model to TensorFlow Lite and use a smaller model.
B.Increase the number of prediction nodes in the AI Platform Prediction cluster.
C.Enable XLA (Accelerated Linear Algebra) compilation on model loading.
D.Apply quantization to the model weights to reduce size.
AnswerC

XLA compiles and optimizes the TensorFlow graph, often improving latency without affecting accuracy.

Why this answer

Enabling XLA (Accelerated Linear Algebra) compilation on model loading optimizes the TensorFlow computation graph by fusing operations and reducing runtime overhead, which directly lowers prediction latency without altering model weights or architecture. XLA works by compiling the graph into efficient machine code at load time, improving execution speed while preserving the original model accuracy.

Exam trap

A common pitfall in Google Cloud exams is confusing latency reduction with throughput scaling; candidates often choose to increase prediction nodes (Option B) thinking it reduces latency, when it actually only improves concurrent request handling.

How to eliminate wrong answers

Option A is wrong because converting to TensorFlow Lite and using a smaller model typically reduces model size and latency but often at the cost of accuracy due to pruning or reduced precision, and it is not a latency optimization that preserves accuracy. Option B is wrong because increasing the number of prediction nodes scales horizontally to handle more concurrent requests but does not reduce the per-request latency; it addresses throughput, not the latency of a single prediction. Option D is wrong because applying quantization to model weights reduces model size and can improve latency but usually introduces a trade-off with accuracy, especially with post-training quantization, and the question specifies 'without significantly impacting model accuracy,' making XLA a safer choice.

Full explanation →
518
MCQmedium

A company needs to store petabytes of time-series IoT sensor data and query it with single-digit millisecond latency at millions of reads per second. The data has a simple key-value structure with timestamps. Which Google Cloud database is MOST appropriate?

A.Cloud Bigtable
B.BigQuery
C.Cloud Spanner
D.Firestore
AnswerA

Bigtable is the correct choice: wide-column NoSQL, designed for time-series and IoT workloads, single-digit ms latency, and scales to millions of QPS with additional nodes.

Why this answer

Cloud Bigtable is a fully managed, scalable NoSQL database designed for large analytical and operational workloads, handling petabytes of data with consistent sub-10ms latency at millions of reads per second. Its key-value storage model and automatic sharding make it ideal for time-series IoT sensor data with simple timestamp-based keys, supporting high-throughput, low-latency access without the overhead of relational features.

Exam trap

The trap here is assuming that 'petabytes of data' automatically means BigQuery, ignoring the critical requirement for single-digit millisecond latency at millions of reads per second. BigQuery is an analytical warehouse optimized for large-scale analytics, not for sub-10ms point lookups.

How to eliminate wrong answers

Option B (BigQuery) is wrong because it is a serverless data warehouse optimized for analytical SQL queries on large datasets, not for single-digit millisecond point reads at millions of operations per second; its latency is typically in the seconds range for interactive queries. Option C (Cloud Spanner) is wrong because it is a globally distributed relational database with strong consistency and ACID transactions, which introduces overhead unsuitable for the simple key-value time-series pattern and cannot match Bigtable's throughput for millions of reads per second. Option D (Firestore) is wrong because it is a mobile and web document database with limited throughput (up to 10,000 writes/second per database) and is not designed for petabyte-scale time-series data or sub-millisecond latency at millions of reads per second.

Full explanation →
519
Multi-Selectmedium

Which THREE Google Cloud services are typically used together in a production ML pipeline?

Select 3 answers
A.Cloud Storage
B.Cloud Functions
C.Vertex AI Training
D.Vertex AI Prediction
E.BigQuery
AnswersA, C, D

For storing training data, model artifacts, etc.

Why this answer

Cloud Storage is correct because it serves as the central artifact repository in a production ML pipeline on Google Cloud. It stores training data, model artifacts, and prediction inputs/outputs, enabling seamless integration with Vertex AI Training for model training and Vertex AI Prediction for serving. Without Cloud Storage, there is no durable, scalable, and cost-effective way to manage the large datasets and model binaries required for production ML workflows.

Exam trap

The trap here is that candidates confuse 'services used in an ML pipeline' with 'services that can be used somewhere in ML' — Cloud Functions and BigQuery are often used in ML workflows (e.g., triggering retraining or storing features), but they are not the three core services that are typically used together in a production ML pipeline for training, storing artifacts, and serving predictions.

Full explanation →
520
MCQmedium

A data engineer needs to ingest daily Salesforce reports into BigQuery without writing custom code. The reports are exported to an Amazon S3 bucket on a schedule. Which service should they use to automate the transfer?

A.Cloud Dataproc
B.BigQuery Data Transfer Service
C.Cloud Composer
D.Cloud Storage Transfer Service
AnswerB

Supports Amazon S3 as a source for scheduled transfers directly into BigQuery.

Why this answer

The BigQuery Data Transfer Service (BQDTS) is the correct choice because it supports scheduled, automatic ingestion of Salesforce reports exported to Amazon S3 into BigQuery without requiring custom code. It can connect to S3 as a source and load the data into BigQuery tables on a schedule, handling schema detection and incremental updates. This meets the requirement of no-code automation from an S3 bucket.

Exam trap

The trap here is that candidates often confuse Cloud Storage Transfer Service (which only moves files between storage buckets) with BigQuery Data Transfer Service (which directly ingests from SaaS applications like Salesforce into BigQuery), leading them to pick option D when the requirement is for a no-code, direct-to-BigQuery solution.

How to eliminate wrong answers

Option A is wrong because Cloud Dataproc is a managed Spark/Hadoop service for running big data processing jobs, not a no-code data ingestion tool; it would require writing custom code to extract from Salesforce and load into BigQuery. Option C is wrong because Cloud Composer is a managed Apache Airflow service for orchestrating workflows; while it could be used to build a custom pipeline, it requires writing DAGs and code, which contradicts the 'without writing custom code' requirement. Option D is wrong because Cloud Storage Transfer Service is designed for moving data between cloud storage systems (e.g., S3 to GCS) and does not directly ingest data into BigQuery or connect to Salesforce APIs.

Full explanation →
521
MCQmedium

A Dataflow pipeline reads log files from Cloud Storage, parses them into LogEvent objects, and writes to BigQuery. The pipeline fails with the above errors. What is the most likely cause?

A.The LogEvent class does not have a no-argument constructor.
B.The pipeline is missing required import statements for LogEvent.
C.The BigQuery table schema does not match the LogEvent fields.
D.The log files are not in the expected format, causing parsing failures.
AnswerA

Beam requires a no-arg constructor for Avro or Serializable coders.

Why this answer

Apache Beam's SDK requires that custom types used as PCollection elements (like LogEvent) have a no-argument constructor so that the framework can deserialize objects during distributed processing, especially when using the Dataflow runner. Without it, the pipeline fails at runtime with a serialization error because Beam's default coder (e.g., SerializableCoder) cannot reconstruct the object.

Exam trap

The trap here is that candidates confuse runtime serialization errors with compile-time import issues or schema mismatches, overlooking the fundamental requirement for a no-argument constructor in Beam's default coders.

How to eliminate wrong answers

Option B is wrong because missing import statements would cause a compile-time error, not a runtime pipeline failure with the described errors. Option C is wrong because a BigQuery table schema mismatch would produce a write-time error (e.g., schema mismatch), not a serialization failure during parsing. Option D is wrong because parsing failures from malformed log files would result in exceptions during the parse step, not a serialization error related to the LogEvent class itself.

Full explanation →
522
MCQhard

You are designing a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle late-arriving data (up to 1 hour) and group events into 10-minute windows. Which configuration is correct?

A.Use global windows with a trigger that fires every 10 minutes
B.Use sliding windows of 10 minutes with a 5-minute period and allowed lateness of 1 hour
C.Use fixed windows of 10 minutes with allowed lateness of 0 seconds
D.Use fixed windows of 10 minutes with allowed lateness of 1 hour and a trigger that fires after watermark plus early firings
AnswerD

This allows late data up to 1 hour and provides timely results.

Why this answer

To handle late data, you need to set the allowed lateness to 1 hour. The trigger with AfterWatermark with early firings ensures results are emitted on time and updated when late data arrives.

Full explanation →
523
MCQmedium

A data team runs regular analytical queries on a BigQuery table that stores 2 years of sales data (approximately 10 TB). Queries frequently filter on a `sale_date` column and also group by `product_id`. To optimize cost and performance, which design approach is most effective?

A.Do not partition; only cluster by `sale_date`.
B.Partition by `sale_date` and set a table expiration of 90 days.
C.Partition the table by `sale_date` and cluster by `product_id`.
D.Partition by `product_id` and cluster by `sale_date`.
AnswerC

Partitioning by date enables partition elimination on date filters; clustering by product_id co-locates rows with the same product_id within each partition, improving GROUP BY performance.

Why this answer

Partitioning by `sale_date` allows BigQuery to perform partition pruning, eliminating scans of irrelevant date ranges, while clustering by `product_id` physically co-locates rows with the same product ID within each partition. This combination minimizes the data scanned for queries that filter on `sale_date` and group by `product_id`, directly reducing both cost (bytes billed) and query latency.

Exam trap

Google Cloud often tests the misconception that partitioning by a high-cardinality column like `product_id` is acceptable, but the trap here is that BigQuery enforces a hard limit of 4,000 partitions per table, making such a design infeasible and forcing candidates to recognize that clustering is the correct mechanism for high-cardinality grouping columns.

How to eliminate wrong answers

Option A is wrong because without partitioning, BigQuery must scan the entire 10 TB table even for queries filtering on a narrow date range, leading to unnecessarily high costs and slower performance. Option B is wrong because setting a table expiration of 90 days would delete historical data needed for 2-year analysis, and partitioning alone without clustering does not optimize the GROUP BY on `product_id` within each partition. Option D is wrong because partitioning by `product_id` (a high-cardinality column) would create millions of tiny partitions, exceeding BigQuery's partition limit (4,000 partitions per table) and causing poor performance and management overhead.

Full explanation →
524
Drag & Dropmedium

Drag and drop the steps to create a Cloud Storage bucket with uniform bucket-level access into the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps
Order
1Step 1
2Step 2
3Step 3
4Step 4

Why this order

Uniform bucket-level access simplifies permissions by using IAM policies at the bucket level instead of ACLs.

Full explanation →
525
MCQmedium

A company wants to use Cloud Data Fusion for ETL pipelines. They need to integrate with custom transformations not available in the marketplace. What should they do?

A.Switch to Dataproc and write a Spark job.
B.Use the Data Fusion Hub to download a custom plugin.
C.Use Dataprep to create the transformation.
D.Write a custom plugin using the CDAP SDK and deploy it.
AnswerD

The CDAP SDK allows building custom plugins.

Why this answer

Cloud Data Fusion supports custom plugins using the CDAP SDK.

Full explanation →
Page 6

Page 7 of 12

Page 8

All pages

123456789101112

Practice PDE by domain

Target a specific domain to shore up weak areas.

Designing Data Processing SystemsIngesting and Processing the DataStoring the DataPreparing and Using Data for AnalysisMaintaining and Automating Data Workloads
See all domains with question counts →