Knowledge + Practice

Google Professional Data Engineer (PDE) — Questions 526–600

890 questions total · 12pages · All types, answers revealed

Take a mock exam Exam hub

Page 8 of 12

526

MCQeasy

You need to ingest Google Ads performance data into BigQuery on a daily basis for reporting. Which service should you use?

A.BigQuery Data Transfer Service for Google Ads

B.Cloud Scheduler to call Google Ads API and load to BigQuery

C.Pub/Sub with a Google Ads subscriber

D.Storage Transfer Service for Google Ads

AnswerA

This service is specifically designed to import data from Google Ads into BigQuery on a scheduled basis.

Why this answer

The BigQuery Data Transfer Service for Google Ads is the correct choice because it provides a fully managed, scheduled connector that automatically ingests Google Ads performance data into BigQuery on a daily basis without requiring any custom code. It handles authentication, schema mapping, and incremental loads, making it the simplest and most reliable solution for this specific use case.

Exam trap

Google often tests the distinction between fully managed services (like BigQuery Data Transfer Service) and generic infrastructure components (like Cloud Scheduler or Pub/Sub) that require custom development, leading candidates to overcomplicate the solution by choosing a more flexible but less appropriate option.

How to eliminate wrong answers

Option B is wrong because Cloud Scheduler is a cron job service that can trigger HTTP requests, but it does not natively integrate with the Google Ads API or handle the complex authentication, pagination, and schema mapping required to load data into BigQuery; you would still need to build and maintain a custom application. Option C is wrong because Pub/Sub is a messaging service for asynchronous event streaming, not a batch ingestion tool; while you could theoretically publish Google Ads data to Pub/Sub, there is no native Google Ads subscriber, and you would need to build a custom subscriber to write to BigQuery, which is far more complex than using the dedicated transfer service. Option D is wrong because Storage Transfer Service is designed for moving data from on-premises or cloud storage (like S3 or HTTP endpoints) into Google Cloud Storage, not for directly ingesting data from Google Ads into BigQuery.

Full explanation →

527

Multi-Selecthard

A company is designing a data lake on Cloud Storage for analytics. They need to store data in various formats (Avro, Parquet, CSV) and enable efficient querying with BigQuery and Dataproc. Which THREE practices should they follow?

Select 3 answers

A.Use BigLake to create BigQuery tables that reference Cloud Storage data.

B.Store data in columnar formats like Parquet for analytics workloads.

C.Disable encryption on the bucket to improve read performance.

D.Partition data by date in a logical folder structure (e.g., /data/yyyy/mm/dd).

E.Store all data in CSV format for simplicity.

AnswersA, B, D

Enables querying data without loading.

Why this answer

BigLake allows you to create BigQuery tables that reference data stored in Cloud Storage, enabling unified governance and fine-grained access control without moving data. This is essential for a data lake architecture where BigQuery and Dataproc need to query the same underlying data in various formats like Avro, Parquet, and CSV.

Exam trap

Google Cloud often tests the misconception that disabling encryption improves performance, but Cloud Storage encryption is transparent and has no measurable impact on read throughput, so candidates should recognize that security controls are non-negotiable in cloud data lakes.

Full explanation →

528

MCQeasy

A company uses Cloud Functions to process events from Cloud Storage. They notice that occasionally functions are not triggered. What should they check first to ensure solution quality?

A.Verify that the Cloud Storage bucket has notifications configured for the correct event type.

B.Check the logs for function execution.

C.Increase the function memory allocation.

D.Increase the function timeout.

AnswerA

A misconfigured notification will prevent the function from being triggered at all.

Why this answer

The most common reason for Cloud Functions not being triggered by Cloud Storage events is that the bucket's notification configuration is missing or misconfigured. Cloud Functions relies on Pub/Sub notifications from the bucket to invoke the function; if the notification is not set for the correct event type (e.g., `OBJECT_FINALIZE`), the function will never be triggered. Therefore, verifying the notification configuration is the first and most direct diagnostic step.

Exam trap

Google often tests the misconception that performance tuning (memory or timeout) is the first step to fix trigger issues, when the root cause is almost always a missing or misconfigured event notification.

How to eliminate wrong answers

Option B is wrong because checking logs for function execution assumes the function was invoked, but if the trigger is not firing, there will be no execution logs to review. Option C is wrong because increasing memory allocation addresses performance issues like out-of-memory errors, not trigger failures. Option D is wrong because increasing the timeout addresses function execution duration limits, not the absence of invocation.

Full explanation →

529

MCQeasy

An organization wants to automate their batch data processing pipeline using Cloud Composer. The pipeline consists of multiple tasks: extract from Cloud Storage, transform with Dataflow, and load into BigQuery. Which Airflow operator should be used to run Dataflow jobs?

A.BigQueryInsertJobOperator

B.DataflowCreatePythonJobOperator

C.GCSToBigQueryOperator

D.DataprocSubmitJobOperator

AnswerB

This operator submits a Dataflow job written in Python.

Why this answer

B is correct because the DataflowCreatePythonJobOperator is specifically designed to submit and manage Apache Beam pipelines written in Python as Dataflow jobs in Google Cloud. This operator handles the creation of a Dataflow job from a Python file, which aligns with the requirement to run Dataflow transformations within a Cloud Composer DAG.

Exam trap

Google Cloud often tests the distinction between Dataflow and Dataproc operators, so the trap here is that candidates might confuse DataprocSubmitJobOperator (for Hadoop/Spark) with Dataflow operators, especially when the question mentions 'transform' without specifying the processing framework.

How to eliminate wrong answers

Option A is wrong because BigQueryInsertJobOperator is used to run BigQuery jobs (e.g., queries, load jobs), not to submit Dataflow pipelines. Option C is wrong because GCSToBigQueryOperator loads data directly from Cloud Storage to BigQuery without using Dataflow for transformation, bypassing the required transform step. Option D is wrong because DataprocSubmitJobOperator submits jobs to Dataproc (Hadoop/Spark clusters), not to Dataflow, which is a different processing service.

Full explanation →

530

MCQeasy

A data engineer needs to process streaming data from thousands of IoT devices and generate real-time dashboards. The data volume is low but requires exactly-once processing semantics. Which Google Cloud service combination should they use?

A.Cloud Pub/Sub + Cloud Data Fusion

B.Cloud Pub/Sub + Cloud Dataproc

C.Cloud Pub/Sub + Cloud Dataflow

D.Cloud Pub/Sub + Cloud Dataprep

AnswerC

Cloud Pub/Sub for ingestion with at-least-once delivery, combined with Cloud Dataflow which provides exactly-once processing via its streaming engine.

Why this answer

Dataflow supports exactly-once processing via its streaming engine and checkpointing. Pub/Sub is the ingest service. Together they provide the required semantics.

Full explanation →

531

Multi-Selectmedium

A data scientist wants to use Vertex AI Workbench for exploratory data analysis. Which TWO statements are true about Vertex AI Workbench?

Select 2 answers

A.It is a serverless service that scales to zero when not in use.

B.It supports custom container images for the notebook environment.

C.It can only be used with TensorFlow.

D.It provides a managed JupyterLab environment with pre-installed ML libraries.

E.It includes a built-in SQL query editor for BigQuery.

AnswersB, D

Correct: supports custom containers.

Why this answer

Vertex AI Workbench provides managed Jupyter notebooks with pre-installed ML frameworks. It integrates with BigQuery via the Python client. It does not provide a SQL editor; that's BigQuery.

It supports custom containers. It is not serverless; it runs on Compute Engine VMs.

Full explanation →

532

MCQhard

A company runs a data pipeline that ingests clickstream events from multiple websites into Cloud Pub/Sub, then processed by Dataflow to generate user sessions, and written to BigQuery for analytics. The pipeline runs 24/7. Recently, the team noticed that some sessions are incomplete due to missing events, and data quality checks reveal that about 2% of sessions have gaps of more than 30 minutes. The pipeline uses fixed 30-minute windows for sessionization, with allowed lateness set to 10 minutes. They have Cloud Monitoring dashboards tracking system throughput and pipeline lag but do not have custom metrics tracking per-element delays or watermark progress. The team suspects two possible causes: (a) the Pub/Sub subscription accumulates backlog and some messages are delivered after the window end; (b) the Dataflow job has insufficient workers causing checkpoint failures. The team needs to determine the root cause and improve data quality. What is the best first course of action?

A.Change the Pub/Sub subscription to pull mode with more aggressive flow control settings.

B.Increase the number of Dataflow workers and set autoscaling to the maximum allowed.

C.Modify the Dataflow pipeline to use session windows instead of fixed windows, and increase allowed lateness to 60 minutes.

D.Set up a Dataflow monitoring dashboard that tracks the watermark delay and create an alert when it exceeds the allowed lateness.

AnswerD

This directly monitors the pipeline's ability to process events within the window, confirming if late data is the root cause.

Why this answer

To determine whether late-arriving messages are the issue, the team should monitor the Dataflow watermark delay, which indicates how far behind the pipeline is compared to the event time. Setting up a metric and alert on watermark delay > allowed lateness will confirm if late data is being dropped.

Full explanation →

533

MCQmedium

A BigQuery table contains streaming data from Cloud Pub/Sub. The table is partitioned by ingestion time. A user runs a query that accesses data from the last 5 minutes and gets correct results. After 90 minutes, the user runs the same query again but notices that some rows are missing. What is the most likely cause?

A.The query is using time travel to a snapshot before the streaming buffer was committed

B.The query is using cached results that exclude recent data

C.The schema of the table was modified after the initial query

D.The table has a partition expiration of 30 days

AnswerA

Time travel queries return data from a snapshot; if the snapshot is before the buffer is flushed, recent data is missing.

Why this answer

BigQuery's streaming buffer provides low-latency access to recently ingested data, but this data is not immediately committed to managed storage. After the streaming buffer is flushed (typically within 90 minutes), the data becomes available in the table's base storage. If the user runs a query using time travel (e.g., `FOR SYSTEM_TIME AS OF`) to a snapshot taken before the buffer was committed, the query will only see data that was in managed storage at that snapshot time, missing rows that were still in the streaming buffer at that point.

Exam trap

Google Cloud often tests the misconception that cached results or schema changes are responsible for data inconsistencies, when the real issue is the separation between BigQuery's streaming buffer and managed storage, and how time travel queries only see committed data.

How to eliminate wrong answers

Option B is wrong because BigQuery caches query results only for identical queries within a 24-hour period, but the user ran the same query after 90 minutes; if cached results were used, they would include the same rows as the initial query, not missing rows. Option C is wrong because schema modifications do not cause rows to disappear from query results; they may affect column access or data types but do not remove existing rows. Option D is wrong because a partition expiration of 30 days would only remove partitions older than 30 days, not affect data from the last 5 minutes or 90 minutes.

Full explanation →

534

Drag & Dropmedium

Drag and drop the steps to create a Cloud Composer environment for Apache Airflow into the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps

Order

1Step 1

2Step 2

3Step 3

4Step 4

Why this order

Cloud Composer provides a managed Airflow environment for orchestrating workflows.

Full explanation →

535

MCQhard

A team is using BigQuery to analyze petabyte-scale data. They notice that queries are slow and expensive due to full table scans. They have already partitioned by date. What additional optimization should they implement?

A.Use materialized views

B.Cluster by frequently filtered columns

C.Convert to native tables

D.Use query caching

AnswerB

Clustering reduces bytes read when filtering on those columns.

Why this answer

Clustering by frequently filtered columns (option B) organizes data within each partition based on the sort order of those columns. This allows BigQuery to prune blocks during query execution, significantly reducing the amount of data scanned and improving both performance and cost. Since the table is already partitioned by date, clustering adds a secondary ordering that targets the most common filter predicates, avoiding full table scans within each partition.

Exam trap

Google Cloud often tests the distinction between partitioning and clustering, where candidates mistakenly believe partitioning alone is sufficient for all filtering scenarios, but clustering is required to avoid full scans on non-date columns.

How to eliminate wrong answers

Option A is wrong because materialized views precompute and store query results, which can speed up repeated aggregations but do not reduce the scan cost of ad-hoc filters on raw data; they are not a substitute for physical data organization like clustering. Option C is wrong because BigQuery tables are already native (managed) tables; converting to native tables is not a valid operation and does not address scan efficiency. Option D is wrong because query caching only returns results for identical queries run within 24 hours, but it does not reduce the scan cost or improve performance for new or slightly different queries that still trigger full table scans.

Full explanation →

536

MCQhard

Your Vertex AI model deployed on an endpoint is experiencing high tail latency during online predictions. The model uses a large embedding layer, and the input size varies. You have enabled automatic scaling with a minimum of 2 replicas and maximum of 10. What is the most likely cause of the latency spikes and the best first step to diagnose?

A.The model's SavedModel is too large due to the embedding layer; reduce embedding dimensions to lower latency.

B.The endpoint's target CPU utilization might be set too low, causing rapid scale-down and cold starts. Check Cloud Logging for scaling events.

C.The model uses a custom prediction routine that is not optimized; use tf.function to improve performance.

D.Enable model monitoring for online prediction and add a buffer to the endpoint's machine type.

AnswerB

If target utilization is low, replicas scale down quickly; cold starts on new requests cause latency. Logs show scaling.

Why this answer

High tail latency with variable input sizes and a large embedding layer often points to cold starts from aggressive scaling. When the target CPU utilization is set too low, the endpoint scales down quickly during lulls, and a subsequent burst of requests forces new replicas to spin up, causing latency spikes. Checking Cloud Logging for scaling events is the best first step because it directly reveals whether the endpoint is scaling down and then experiencing cold starts.

Exam trap

Google Cloud often tests the misconception that high tail latency is always due to model size or inference optimization, when in fact the most common cause in managed serving environments is autoscaling misconfiguration leading to cold starts.

How to eliminate wrong answers

Option A is wrong because reducing embedding dimensions would lower model accuracy and does not address the root cause of latency spikes from scaling dynamics; the model size is not the primary driver of tail latency in this scenario. Option C is wrong because while a custom prediction routine could be suboptimal, the question describes a standard model with a large embedding layer and variable input size, and the latency pattern (spikes) is more characteristic of cold starts than of per-request optimization issues; tf.function would help steady-state performance but not sudden spikes. Option D is wrong because model monitoring detects drift or anomalies but does not diagnose scaling-related latency, and adding a buffer to the machine type (e.g., increasing memory) does not fix the scaling policy that causes cold starts.

Full explanation →

537

MCQmedium

A data engineer notices that BigQuery queries are slower than expected. They want to identify the most expensive stages in the query execution. Which tool or command should they use?

A.Use bq show to view job statistics

B.Use bq query --format=prettyjson and look at statistics

C.Use Cloud Monitoring to view query execution graphs

D.Use EXPLAIN statement in BigQuery

AnswerB

The bq query command with --format=prettyjson returns job details including query plan stages and their costs, enabling identification of most expensive stages.

Why this answer

To identify expensive stages in BigQuery query execution, you should run the query with the bq command-line tool using the --format=prettyjson flag. This returns a detailed job object that includes a query plan with stages, each having cost estimates like 'avg execution time' and 'peak memory'. Analyzing these stages reveals the most resource-intensive parts.

Exam trap

Candidates often think EXPLAIN can identify expensive stages, but EXPLAIN only shows the query plan structure without execution cost data; for runtime expense analysis, the bq command with job details is required.

Full explanation →

538

Multi-Selecteasy

Which TWO approaches are recommended for handling late-arriving data in a streaming Dataflow pipeline?

Select 2 answers

A.Use side inputs to provide default values for late data.

B.Use fixed windows with a duration of 1 second to minimize lateness.

C.Configure allowed lateness on the window to accept late data.

D.Set the trigger to fire only at the end of the window.

E.Use a filter transform to drop late-arriving elements.

AnswersA, C

Side inputs can supply missing data.

Why this answer

Side inputs in Apache Beam (the programming model underlying Dataflow) allow you to provide default values or supplementary data to handle late-arriving elements gracefully. When a late element arrives after the window has been emitted, a side input can supply a fallback value, ensuring the pipeline can still process the data without discarding it. This approach is recommended for handling late data in streaming pipelines where completeness is not critical.

Exam trap

Google Cloud often tests the misconception that simply using small windows or dropping late data is a valid handling strategy, when in fact the recommended approaches involve configuring allowed lateness and using side inputs for graceful fallback.

Full explanation →

539

MCQhard

A data pipeline ingests sensor data from IoT devices via Cloud Pub/Sub, processes it with Cloud Dataflow, and writes to BigQuery. The pipeline is failing with high latency and data loss. Which troubleshooting step should be taken first?

A.Check Stackdriver logging for error messages.

B.Disable exactly-once processing in Dataflow.

C.Increase the number of Dataflow workers.

D.Switch to BigQuery streaming inserts.

AnswerA

Identifies root cause.

Why this answer

Stackdriver (now Cloud Logging) is the first place to investigate when a Dataflow pipeline experiences high latency and data loss. Dataflow automatically logs errors, worker failures, and system messages to Cloud Logging, which can reveal root causes such as insufficient resources, stuck steps, or Pub/Sub subscription issues. Checking logs first avoids premature scaling or configuration changes that may not address the actual problem.

Exam trap

Google Cloud often tests the principle of 'diagnose before you optimize' — the trap here is that candidates jump to scaling or switching technologies (options C and D) without first checking logs, which is the fundamental first step in any troubleshooting workflow.

How to eliminate wrong answers

Option B is wrong because disabling exactly-once processing in Dataflow would not fix high latency or data loss; it could actually increase data duplication and make debugging harder, while the core issue remains unaddressed. Option C is wrong because increasing the number of Dataflow workers without first diagnosing the bottleneck (e.g., a hot key, slow transform, or Pub/Sub backlog) can waste resources and may not resolve the underlying cause of latency or loss. Option D is wrong because switching to BigQuery streaming inserts does not address pipeline-level failures; streaming inserts have their own quotas, error handling, and latency characteristics, and the problem likely lies in the Dataflow processing logic or resource allocation, not the sink.

Full explanation →

540

MCQmedium

A company uses BigQuery to run reporting queries on a table that is partitioned by date and clustered by customer_id. Queries filtering by customer_id and a date range are performing poorly. What is the most likely cause?

A.The project lacks sufficient BigQuery slot capacity

B.The table is too large for BigQuery

C.Clustering column order should be date first, then customer_id

D.The date range filter is too wide, causing scans of many partitions

AnswerD

Wide date ranges nullify the benefit of clustering; BigQuery scans many partitions.

Why this answer

When a table is partitioned by date and clustered by customer_id, queries that filter on both columns can still perform poorly if the date range filter is too wide, causing BigQuery to scan many partitions. Even with clustering, scanning a large number of partitions negates the benefit of clustering, as clustering only reduces the data scanned within each partition. The query optimizer must read all partitions that fall within the date range, and if that range is broad, the scan overhead dominates.

Exam trap

The trap here is that candidates often assume clustering alone guarantees fast queries on any filter combination, without understanding that partition pruning happens first and a wide date range undermines the benefit of clustering.

How to eliminate wrong answers

Option A is wrong because insufficient slot capacity would cause slow query execution or queuing, not specifically poor performance on partitioned and clustered tables; the issue here is data scanning inefficiency, not resource contention. Option B is wrong because BigQuery is designed to handle tables of any size, and 'too large' is not a meaningful limitation; the problem is query design, not table size. Option C is wrong because the clustering column order is already correct for the typical query pattern (filtering by customer_id and date range); clustering by date first would not improve performance for queries that filter on customer_id, as clustering only benefits the first column in the order.

Full explanation →

541

Multi-Selectmedium

A company is migrating their on-premises data warehouse to BigQuery. They have a mix of batch and streaming ingestion. The data team wants to optimize query costs. Which THREE practices should they adopt?

Select 3 answers

A.Switch to flat-rate pricing to cap slot usage.

B.Use materialized views for frequently executed aggregations.

C.Partition tables by a date or timestamp column.

D.Limit the number of concurrent queries by setting a maximum slot capacity.

E.Cluster tables on columns that are frequently used in filters and joins.

AnswersB, C, E

Materialized views automatically refresh and are used by the query optimizer to speed up queries and reduce scanned bytes.

Why this answer

Partitioning by date reduces bytes scanned. Using clustered tables improves performance for filter/join queries. Using materialized views can precompute aggregations and reduce scans.

Flat-rate pricing is about reservation management, not cost optimization per query. Limiting slots is not a cost optimization; it may cause throttling.

Full explanation →

542

Multi-Selectmedium

Which THREE components are typically part of a Vertex AI Pipeline for automated model retraining and deployment?

Select 3 answers

A.Cloud Monitoring alerting component

B.Cloud Storage artifact storage component

C.Training component (e.g., CustomContainerTrainingJob)

D.Model evaluation component (e.g., evaluating on a test set)

E.Deployment component (e.g., deploying model to endpoint)

AnswersC, D, E

Training is the core step.

Why this answer

A training component, such as a `CustomContainerTrainingJob`, is the core step in a Vertex AI Pipeline that executes the model training logic. It defines the container image, machine configuration, and hyperparameters, enabling automated retraining when triggered by a schedule or event.

Exam trap

Google Cloud often tests the distinction between pipeline components (which are executable tasks in the DAG) and supporting infrastructure (like Cloud Monitoring or Cloud Storage), leading candidates to select options that are related to the pipeline's operation but not actual components within the pipeline definition.

Full explanation →

543

MCQeasy

You need to react to changes in a GCS bucket (e.g., new object creation) and trigger a Cloud Run service to process the new file. Which Google Cloud service should you use to route the event?

A.Pub/Sub directly with a Cloud Run subscription

B.Cloud Tasks

C.Eventarc

D.Cloud Scheduler

AnswerC

Eventarc handles events from GCS and other sources, routing them to Cloud Run.

Why this answer

Eventarc is the correct choice because it is purpose-built to route events from Google Cloud sources (like Cloud Storage) to Cloud Run. It directly supports Cloud Storage audit logs and Pub/Sub event triggers, allowing you to react to object creation events without custom middleware. Eventarc handles the event routing, filtering, and delivery to your Cloud Run service automatically.

Exam trap

Google often tests the misconception that Pub/Sub is the direct answer for any event routing, but the trap here is that Eventarc is the managed service that simplifies the integration between GCS and Cloud Run, making it the correct choice over raw Pub/Sub.

How to eliminate wrong answers

Option A is wrong because Pub/Sub directly with a Cloud Run subscription requires you to manually configure a Pub/Sub topic and subscription, and Cloud Run can only pull messages via a push subscription; Eventarc abstracts this complexity and provides native integration with Cloud Storage events. Option B is wrong because Cloud Tasks is a task queue for asynchronous execution of HTTP requests, not designed for event-driven routing from GCS; it would require you to manually publish tasks in response to events, adding unnecessary overhead. Option D is wrong because Cloud Scheduler is a cron job scheduler for periodic tasks, not an event router; it cannot react to real-time object creation events in a GCS bucket.

Full explanation →

544

Matchingmedium

Match each machine learning term to its description.

Drag a concept onto its matching description — or click a concept then click the description.

Concepts

Matches

Model trained on labeled data

Model trained on unlabeled data

Agent learns by interacting with environment

Model performs well on training data but poorly on new data

Why these pairings

In machine learning, supervised learning uses labeled data (inputs mapped to known outputs), unsupervised learning finds patterns in unlabeled data, features are input variables, and labels are the target outputs. Common confusions include mixing supervised/unsupervised definitions or swapping features and labels.

Full explanation →

545

Multi-Selecthard

Which THREE best practices should be followed when designing a Dataflow pipeline for real-time data processing?

Select 3 answers

A.Set up monitoring alerts for system lag and data freshness.

B.Use static side inputs that are loaded once at pipeline start.

C.Implement watermark estimation to handle late data.

D.Use global windows with early triggers for low latency.

E.Use idempotent sinks to ensure exactly-once processing.

AnswersA, C, E

Monitoring is critical for streaming pipelines.

Why this answer

Monitoring alerts for system lag and data freshness are essential for maintaining operational visibility in real-time Dataflow pipelines. System lag (the time between data ingestion and processing) and data freshness (how current the processed output is) directly impact the pipeline's ability to meet latency SLAs. Without these alerts, issues like worker backpressure or Pub/Sub subscription backlog can go unnoticed, leading to stale or lost data.

Exam trap

Google Cloud often tests the misconception that static side inputs are acceptable for streaming pipelines, but they are only appropriate for batch or bounded data; real-time pipelines require side inputs that can be periodically refreshed (e.g., via a streaming source or a periodic lookup).

Full explanation →

546

MCQmedium

A company wants to train a machine learning model to predict customer churn using BigQuery ML. The dataset has a severe class imbalance (only 2% churn). Which approach should the data engineer take to handle this imbalance within BigQuery ML?

A.Use SMOTE directly in BigQuery SQL before training

B.Set the CLASS_WEIGHTS option to 'balanced' in the CREATE MODEL statement

C.Create a custom Vertex AI model using TensorFlow and use the class_weight parameter

D.Oversample the minority class using a SQL query that duplicates rows

AnswerB

BigQuery ML's CLASS_WEIGHTS option can be set to 'balanced' to automatically compute weights inversely proportional to class frequencies.

Why this answer

BigQuery ML supports class weights via the CLASS_WEIGHTS option in CREATE MODEL, which adjusts the loss function to penalize misclassifications of the minority class more heavily.

Full explanation →

547

Multi-Selecthard

A data analyst wants to compute the rank of sales per region and also the difference in sales between consecutive months for each region. Which BigQuery analytic functions should they use? (Select TWO)

Select 2 answers

A.RANK()

B.ROW_NUMBER()

C.LAG()

D.LEAD()

E.NTILE()

AnswersA, C

RANK() computes the rank of sales per region.

Why this answer

RANK() computes the rank of rows within a partition. LAG() accesses data from a previous row in the same result set, which can be used to compute differences. ROW_NUMBER() assigns unique sequential integers, not rank.

NTILE() distributes rows into buckets. LEAD() accesses next row, not previous.

Full explanation →

548

Multi-Selectmedium

You want to optimize BigQuery costs for a large dataset that is frequently queried by time range. You also need to ensure that predictable workloads have dedicated slot capacity. Which TWO strategies should you combine? (Choose 2)

Select 2 answers

A.Use query caching

B.Partition the table by date

C.Purchase committed use reservations for baseline capacity

D.Create a materialized view for the entire table

E.Enable autoscaling slots

AnswersB, C

Partitioning by date limits the bytes scanned per query, reducing cost.

Why this answer

Partitioned tables reduce bytes scanned by time-range queries. Committed use reservations provide predictable slot capacity at a discount. Autoscaling does not provide dedicated capacity; caching is automatic.

Full explanation →

549

MCQmedium

You are migrating an existing Kafka cluster to Google Cloud using Dataproc. The cluster handles high-throughput streaming data with strict ordering requirements per partition. Which choice of Dataproc configuration is most appropriate?

A.Use Dataflow with Kafka IO instead of Dataproc.

B.Use Dataproc with local SSDs for better performance, and enable autoscaling.

C.Use Dataproc with preemptible workers to reduce cost, and attach standard persistent disks.

D.Use Dataproc with non-preemptible workers and persistent SSD storage for brokers.

AnswerD

Non-preemptible workers provide stability for Kafka brokers, and SSDs offer low latency for high-throughput streaming.

Why this answer

Kafka brokers in a Dataproc cluster require persistent, non-preemptible workers to maintain data durability and strict ordering per partition. Preemptible workers can be terminated at any time, causing data loss or rebalancing that violates ordering guarantees. Persistent SSD storage provides the low-latency I/O needed for high-throughput Kafka workloads, while non-preemptible instances ensure broker stability and consistent replication.

Exam trap

Google Cloud often tests the misconception that preemptible VMs or local SSDs are acceptable for stateful, ordered workloads like Kafka, when in fact they violate durability and ordering guarantees due to ephemeral storage and abrupt termination.

How to eliminate wrong answers

Option A is wrong because Dataflow with Kafka IO is a serverless stream processing service, not a Kafka cluster migration target; the question asks about migrating an existing Kafka cluster to Dataproc, not replacing it with a different processing paradigm. Option B is wrong because local SSDs are ephemeral and lose data on instance termination, which is incompatible with Kafka's durability and ordering requirements; autoscaling can cause partition rebalancing that disrupts strict ordering. Option C is wrong because preemptible workers can be terminated at any time, leading to data loss and partition leader re-elections that break ordering guarantees; standard persistent disks have higher latency than SSDs, degrading Kafka's throughput.

Full explanation →

550

Multi-Selectmedium

A company wants to use BigQuery ML to build a recommendation system for movies. The data includes user IDs, movie IDs, and ratings. Which BigQuery ML model types are suitable for this? (Select TWO)

Select 2 answers

A.ARIMA_PLUS

B.k-means

C.AutoML Tables

D.Boosted tree classifier

E.Matrix factorization

AnswersD, E

Boosted trees can be used to predict ratings as a classification problem.

Why this answer

Matrix factorization (via implicit or explicit feedback) is specifically designed for recommendation systems. Boosted tree classifiers can also be used for predicting ratings as a classification problem. AutoML Tables is not a model type in BigQuery ML.

ARIMA_PLUS is for time-series. k-means is for clustering, not recommendations directly.

Full explanation →

551

Multi-Selecthard

Your company runs a Dataflow streaming pipeline that processes user activity from Pub/Sub and writes aggregated results to BigQuery. Lately, the pipeline is experiencing high latency and backlog growth during peak hours. You need to troubleshoot and improve performance. Which THREE actions should you take? (Choose 3.)

Select 3 answers

A.Change the worker machine type to a higher CPU/memory configuration

B.Decrease the window duration to reduce data per window

C.Enable Dataflow Streaming Engine

D.Increase the number of workers in the pipeline

E.Add additional Pub/Sub subscriptions to the same topic

AnswersA, C, D

More CPU/memory per worker can speed up processing if the transform is compute-intensive.

Why this answer

Increasing the number of workers allows the pipeline to process more data in parallel. Using streaming engine can improve throughput and reduce latency by offloading state management. Adjusting the worker machine type to use more CPU/memory can help if the processing is compute-intensive.

Adding more subscriptions would not help because the pipeline reads from a single subscription. Changing the window size affects business logic but not necessarily performance. Combining these three optimizations addresses common bottlenecks.

Full explanation →

552

MCQeasy

A streaming Dataflow job is processing messages from Cloud Pub/Sub. The job is underutilizing resources and the throughput is lower than expected. Which parameter should be adjusted to increase parallelism?

A.Change the workerMachineType to a higher CPU machine

B.Increase the number of workers via maxNumWorkers

C.Set the streaming engine to Dataflow Streaming Engine

D.Set autoscalingAlgorithm to THROUGHPUT_BASED

AnswerB

More workers allow more parallelism.

Why this answer

The job is underutilizing resources, meaning the existing workers are not fully loaded. Increasing the number of workers via maxNumWorkers directly increases parallelism by allowing Dataflow to distribute work across more VMs, which can increase throughput without changing the per-worker resource profile. This parameter controls the upper bound on the number of workers, enabling the autoscaler to scale out when there is backlog.

Exam trap

Google Cloud often tests the misconception that increasing per-worker resources (CPU/memory) is the primary way to improve throughput in a streaming job, when in fact underutilization indicates the need to scale out workers rather than scale up individual workers.

How to eliminate wrong answers

Option A is wrong because changing workerMachineType to a higher CPU machine increases per-worker compute capacity but does not address underutilization; if workers are idle, adding more CPU per worker will not increase parallelism or throughput. Option C is wrong because Dataflow Streaming Engine is a service that offloads shuffle and state management to the backend, reducing per-worker overhead and improving scalability, but it does not directly increase parallelism; it changes the execution model. Option D is wrong because setting autoscalingAlgorithm to THROUGHPUT_BASED is already the default for streaming jobs; it enables autoscaling based on throughput metrics, but without adjusting maxNumWorkers, the autoscaler cannot scale beyond the default limit, so throughput remains capped.

Full explanation →

553

MCQhard

A healthcare startup is deploying a natural language processing (NLP) model for extracting medical entities from clinical notes. The model is a fine-tuned BERT model served on Vertex AI Prediction using a custom container. The team observes that prediction latency is around 500ms per request, but they need to handle up to 100 requests per second (QPS) with end-to-end latency under 200ms. The model currently runs on n1-standard-4 machines (4 vCPU, 15 GB memory). During load testing, CPU utilization reaches 90% and memory usage is 12 GB. The team is considering options to meet the requirements. Which action should they take?

A.Use a machine type with a GPU, such as n1-standard-4 with a NVIDIA Tesla T4 accelerator, and optimize the model with TensorRT.

B.Switch to n1-highmem-4 machines to provide more memory for the model.

C.Deploy the model using TensorFlow Serving with CPU-only nodes and increase the number of replicas.

D.Move the model to Cloud Run with automatic scaling to handle the QPS.

AnswerA

GPU accelerates BERT inference and TensorRT further optimizes latency.

Why this answer

The bottleneck is CPU-bound inference (90% CPU utilization) with memory well within limits (12 GB of 15 GB). Adding a GPU (NVIDIA Tesla T4) and optimizing with TensorRT reduces per-request latency via hardware acceleration and graph optimizations, enabling sub-200ms inference at 100 QPS. This directly addresses the latency requirement without changing the machine family or scaling strategy.

Exam trap

Google Cloud often tests the misconception that scaling horizontally (more replicas or Cloud Run) solves latency problems, when the real issue is per-request compute bottleneck that requires hardware acceleration or model optimization.

How to eliminate wrong answers

Option B is wrong because memory is not the bottleneck (12 GB used out of 15 GB); increasing memory does not reduce CPU-bound inference latency. Option C is wrong because TensorFlow Serving on CPU-only nodes still relies on CPU compute, and increasing replicas adds cost and complexity without addressing the fundamental latency per request; the CPU utilization is already saturated, so more replicas would require horizontal scaling but still not guarantee sub-200ms latency per request. Option D is wrong because Cloud Run's automatic scaling handles QPS but does not reduce per-request latency; the model's inference time remains CPU-bound, and Cloud Run's cold starts and CPU-only instances would not meet the 200ms latency target.

Full explanation →

554

MCQmedium

A company wants to use dbt to transform data in BigQuery. Their source data is loaded daily into staging tables. They need to run dbt transformations on a schedule and only process tables that have changed. Which dbt feature should they use?

A.dbt snapshots

B.dbt incremental models

C.dbt seeds

D.dbt sources

AnswerB

Incremental models only process new/changed records, reducing cost and runtime.

Why this answer

dbt incremental models allow processing only new or changed records based on a configured timestamp or unique key. dbt snapshots capture historical changes. dbt seeds load CSV files. dbt sources are for configuration, not incremental processing.

Full explanation →

555

Multi-Selectmedium

Your company uses Cloud Composer to orchestrate a data pipeline that includes Dataproc Spark jobs and BigQuery load operations. You need to pass the output file path from the Spark job to the next BigQuery task in the DAG. Which two mechanisms can you use to share data between tasks? (Choose TWO.)

Select 2 answers

A.Store the output path as a Cloud Composer variable.

B.Publish the output path to a Pub/Sub topic and subscribe in the next task.

C.Write the output path to a Cloud Storage object and read it in the next task.

D.Use BigQuery as an intermediary to store the output path.

E.Use Airflow XComs to push the output path from the Spark task and pull it in the BigQuery task.

AnswersC, E

Cloud Storage is a durable store that both tasks can access.

Why this answer

Airflow XComs allow tasks to exchange small amounts of data (e.g., file paths) by pushing and pulling values. Cloud Storage can be used as an intermediate store: the Spark job writes output to GCS, and the BigQuery task reads from that location. BigQuery does not directly communicate with Dataproc.

Cloud Composer variables are for global configuration, not task-to-task. Pub/Sub is not needed for simple file path sharing.

Full explanation →

556

MCQmedium

A data engineering team needs to process a large volume of CSV files stored in Cloud Storage using Dataproc. The files are generated hourly and each contains millions of rows. They want to minimize the number of Dataproc cluster nodes to reduce cost while processing within an hour. Which configuration should they recommend?

A.Use a cluster with preemptible worker nodes only.

B.Use a cluster with local SSDs for temporary storage.

C.Use a cluster with a few large worker nodes and use Spark static allocation.

D.Use a cluster with many small worker nodes and use Spark dynamic allocation.

AnswerD

Dynamic allocation adjusts resources based on workload; small nodes provide granular scaling.

Why this answer

Using many small worker nodes with Spark dynamic allocation allows the cluster to scale resources precisely to the workload, minimizing idle capacity and cost. Dynamic allocation enables executors to be added or removed based on the processing demands of the hourly CSV files, ensuring the job completes within the hour without over-provisioning nodes.

Exam trap

Google Cloud often tests the misconception that larger nodes are always more cost-effective for big data processing, but in practice, many small nodes with dynamic allocation reduce idle resource waste and better match the parallelism needs of distributed file processing.

How to eliminate wrong answers

Option A is wrong because preemptible worker nodes only can be terminated at any time by Google Cloud, risking job failure or delays when processing millions of rows per hour, and they cannot be the sole worker nodes for a reliable Dataproc cluster. Option B is wrong because local SSDs improve I/O performance for shuffle operations but do not directly reduce the number of nodes or cost; they add cost per node and are not a configuration for minimizing node count. Option C is wrong because using a few large worker nodes with Spark static allocation reserves a fixed number of executors regardless of actual workload, leading to underutilization and higher cost if the job does not need all resources, and it does not adapt to the hourly data volume variations.

Full explanation →

557

MCQmedium

A company uses Cloud Composer to orchestrate a daily ETL pipeline that includes multiple Dataproc jobs. The pipeline processes sensitive financial data. The security team requires that all data in transit be encrypted, and all Cloud Storage buckets used by the pipeline should have uniform bucket-level access enabled and VPC Service Controls. The pipeline currently uses a single Cloud Composer environment in us-east1. The Dataproc clusters are created using the standard image and use custom service accounts with minimal permissions. The pipeline runs successfully during testing, but in production, the Dataproc jobs fail with 'Access Denied' errors when trying to write to a Cloud Storage bucket. The bucket has uniform bucket-level access enabled and is inside a VPC Service Controls perimeter. The Dataproc service account has the Storage Object Admin role at the project level. What is the most likely cause of the access denied error?

A.The service account does not have the Storage Object Admin role on the bucket.

B.Data in transit encryption is not enabled for the Cloud Storage bucket.

C.Uniform bucket-level access prevents writes from service accounts.

D.The Dataproc cluster is not in the VPC Service Controls perimeter.

AnswerD

VPC Service Controls deny access from resources outside the perimeter.

Why this answer

The Dataproc cluster is created outside the VPC Service Controls perimeter, so even though the service account has the Storage Object Admin role at the project level, requests from the cluster are blocked by the perimeter's ingress/egress rules. VPC Service Controls enforce a security boundary that prevents resources outside the perimeter from accessing protected services like Cloud Storage, regardless of IAM permissions. The 'Access Denied' error in production, despite successful testing, strongly indicates a perimeter configuration mismatch.

Exam trap

Google Cloud often tests the distinction between IAM permissions and VPC Service Controls boundaries, tricking candidates into thinking a project-level IAM role is sufficient when the real blocker is network-level perimeter enforcement.

How to eliminate wrong answers

Option A is wrong because the service account has the Storage Object Admin role at the project level, which grants write access to all buckets in the project, including this one; uniform bucket-level access does not override project-level IAM roles. Option B is wrong because data in transit encryption is automatically enforced by Google Cloud for all API calls to Cloud Storage (using HTTPS/TLS), and the question states the pipeline already encrypts data in transit, so this is not the cause of the error. Option C is wrong because uniform bucket-level access does not prevent writes from service accounts; it simply disables ACLs and requires all access decisions to be made via IAM policies, which the service account already has via its project-level role.

Full explanation →

558

Multi-Selectmedium

Which TWO actions should be taken to optimize a Dataflow streaming pipeline that is experiencing high system lag and backpressure? (Choose two.)

Select 2 answers

A.Use a higher memory machine type for all workers.

B.Increase the number of worker threads by adjusting the streaming worker's parallelism hint.

C.Enable autoscaling and increase the maximum number of workers.

D.Reduce the number of workers to decrease cost.

E.Set maxNumWorkers to 1 to force single-worker processing.

AnswersB, C

More threads can increase throughput per worker.

Why this answer

Increasing the parallelism hint allows each worker to process more bundles concurrently, which can reduce backpressure by improving throughput without adding more workers. Option C is correct because enabling autoscaling and increasing the maximum number of workers allows the pipeline to dynamically scale out to handle increased load, directly mitigating high system lag and backpressure.

Exam trap

Google Cloud often tests the misconception that simply adding more memory or reducing workers will solve backpressure, when in fact the correct approaches involve increasing parallelism or scaling out the worker pool.

Full explanation →

559

MCQeasy

A company wants to stream real-time clickstream data from a website into BigQuery for near-real-time analytics. They expect peaks of 10,000 events per second. Which combination of services is most suitable for ingestion?

A.Cloud Storage → Cloud Functions → BigQuery

B.Direct Web → Dataflow → BigQuery

C.Pub/Sub → Dataflow → BigQuery (Storage Write API)

D.Pub/Sub → Dataflow → BigQuery (legacy streaming inserts)

AnswerC

This is the modern recommended architecture: Pub/Sub for ingestion, Dataflow for processing, Storage Write API for high-throughput streaming ingestion into BigQuery.

Why this answer

Pub/Sub is designed for high-throughput event ingestion, Dataflow provides real-time stream processing, and the BigQuery Storage Write API offers exactly-once semantics and high throughput. Therefore, option C is the most suitable. Option A uses Cloud Functions, which are not designed for high-throughput streaming (10,000 events per second would likely exceed typical limits).

Option B sends data directly from the web to Dataflow without a buffer, which could lead to data loss during spikes; Pub/Sub provides a durable buffer. Option D uses legacy streaming inserts, which are deprecated and have lower throughput and no exactly-once guarantees.

Full explanation →

560

MCQeasy

Which Google Cloud service is a serverless, highly scalable data warehouse for analytical queries, supporting SQL and integration with BI tools?

A.Firestore

B.Cloud SQL

C.Cloud Spanner

D.BigQuery

AnswerD

Correct: BigQuery is the serverless analytics warehouse.

Why this answer

BigQuery is a serverless, highly scalable data warehouse designed for analytical queries over large datasets. It supports standard SQL and integrates seamlessly with BI tools like Looker and Tableau, making it the correct choice for this use case.

Exam trap

The trap here is that candidates may confuse Google Cloud Spanner's global scale and SQL support with data warehousing capabilities, but Spanner is optimized for transactional consistency, not analytical query performance or BI tool integration.

How to eliminate wrong answers

Option A is wrong because Firestore is a NoSQL document database for mobile and web app development, not a data warehouse for analytical SQL queries. Option B is wrong because Cloud SQL is a fully managed relational database for OLTP workloads, not a serverless data warehouse optimized for large-scale analytics. Option C is wrong because Cloud Spanner is a globally distributed, strongly consistent relational database service for transactional workloads, not a data warehouse designed for analytical queries and BI integration.

Full explanation →

561

MCQhard

A data engineer needs to alert when Pub/Sub subscription has messages older than 1 hour. Which Cloud Monitoring metric and filter should they use?

A.Metric: topic/send_message_operation_count; filter: topic_id

B.Metric: subscription/ack_message_count; filter: subscription_id

C.Metric: subscription/num_undelivered_messages; filter: subscription_id

D.Metric: subscription/oldest_unacked_message_age; filter: subscription_id

AnswerD

Correct metric and filter for alerting on message age.

Why this answer

The metric subscription/oldest_unacked_message_age gives the age of the oldest unacknowledged message. Filtering by subscription ID targets the specific subscription.

Full explanation →

562

Drag & Dropmedium

Drag and drop the steps to set up a Pub/Sub topic with a push subscription to an HTTPS endpoint into the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps

Order

1Step 1

2Step 2

3Step 3

4Step 4

Why this order

The correct order to set up a Pub/Sub topic with a push subscription to an HTTPS endpoint is to first create the topic, then create a push subscription specifying the HTTPS endpoint URL. This ensures the topic exists before the subscription references it, and the push endpoint is configured at subscription creation time. Common mistakes include creating the subscription before the topic, omitting the endpoint, or attempting to convert a pull subscription to push, which is not supported.

Full explanation →

563

Multi-Selecthard

You are designing a streaming pipeline that must guarantee exactly-once processing. Which three services or features can help achieve this? (Choose THREE.)

Select 3 answers

A.Cloud Functions for post-processing

B.BigQuery streaming inserts with a unique key for deduplication

C.Cloud Spanner for deduplication state across the pipeline

D.Cloud Pub/Sub with duplicate detection (using message IDs)

E.Dataflow with idempotent write operations to BigQuery

AnswersC, D, E

Using Cloud Spanner as a global state store allows tracking processed event IDs for deduplication.

Why this answer

Cloud Spanner is correct because it provides globally distributed, strongly consistent transactions that can be used to maintain deduplication state across the entire streaming pipeline. By storing a unique key for each processed event in Spanner, the pipeline can atomically check and record whether an event has already been handled, ensuring exactly-once semantics even in the face of retries or failures.

Exam trap

Google Cloud often tests the misconception that BigQuery streaming inserts can guarantee exactly-once processing via a unique key, when in fact BigQuery only supports at-least-once delivery and requires external deduplication mechanisms like Cloud Spanner or Dataflow with idempotent writes.

Full explanation →

564

Drag & Dropmedium

Drag and drop the steps to set up Cloud IAP (Identity-Aware Proxy) for an App Engine app into the correct order.

Drag steps to the numbered slots on the right, or tap a step then tap a slot.

Steps

Order

1Step 1

2Step 2

3Step 3

4Step 4

Why this order

IAP verifies identity and authorization before allowing access to the application.

Full explanation →

565

MCQeasy

A company processes CSV files that are uploaded to Cloud Storage by external partners. Each file is around 500 MB, and they need to be parsed and loaded into BigQuery. The processing must start as soon as the file arrives. What is the most efficient serverless architecture?

A.Cloud Storage triggers a Cloud Function that publishes events to Pub/Sub; a Dataflow streaming pipeline reads from Pub/Sub and writes to BigQuery.

B.Use Cloud Scheduler to periodically check for new files and process them with Dataflow batch jobs.

C.Cloud Storage triggers a Dataproc job that reads the file and loads it into BigQuery.

D.Cloud Storage triggers a Cloud Function that directly loads the data into BigQuery using the BigQuery API.

AnswerA

Serverless and scales well with file uploads.

Why this answer

It combines Cloud Storage event-driven triggers with Pub/Sub for reliable asynchronous message delivery, and uses Dataflow streaming with autoscaling to handle 500 MB files efficiently. This serverless architecture ensures processing starts immediately upon file arrival, scales to handle large files without manual intervention, and leverages BigQuery's streaming inserts for near-real-time data loading.

Exam trap

Google Cloud often tests the misconception that Cloud Functions can handle large file processing directly, but the 9-minute timeout and memory limits make them unsuitable for files over a few hundred MB, pushing candidates toward the seemingly simpler Option D.

How to eliminate wrong answers

Option B is wrong because Cloud Scheduler polling introduces latency and inefficiency, as it checks for new files on a fixed schedule rather than reacting instantly, which violates the requirement that processing must start as soon as the file arrives. Option C is wrong because Dataproc is a managed Hadoop/Spark service that requires cluster provisioning and startup time, adding overhead for a simple CSV-to-BigQuery load; it is not serverless and not the most efficient for this use case. Option D is wrong because Cloud Functions have a 9-minute timeout and 2 GB memory limit, making them unsuitable for parsing and loading a 500 MB CSV file directly via the BigQuery API, which would likely exceed these constraints and cause failures.

Full explanation →

566

MCQmedium

A company needs to process streaming sensor data and run both real-time analytics and batch reanalysis on historical data. They want to minimize infrastructure management. Which architecture and service combination is MOST suitable?

A.Kappa architecture with Pub/Sub and Dataflow for both real-time and batch processing

B.Lambda architecture with Pub/Sub for streaming and Cloud Storage for batch, processed by Dataflow

C.Batch processing only with Dataflow and Cloud Storage, ignoring real-time needs

D.Kappa architecture with Pub/Sub Lite and Dataflow Serverless

AnswerA

Kappa architecture uses a single streaming pipeline, and Dataflow can replay from Pub/Sub for batch reanalysis, minimizing management.

Why this answer

Kappa architecture processes all data as a stream, avoiding separate batch/speed layers. Pub/Sub ingests streaming data, and Dataflow with Apache Beam can handle both real-time and batch (replay) pipelines, minimizing infrastructure management.

Full explanation →

567

Multi-Selecteasy

A company needs to store and analyze large amounts of unstructured data (images, videos) and structured data (CSV logs) in a cost-effective manner. The data should be accessible for analytics with BigQuery. Which two services should they use? (Choose TWO.)

Select 2 answers

A.Cloud SQL

B.Cloud Spanner

C.BigQuery

D.Cloud Storage

E.Firestore

AnswersC, D

BigQuery can query data stored in Cloud Storage via external tables, enabling analytics.

Why this answer

Cloud Storage is the best option for storing unstructured and structured files cost-effectively. BigQuery can analyze this data directly via external tables or after loading, making it a powerful analytics platform.

Full explanation →

568

MCQhard

An IoT application writes sensor readings to Cloud Bigtable with a row key of 'deviceID#timestamp'. The team notices high write latency and hotspots on a few nodes. Which row key design change would most likely improve performance?

A.Add a random prefix to the row key (e.g., hash of deviceID modulo 1000)

B.Reverse the key to 'timestamp#deviceID'

C.Use a single column family with many columns

D.Store all data in one row per device

AnswerA

Hashing the device ID distributes writes across tablet servers, reducing hotspots.

Why this answer

Prefixing timestamps can cause hotspots because writes go to the same tablet server for the same time range. Hashing the device ID or using a field-leveled design (e.g., deviceID inverted) distributes writes across nodes. Adding a random prefix helps but salting with a hash is more systematic.

Full explanation →

569

MCQmedium

You are monitoring a Dataproc cluster and notice that the cluster utilisation is high, but jobs are running slowly. The cluster uses preemptible workers for cost savings. What is the most likely cause of the performance degradation?

A.The primary workers are using standard disks instead of SSDs.

B.The preemptible workers are being preempted frequently, causing task retries and slowdowns.

C.The cluster is under-provisioned; increase the number of preemptible workers.

D.The cluster is using an older image version; upgrade to the latest.

AnswerB

Preemptible workers have a high chance of termination, which affects job performance.

Why this answer

Preemptible workers can be terminated at any time, causing job restarts and slower execution. The high utilisation indicates many workers are preempted, leading to recomputation.

Full explanation →

570

MCQmedium

A company runs Apache Kafka on Dataproc for real-time event streaming. They want to archive the Kafka topics to Cloud Storage for long-term retention and later analysis in BigQuery. Which approach is the most cost-effective and operationally simple?

A.Use Apache Spark streaming on Dataproc to read from Kafka and write to GCS

B.Use Kafka MirrorMaker to replicate topics to a second cluster that writes to GCS

C.Use the Pub/Sub connector to publish Kafka messages to Pub/Sub, then a Dataflow job to write to GCS

D.Use Kafka Connect with the GCS Sink Connector to write directly to Cloud Storage

AnswerD

Kafka Connect GCS Sink Connector is purpose-built, simple to configure, and runs on the same Dataproc cluster.

Why this answer

Kafka Connect with the GCS Sink Connector is purpose-built for exactly this use case: it directly streams Kafka topics to Cloud Storage in Avro, Parquet, or JSON format without requiring intermediate processing clusters or services. This approach minimizes operational overhead (no Spark or Dataflow jobs to manage) and is cost-effective since it runs as a lightweight connector within the existing Kafka ecosystem, leveraging Dataproc's managed Kafka cluster.

Exam trap

A common mistake in Google exams is to think that streaming data to Cloud Storage requires a full streaming pipeline (Spark, Dataflow) or an intermediary service like Pub/Sub, when in fact Kafka Connect provides a native, lightweight, and cost-effective sink directly to GCS.

How to eliminate wrong answers

Option A is wrong because using Apache Spark streaming on Dataproc to read from Kafka and write to GCS introduces unnecessary compute overhead, latency, and operational complexity (managing Spark jobs, checkpointing, and resource scaling) compared to a direct connector. Option B is wrong because Kafka MirrorMaker is designed for cross-cluster replication, not for writing to GCS; it would require an additional sink to write to GCS, adding complexity and cost without any benefit. Option C is wrong because routing Kafka messages through Pub/Sub adds latency, extra cost (Pub/Sub egress and Dataflow processing), and operational complexity (managing a Pub/Sub topic, subscription, and Dataflow pipeline) when a direct connector to GCS exists.

Full explanation →

571

MCQmedium

A gaming company uses Avro schemas for its streaming event data. They anticipate adding new optional fields to events over time. They need to ensure backward compatibility so that existing pipelines continue to work. Which strategy should they adopt?

A.Use Avro with a schema registry that enforces backward-compatible changes

B.Use JSON instead of Avro and ignore unknown fields

C.Use Protocol Buffers with breaking changes

D.Use FlatBuffers for performance

AnswerA

Avro's schema evolution rules allow adding optional fields without breaking existing consumers, and a schema registry enables version management.

Why this answer

Avro, combined with a schema registry, allows schema evolution with backward compatibility. The registry enforces rules such as adding optional fields with defaults, ensuring that consumers using older schemas can still deserialize new data without breaking. This directly addresses the requirement for existing pipelines to continue working as new optional fields are added.

Exam trap

Google Cloud often tests the misconception that any serialization format (like JSON or Protocol Buffers) inherently supports backward compatibility, but the key is the combination of a schema registry with enforced evolution rules, which only Avro explicitly provides in this context.

How to eliminate wrong answers

Option B is wrong because JSON lacks a schema enforcement mechanism; while ignoring unknown fields is possible, JSON does not provide built-in compatibility guarantees or schema evolution rules, making it error-prone in large-scale streaming systems. Option C is wrong because Protocol Buffers can support backward compatibility, but the option specifies 'breaking changes,' which would violate the requirement for backward compatibility. Option D is wrong because FlatBuffers prioritize performance (zero-copy deserialization) but do not inherently enforce backward-compatible schema evolution, and they are less suited for streaming event data with frequent schema changes.

Full explanation →

572

Multi-Selecthard

Which THREE steps are essential for implementing a continuous training pipeline with Vertex AI?

Select 3 answers

A.If the new model passes evaluation, deploy it to a production endpoint.

B.Manually approve each new model version before deployment.

C.Deploy the original model once and set it to auto-update.

D.Set up a trigger to start a training pipeline when new training data is available (e.g., via Cloud Storage events).

E.Include a step in the pipeline that evaluates the new model against a validation set.

AnswersA, D, E

Automated deployment upon passing evaluation completes the continuous pipeline.

Why this answer

A continuous training pipeline aims to automate model updates. After a new model is trained and evaluated, deploying it to a production endpoint (e.g., using Vertex AI Endpoints) is the essential final step to serve predictions from the improved model. This completes the automation loop without manual intervention, assuming the evaluation passes predefined thresholds.

Exam trap

Candidates often mistakenly include manual approval (B) as essential in a continuous training pipeline, or believe models can auto-update (C) without explicit pipeline steps. For Vertex AI, the required steps are triggering via events, evaluation, and automated deployment upon passing checks.

Full explanation →

573

MCQhard

A data pipeline ingests real-time events from Cloud Pub/Sub into BigQuery using Dataflow. The pipeline uses a sliding window of 5 minutes with a 1-minute period to aggregate event counts. Recently, the pipeline started failing with 'The worker failed to provide a heartbeat.' The Dataflow logs show high CPU usage on the workers. What is the best course of action to resolve the issue?

A.Increase the number of workers and enable autoscaling to distribute the load.

B.Reduce the number of workers to minimize coordination overhead.

C.Use a global window with a trigger to reduce state size.

D.Change the windowing to a fixed 5-minute window to reduce computations.

AnswerA

More workers can handle the CPU load from streaming inserts.

Why this answer

The 'worker failed to provide a heartbeat' error combined with high CPU usage indicates that workers are overloaded and cannot process data fast enough to maintain their heartbeat to the Dataflow service. Increasing the number of workers and enabling autoscaling distributes the computational load across more machines, reducing per-worker CPU pressure and allowing heartbeats to be sent on time. This directly addresses the root cause of resource exhaustion.

Exam trap

Google Cloud often tests the misconception that reducing workers or changing window types is a universal fix for resource exhaustion, when in fact the immediate solution for heartbeat failures due to high CPU is to scale out the worker pool.

How to eliminate wrong answers

Option B is wrong because reducing the number of workers would concentrate the same workload on fewer machines, increasing per-worker CPU usage and worsening the heartbeat failure. Option C is wrong because using a global window with a trigger does not reduce state size for sliding windows; it would accumulate all events into a single unbounded window, potentially increasing memory pressure and CPU overhead. Option D is wrong because changing to a fixed 5-minute window does not reduce computations compared to a sliding window with a 1-minute period; it actually changes the semantics (non-overlapping windows) and may still cause high CPU if the underlying load is unchanged.

Full explanation →

574

MCQmedium

A team notices that the latency for online predictions from a Vertex AI endpoint has increased significantly over the past hour. The model is a large TensorFlow model deployed with automatic scaling (minReplicaCount=2, maxReplicaCount=10). The CPU utilization of the deployed instances is consistently above 85%. What is the most likely cause of the increased latency?

A.The network latency between the client and the endpoint has increased due to regional issues.

B.The model is deployed with GPU acceleration, but the instances are using incorrect CUDA drivers.

C.The model is too large for the instance memory, causing disk swapping.

D.The model is CPU-bound, and the current replicas are saturated, causing queuing.

AnswerD

High CPU utilization indicates the replicas are at capacity, leading to request queuing and higher latency.

Why this answer

The consistently high CPU utilization (above 85%) indicates that the existing replicas are saturated, unable to process incoming requests quickly enough. When all replicas are busy, new requests are queued, which directly increases latency. Automatic scaling can add more replicas up to maxReplicaCount=10, but if the scaling is slow or the traffic spike is sudden, queuing occurs first, causing the observed latency increase.

Exam trap

Google Cloud often tests the distinction between symptoms of CPU saturation (queuing/latency) versus memory or GPU issues; the trap here is that candidates may incorrectly attribute latency to network or hardware driver problems when the clear indicator is sustained high CPU utilization on existing instances.

How to eliminate wrong answers

Option A is wrong because network latency between client and endpoint is not indicated by CPU utilization of deployed instances; regional network issues would affect all requests uniformly, not correlate with high CPU. Option B is wrong because incorrect CUDA drivers would cause GPU-related errors or failures, not consistently high CPU utilization; the model would likely fail to run or produce errors, not just increase latency. Option C is wrong because disk swapping due to insufficient memory would manifest as high disk I/O and memory pressure, not primarily high CPU utilization; the symptom described is CPU-bound, not memory-bound.

Full explanation →

575

MCQeasy

A company wants to implement a data lake on Google Cloud to store raw sensor data (unstructured binary files) and allow data scientists to run SQL queries on processed data. They expect to store terabytes of data and have different access patterns. Which combination of GCP services best meets these requirements?

A.Bigtable for raw data and Cloud Spanner for processed data

B.Cloud Storage for both raw and processed data

C.Cloud SQL for raw data and Cloud Dataproc for processing

D.Cloud Storage for raw data and BigQuery for processed data

AnswerD

Cloud Storage stores any file type cost-effectively, and BigQuery provides fast SQL queries on structured data.

Why this answer

Cloud Storage is the ideal service for storing raw, unstructured binary sensor data at petabyte scale, offering low-cost, durable object storage with multiple access tiers. BigQuery is a serverless, highly scalable data warehouse that allows data scientists to run SQL queries on processed data, with features like columnar storage and automatic optimization for analytical workloads. This combination directly addresses the need for raw storage and SQL-based analytics on processed data.

Exam trap

Google Cloud often tests the misconception that Cloud Storage can serve as a queryable database for SQL, when in fact it requires an external query engine like BigQuery or Dataproc for SQL access.

How to eliminate wrong answers

Option A is wrong because Bigtable is a NoSQL wide-column database optimized for real-time, low-latency access, not for storing raw unstructured binary files, and Cloud Spanner is a globally distributed relational database for transactional workloads, not for analytical SQL queries on processed data. Option B is wrong because while Cloud Storage can store both raw and processed data, it does not natively support SQL queries; data scientists would need an additional service like BigQuery or Dataproc to run SQL. Option C is wrong because Cloud SQL is a relational database for structured data, not designed for raw unstructured binary files, and Cloud Dataproc is a managed Spark/Hadoop service for processing, not a SQL query engine for processed data.

Full explanation →

576

MCQeasy

A data engineer wants to automatically move objects from Standard storage class to Nearline after 30 days, and then to Archive after 365 days. Which Cloud Storage feature should they configure?

A.Object Versioning

B.Retention Policy

C.Bucket Lock

D.Object Lifecycle rule with SetStorageClass actions

AnswerD

Lifecycle rules can change storage class based on object age.

Why this answer

Object Lifecycle rules in Google Cloud Storage allow you to automatically transition objects between storage classes (e.g., from Standard to Nearline after 30 days, then to Archive after 365 days) using the SetStorageClass action. This feature is specifically designed for automated lifecycle management, including deletion and class transitions, based on object age or other conditions.

Exam trap

Google often tests the distinction between lifecycle management (which changes storage classes) and retention/versioning features (which protect data but do not automate class transitions), leading candidates to confuse Object Versioning or Retention Policy with lifecycle rules.

How to eliminate wrong answers

Option A is wrong because Object Versioning is a feature that preserves non-current object versions to protect against accidental deletion or overwriting; it does not automate storage class transitions. Option B is wrong because Retention Policy is used to enforce a minimum retention period on objects, preventing deletion or modification, but it cannot change storage classes over time. Option C is wrong because Bucket Lock is a mechanism to permanently lock a retention policy, making it immutable; it does not provide any lifecycle-based storage class transitions.

Full explanation →

577

MCQmedium

A company uses Cloud Composer to orchestrate data pipelines. One DAG fails intermittently with the error: 'Task received SIGTERM signal.' The task runs a long-running Dataproc job. What is the most likely cause?

A.The Dataproc cluster was preempted by Google Cloud.

B.The Dataproc job failed due to an error in the code.

C.The Cloud Composer environment ran out of disk space.

D.The Airflow task timed out due to the default execution timeout.

AnswerD

SIGTERM indicates the task was killed, possibly due to timeout.

Why this answer

The default Airflow task execution timeout is 28 days in Cloud Composer, but individual tasks can have a shorter `execution_timeout` set in the DAG definition. When a long-running Dataproc job exceeds this timeout, Airflow sends a SIGTERM signal to the task to kill it, resulting in the observed error. This is the most likely cause because the error message directly indicates a forced termination by the Airflow scheduler, not an infrastructure or code failure.

Exam trap

The trap here is that candidates often attribute SIGTERM errors to infrastructure issues like cluster preemption or disk space, when in fact the error is a direct result of Airflow's task timeout mechanism, which is a common misconfiguration in long-running pipeline tasks.

How to eliminate wrong answers

Option A is wrong because Dataproc cluster preemption would cause a different error (e.g., 'Cluster not found' or 'Job failed due to node loss'), not a SIGTERM signal from Airflow. Option B is wrong because a code error in the Dataproc job would produce a job failure status and a different error message (e.g., 'Job failed with exit code 1'), not a SIGTERM from the orchestrator. Option C is wrong because running out of disk space in the Cloud Composer environment would cause worker crashes or DAG parsing errors, not a targeted SIGTERM to a specific task.

Full explanation →

578

MCQhard

Your Dataflow pipeline reads from Pub/Sub, performs transformations, and writes to BigQuery. You notice that the pipeline's autoscaling is not keeping up with sudden spikes in traffic, causing increased lag. The pipeline uses Classic Templates. Which change would most effectively improve autoscaling responsiveness?

A.Enable Dataflow Streaming Engine on the pipeline.

B.Switch to Dataflow Prime with Vertical Autoscaling enabled.

C.Increase the initial number of workers to handle the spike.

D.Use Flex Templates instead of Classic Templates.

AnswerA

Streaming Engine improves autoscaling by decoupling compute from state, allowing workers to scale more quickly.

Why this answer

Enabling Dataflow Streaming Engine reduces the overhead of checkpointing and state management by offloading them to the service side, which allows the pipeline to scale more quickly in response to sudden traffic spikes. This directly addresses the autoscaling lag because Streaming Engine decouples compute from state, enabling faster worker adjustments without the bottleneck of persistent disk-based shuffle.

Exam trap

A common misconception in Google Professional Data Engineer exams is that Flex Templates improve runtime performance or autoscaling, when in fact they only affect deployment flexibility, not the underlying execution engine's scaling behavior.

How to eliminate wrong answers

Option B is wrong because Dataflow Prime with Vertical Autoscaling adjusts the CPU/memory of existing workers, not the number of workers, so it does not improve horizontal autoscaling responsiveness to sudden traffic spikes. Option C is wrong because increasing the initial number of workers only sets a starting point; it does not improve the pipeline's ability to scale up dynamically during a spike, and it may waste resources during low traffic. Option D is wrong because Flex Templates only affect how the pipeline is deployed and parameterized, not the runtime autoscaling behavior; Classic Templates and Flex Templates share the same autoscaling mechanisms.

Full explanation →

579

MCQhard

You are designing a disaster recovery strategy for a critical streaming data processing pipeline. The pipeline reads from Cloud Pub/Sub, processes with Dataflow streaming, and writes to BigQuery. The required RPO is less than 1 minute, and RTO is less than 5 minutes. Which architecture should you implement?

A.Use cross-region replication with two separate Dataflow pipelines reading from a Pub/Sub cross-region subscription and writing to a BigQuery cross-region dataset

B.Run the pipeline using Dataflow batch mode with a 1-minute trigger and store intermediate results in Cloud Storage

C.Deploy resources in a single region with regular backups to Cloud Storage

D.Use a single Dataflow pipeline with a standby cluster in another region, but failover is manual

AnswerA

Cross-region replication ensures data is available in another region with minimal latency, meeting RPO and RTO.

Why this answer

Cross-region replication for Pub/Sub ensures messages are available in a secondary region with sub-second latency, and a separate Dataflow pipeline reading from a cross-region subscription provides active-active processing. BigQuery cross-region dataset replication (using the 'cross-region' dataset location, e.g., EU or US multi-region, or a specific dual-region configuration) ensures data durability and availability within the RPO of <1 minute. This architecture meets both RPO and RTO by eliminating single points of failure and enabling automatic failover without manual intervention.

Exam trap

The trap here is that candidates often assume a single pipeline with a standby cluster is sufficient, but they overlook that manual failover cannot meet the strict RTO of <5 minutes, and that cross-region replication must be active-active (not active-passive) to achieve sub-minute RPO.

How to eliminate wrong answers

Option B is wrong because Dataflow batch mode with a 1-minute trigger cannot achieve sub-minute RPO; batch processing introduces inherent latency and does not provide continuous streaming, so the RPO of <1 minute is not guaranteed. Option C is wrong because deploying in a single region with regular backups to Cloud Storage fails to meet the RTO of <5 minutes; restoring from backups takes significantly longer than 5 minutes, and there is no active standby to fail over to. Option D is wrong because a manual failover process cannot achieve the RTO of <5 minutes; manual intervention introduces unpredictable delays, and a standby cluster without automatic failover violates the RTO requirement.

Full explanation →

580

MCQmedium

A company runs a critical batch pipeline using Cloud Dataflow. The pipeline processes financial transactions and runs every hour. Recently, some runs have failed due to transient errors (e.g., network timeouts). The engineer wants to automatically retry failed runs without manual intervention. The pipeline is launched from a Cloud Composer DAG using DataflowPythonOperator. What is the BEST way to handle retries?

A.Add a DataflowJobStatusSensor in the DAG that waits for job completion and retries if failed.

B.Set the 'retries' parameter in the DAG's default_args to a positive integer.

C.Configure the Dataflow pipeline to automatically retry on failure using the --numberOfWorkerHarnessThreads option.

D.Use a Cloud Function triggered by Cloud Scheduler to re-launch the pipeline if the Dataflow job fails.

AnswerB

This allows Airflow to retry the entire task (which launches the Dataflow job) if it fails due to transient errors.

Why this answer

Cloud Composer (Apache Airflow) natively supports task-level retries via the 'retries' parameter in default_args. When a DataflowPythonOperator fails due to a transient error, Airflow automatically re-executes the task up to the specified number of retries, without requiring custom sensors or external triggers. This is the simplest and most reliable mechanism for handling transient failures in a DAG-driven pipeline.

Exam trap

The trap here is that candidates confuse Dataflow-level retry options (like --maxRetryAttempts) with Airflow task-level retries, or assume that a sensor or external trigger is required to detect and retry failures, when in fact Airflow's native retry parameter is the simplest and most appropriate solution for transient errors in a DAG-managed pipeline.

How to eliminate wrong answers

Option A is wrong because a DataflowJobStatusSensor only monitors job status and does not automatically retry the pipeline; it would require additional branching logic to relaunch the job, adding unnecessary complexity. Option C is wrong because --numberOfWorkerHarnessThreads controls parallelism within the Dataflow worker, not retry behavior on pipeline failure; retries are configured via --maxRetryAttempts or similar Dataflow pipeline options, not this flag. Option D is wrong because using a Cloud Function and Cloud Scheduler introduces an external dependency and latency, whereas Airflow's built-in retry mechanism is more direct and integrated with the DAG's execution context.

Full explanation →

581

Multi-Selectmedium

Which TWO actions are recommended to improve the reliability of a Cloud Dataflow streaming pipeline that processes event data from Pub/Sub?

Select 2 answers

A.Use a pull subscription with a 10-second acknowledgment deadline.

B.Enable Dataflow Streaming Engine.

C.Enable exactly-once processing sinks (e.g., BigQuery with guaranteed row-level insertion).

D.Disable autoscaling to prevent worker churn.

E.Use micro-batch processing with a small batch size.

AnswersB, C

Streaming Engine offloads state management to the backend, improving reliability.

Why this answer

Enabling Dataflow Streaming Engine moves state and computation from worker VMs to the backend service, reducing the impact of worker scaling and preemption. This improves reliability by providing consistent performance and fault tolerance for streaming pipelines, especially those with high throughput or stateful processing.

Exam trap

The trap here is that candidates often confuse reliability with throughput or latency, and may incorrectly choose micro-batching or disabling autoscaling as reliability improvements, when in fact Dataflow's reliability comes from its managed backend services like Streaming Engine.

Full explanation →

582

MCQhard

You are designing a data pipeline that must process sensitive customer data with strict access controls. The data is ingested via Cloud Pub/Sub, processed by Cloud Dataflow, and stored in BigQuery. The security team requires that data is encrypted at rest and in transit, and that access is limited to specific service accounts. Which implementation strategy meets all requirements?

A.Use Cloud KMS for BigQuery only; leave Dataflow with default encryption

B.Use VPC Service Controls and Cloud Armor for network security

C.Use default Google-managed encryption keys and IAM roles only

D.Use CMEK for Pub/Sub, Dataflow, and BigQuery, and VPC-SC with per-service service accounts

AnswerD

CMEK ensures encryption control; VPC-SC and service accounts enforce access.

Why this answer

It combines Customer-Managed Encryption Keys (CMEK) for all three services (Pub/Sub, Dataflow, BigQuery) to ensure data is encrypted at rest with keys controlled by the customer, and uses VPC Service Controls (VPC-SC) with per-service service accounts to enforce network perimeter security and least-privilege access. This meets the requirements for encryption at rest and in transit (CMEK also covers in-transit encryption via TLS) and strict access controls via service accounts and VPC-SC.

Exam trap

Google Cloud often tests the misconception that network security tools like VPC Service Controls or Cloud Armor alone satisfy encryption requirements, or that default encryption is sufficient when customer-managed keys are explicitly required.

How to eliminate wrong answers

Option A is wrong because it only applies Cloud KMS to BigQuery, leaving Dataflow with default Google-managed encryption, which does not meet the requirement for customer-controlled encryption at rest across all services. Option B is wrong because VPC Service Controls and Cloud Armor provide network security and perimeter controls but do not address data encryption at rest or in transit, which is a separate requirement. Option C is wrong because default Google-managed encryption keys and IAM roles alone do not provide customer-controlled encryption keys (CMEK) or the granular access controls enforced by VPC-SC with per-service service accounts.

Full explanation →

583

MCQeasy

An organization uses BigQuery on-demand pricing. To control costs, they want to estimate the bytes processed by a query before running it. Which command or method should they use?

A.Use the bq query --dry_run command

B.Use bq ls to list table sizes

C.Use BigQuery reservations to get cost estimate

D.Use INFORMATION_SCHEMA.JOBS_BY_PROJECT to view past costs

AnswerA

Dry run provides byte estimate without running the query.

Why this answer

BigQuery dry run estimates bytes processed without executing the query. It can be done via CLI with --dry_run flag or in the console.

Full explanation →

584

MCQmedium

You need to create a BigQuery table that stores customer transaction data. The table will be queried frequently by a customer_id column to retrieve recent transactions (last 30 days). Which table design optimizes query performance and cost?

A.Partition by customer_id and cluster by transaction_date

B.Partition by ingestion_time and cluster by customer_id

C.Cluster by transaction_date and customer_id without partitioning

D.Partition by transaction_date and cluster by customer_id

AnswerD

This design minimizes scanned bytes by pruning partitions on date and cluster blocks on customer_id.

Why this answer

Partitioning by transaction_date allows queries to scan only relevant partitions. Clustering by customer_id sorts data within each partition by customer_id, further reducing the amount of data scanned for queries filtering on customer_id. This combination is best for time-range queries with frequent customer_id filters.

Full explanation →

585

MCQhard

A company uses Vertex AI Feature Store for serving features. They have a high-throughput online serving requirement. Which configuration should they use?

A.Cloud Storage with high-memory instances

B.Bigtable as serving source

C.Firestore

D.Vertex AI Feature Store with online serving enabled

AnswerD

Vertex AI Feature Store is purpose-built for high-throughput online feature serving.

Why this answer

Vertex AI Feature Store with online serving enabled is the correct choice because it is specifically designed for low-latency, high-throughput retrieval of feature values for online predictions. It uses a managed Bigtable backend optimized for real-time serving, ensuring consistent performance under high request loads without requiring manual infrastructure management.

Exam trap

Google Cloud often tests the misconception that any low-latency database (like Bigtable or Firestore) can directly replace Vertex AI Feature Store, ignoring the managed orchestration, feature registry, and point-in-time lookup capabilities that are essential for consistent online serving in ML workflows.

How to eliminate wrong answers

Option A is wrong because Cloud Storage is a blob storage service with high latency and no indexing for real-time feature lookups, making it unsuitable for high-throughput online serving. Option B is wrong because Bigtable is a NoSQL database that can serve features, but it requires manual configuration, scaling, and integration with Vertex AI Feature Store, whereas the Feature Store provides a managed, optimized serving layer with built-in consistency and monitoring. Option C is wrong because Firestore is a document database designed for mobile and web apps with moderate throughput, not for the sub-millisecond latency and high concurrency required by ML feature serving at scale.

Full explanation →

586

MCQmedium

You are designing a streaming pipeline that needs to handle sudden spikes in traffic without losing data. The pipeline uses Pub/Sub and Dataflow. Which configuration ensures data is not lost if Dataflow falls behind?

A.Use Pub/Sub with a pull subscription and set the message retention duration to 7 days

B.Use Cloud Pub/Sub Lite with a smaller retention period

C.Use Pub/Sub with a push subscription and increase the acknowledgment deadline

D.Use Pub/Sub with exactly-once delivery and Dataflow with at-least-once processing

AnswerA

Pull subscriptions allow Dataflow to control the pace. 7-day retention lets Dataflow catch up after spikes.

Why this answer

Pub/Sub stores messages for up to 7 days, allowing Dataflow to catch up. Dataflow uses checkpointing to track progress. This combination ensures no data loss.

Full explanation →

587

MCQeasy

You have a BigQuery table 'orders' with columns order_id, customer_id, order_amount, and order_date. You need to rank customers by total spend per month, assigning the rank 1 to the highest spender. Which SQL function should you use in a window clause?

A.NTILE()

B.DENSE_RANK()

C.ROW_NUMBER()

D.RANK()

AnswerD

RANK() assigns the same rank to ties and leaves gaps; appropriate for ranking top spenders.

Why this answer

RANK() assigns a rank with gaps for ties; for top-spender ranking, that is appropriate. DENSE_RANK() also works but without gaps; the stem does not specify. ROW_NUMBER() gives unique numbers even for ties.

However, typical ranking with ties uses RANK().

Full explanation →

588

MCQmedium

A company is migrating on-premises Apache Spark jobs to Google Cloud Dataproc. They want to reduce operational overhead and minimize costs. Which architecture is most appropriate?

A.Use Cloud Dataproc Serverless for all Spark jobs.

B.Migrate jobs to Cloud Dataflow.

C.Run Spark on Compute Engine instances with startup scripts.

D.Use Dataproc clusters with auto-scaling and preemptible VMs.

AnswerD

Reduces cost and operational overhead.

Why this answer

Dataproc clusters with auto-scaling and preemptible VMs directly address the need to reduce operational overhead and minimize costs for on-premises Spark migrations. Auto-scaling dynamically adjusts cluster size based on workload, while preemptible VMs (which cost 60-80% less than standard VMs) handle fault-tolerant tasks, making this the most cost-effective and operationally efficient architecture for Spark on Dataproc.

Exam trap

The trap here is that candidates often choose Cloud Dataproc Serverless (Option A) thinking it eliminates all operational overhead, but they overlook that it lacks the cost-saving benefits of preemptible VMs and may not support all Spark features, making auto-scaling clusters with preemptible VMs the more appropriate choice for minimizing costs in a migration scenario.

How to eliminate wrong answers

Option A is wrong because Cloud Dataproc Serverless is designed for batch Spark workloads without cluster management, but it lacks the flexibility and cost optimization of preemptible VMs for long-running or complex jobs, and may not support all Spark configurations or libraries used in on-premises environments. Option B is wrong because Cloud Dataflow is a different processing engine (Apache Beam) that requires rewriting Spark jobs into Beam pipelines, adding migration complexity and operational overhead, not reducing it. Option C is wrong because running Spark on Compute Engine instances with startup scripts requires manual cluster management, scaling, and fault tolerance, increasing operational overhead and negating the benefits of a managed service like Dataproc.

Full explanation →

589

MCQmedium

You have a BigQuery table that is used by multiple teams. To save costs, you want to provide a consistent view of the data as of a specific point in time without creating full copies. Which BigQuery feature should you use?

A.Authorized views

B.Materialized views

C.Table snapshot

D.Table clone

AnswerC

Table snapshots are read-only, point-in-time copies that cost only storage and are ideal for sharing consistent views.

Why this answer

BigQuery table snapshots provide a point-in-time copy of a table that incurs only storage costs for the snapshot (no additional slot usage). They are read-only and can be used to share data without duplicating the base table.

Full explanation →

590

MCQhard

What is the root cause of this error and the correct solution?

A.The BigQuery table requires authorized view access.

B.The user running the job needs the BigQuery Admin role.

C.The Dataflow service account needs the BigQuery User role.

D.The Dataflow worker service account needs the BigQuery Data Viewer role.

AnswerD

BigQuery Data Viewer includes the required getData permission.

Why this answer

Dataflow workers execute under a specific service account (compute engine default or custom), and that service account must have the BigQuery Data Viewer role to read data from BigQuery tables. Without this permission, the workers cannot access the source data, causing the job to fail with access errors. The BigQuery User role is insufficient for reading table data, and the BigQuery Admin role is overly permissive and not required for this task.

Exam trap

Google Cloud often tests the distinction between the Dataflow controller service account (which manages the job) and the Dataflow worker service account (which performs data operations), leading candidates to incorrectly assign permissions to the controller account instead of the worker account.

How to eliminate wrong answers

Option A is wrong because authorized view access is a mechanism to share query results without granting direct table access, but the error here is about the Dataflow service account lacking read permissions on the BigQuery table, not about view authorization. Option B is wrong because the BigQuery Admin role grants full control over BigQuery resources, which is excessive and not necessary; the user running the job does not need admin rights—only the worker service account needs read access. Option C is wrong because the BigQuery User role allows running queries and creating datasets but does not grant read access to table data; the Dataflow service account (which orchestrates the job) does not directly read data—the worker service account does.

Full explanation →

591

MCQhard

A company needs to continuously synchronize customer data changes from an on-premises Oracle database to BigQuery for near-real-time analytics. The Oracle database has Change Data Capture (CDC) enabled. Which Google Cloud service should be used to stream these changes with minimal latency and schema evolution support?

A.Deploy a Dataflow pipeline with a JDBC source and Pub/Sub

B.Use Cloud SQL with a read replica and enable binary logging

C.Use Transfer Appliance to copy Oracle data periodically

D.Use Datastream to stream CDC changes from Oracle to BigQuery

AnswerD

Datastream directly supports Oracle CDC and streams to BigQuery with schema evolution.

Why this answer

Datastream is designed to stream CDC from Oracle (and MySQL/PostgreSQL) directly to BigQuery or GCS, supporting schema evolution and low-latency replication.

Full explanation →

592

Multi-Selecteasy

You need to deploy a reusable Dataflow pipeline that can be executed with different parameters from Cloud Composer. Which TWO components should you use? (Choose 2)

Select 2 answers

A.Direct runner

B.Dataflow Flex Template

C.Cloud Composer with DataflowStartFlexTemplateOperator

D.Dataflow Classic Template

E.Cloud Scheduler

AnswersB, C

Flex Templates are reusable and parameterizable.

Why this answer

Dataflow Flex Templates allow you to create custom templates that can accept runtime parameters. Cloud Composer can trigger these templates using the DataflowStartFlexTemplateOperator. Direct runner options are not needed.

Full explanation →

593

MCQeasy

A data engineer needs to ingest on-premises Oracle CDC data into BigQuery in near real-time with minimal operational overhead. Which service should they use?

A.Pub/Sub + Dataflow

B.Storage Transfer Service

C.Transfer Appliance

D.Datastream

AnswerD

Datastream is purpose-built for serverless CDC from databases to Google Cloud destinations like BigQuery and GCS.

Why this answer

Datastream is purpose-built for streaming change data capture (CDC) from Oracle and other sources into BigQuery with near-real-time latency and minimal operational overhead. It handles schema propagation, checkpointing, and automatic retries, eliminating the need to manage custom ingestion pipelines.

Exam trap

Google often tests the distinction between batch migration tools (Storage Transfer Service, Transfer Appliance) and streaming CDC services (Datastream), leading candidates to choose a batch option when the question explicitly requires near-real-time ingestion.

How to eliminate wrong answers

Option A is wrong because Pub/Sub + Dataflow requires building and maintaining a custom pipeline to handle Oracle CDC, including log mining and transformation logic, which increases operational overhead compared to a managed service. Option B is wrong because Storage Transfer Service is designed for bulk batch transfers of files from cloud or on-premises storage to Google Cloud, not for streaming CDC from a live database. Option C is wrong because Transfer Appliance is a physical device for offline, high-volume data migration, which cannot provide near-real-time streaming and introduces significant latency.

Full explanation →

594

MCQmedium

A financial analytics team uses Looker to explore BigQuery data. They need to allow business users to filter by a custom date range that is not tied to an existing dimension. The date range must be user-input at query time. What is the best approach in Looker?

A.Create an explore with a custom filter field in the Looker UI

B.Use a filter parameter directly on the date dimension

C.Add a dimension with a yesno filter that toggles the date range

D.Create a parameter in LookML using Liquid templating

AnswerD

Parameters allow user input at query time, rendered as filter controls, and can be used in conditions.

Why this answer

Looker uses Liquid templating in LookML to create parameters that render as filter controls at runtime. Users can input values that are then injected into the SQL. Creating a dimension with a yesno filter requires predefined values.

The filter parameter on a dimension only allows selecting from existing values, not arbitrary input.

Full explanation →

595

MCQmedium

An organization needs to trigger a Cloud Run service whenever a new file is uploaded to a specific Cloud Storage bucket. Which service should they use to set up this event-driven architecture?

A.Eventarc with a trigger for Cloud Storage events

B.Pub/Sub notifications on the bucket with a push subscription to Cloud Run

C.Cloud Scheduler calling Cloud Run on a schedule

D.Cloud Functions with a GCS trigger

AnswerA

Why this answer

Eventarc can capture Cloud Storage events (e.g., OBJECT_FINALIZE) and route them to Cloud Run, Cloud Functions, or Workflows. It supports CloudEvents standard.

Full explanation →

596

Multi-Selectmedium

A company is building a real-time anomaly detection pipeline using Dataflow. Events are ingested from Pub/Sub, and the pipeline must compute a sliding window average every minute over a 1-hour window. Which TWO configurations are required for this pipeline? (Choose 2)

Select 2 answers

A.Set the pipeline to use event time for watermarking.

B.Use a Sliding window of 1 hour with a 1-minute slide.

C.Use a Fixed window of 1 minute.

D.Use stateful processing with a custom timer.

E.Set the pipeline to use processing time for watermarking.

AnswersA, B

Event time ensures windows based on actual event occurrence time, necessary for correct sliding window semantics.

Why this answer

A sliding window of 1-hour length with a 1-minute slide period fits the requirement (every minute, compute over last hour). Fixed window of 1 minute would compute only per-minute, not sliding. Using stateful processing with timers is an alternative but not standard for sliding windows.

Dataflow's default watermark is based on event time; processing time would cause incorrect results. The window type and period are the key.

Full explanation →

597

Multi-Selectmedium

A data engineer needs to build a Dataflow pipeline that reads JSON messages from Pub/Sub, transforms them (including filtering, grouping, and enrichment), and writes the results to BigQuery. The pipeline must handle schema evolution in the input messages and minimize data loss. Which THREE settings or features should the engineer use? (Choose THREE.)

Select 3 answers

A.Use side inputs to enrich the data with reference data from BigQuery

B.Set the `withAllowedLateness` to 0 for windowing to minimize latency

C.Set up a dead letter queue (DLQ) for messages that fail to parse or validate

D.Enable autoscaling to handle spikes in message volume

E.Enable Streaming Engine to reduce checkpoint size

AnswersA, C, D

Correct. Side inputs enable enrichment of streaming data with reference data from BigQuery, supporting schema evolution by allowing dynamic lookup.

Why this answer

Side inputs allow the pipeline to enrich streaming data with reference data from BigQuery, which is a common requirement for handling schema evolution and enrichment. Option C is correct: a dead letter queue captures messages that fail to parse or validate, preventing data loss and enabling reprocessing of failed messages. Option D is correct: autoscaling adjusts the number of workers dynamically to handle spikes in message volume, ensuring no data is lost due to backpressure.

Option B is incorrect: setting `withAllowedLateness` to 0 does not help with schema evolution or minimize data loss; it simply drops late data, which could cause data loss. Option E is incorrect: Streaming Engine improves checkpoint performance but is not directly related to schema evolution or data loss minimization.

Full explanation →

598

MCQhard

Refer to the exhibit. A BigQuery dataset is shared with the group 'analysts@example.com' using the IAM policy shown. A user who is a member of this group reports that they cannot run queries on the dataset, though they can see the tables. What is the most likely reason?

A.The group needs the 'roles/bigquery.jobUser' role at the project level.

B.The user is using an incorrect client library version.

C.The user's account is not activated in the group membership.

D.The dataset has an organization policy that denies query access.

AnswerA

DataViewer provides read access but not job submission; jobUser must be granted at the project level to run queries.

Why this answer

The IAM policy grants the 'roles/bigquery.dataViewer' role at the dataset level, which allows the user to see tables but not run queries. To run queries, the user also needs the 'roles/bigquery.jobUser' role at the project level, because BigQuery query jobs are project-scoped resources. Without this role, the user lacks permission to create query jobs, even though they can view dataset metadata.

Exam trap

This question tests the distinction between dataset-level and project-level roles in BigQuery. Candidates often incorrectly assume that dataset-level view permissions are sufficient to run queries, but query jobs require the 'roles/bigquery.jobUser' role at the project level.

How to eliminate wrong answers

Option B is wrong because client library version does not affect IAM permissions; authentication and authorization are handled by Google Cloud IAM, not by the library version. Option C is wrong because if the user's account were not activated in the group membership, they would not be able to see the tables at all, as the dataset-level view permission would not apply. Option D is wrong because an organization policy that denies query access would typically block all query operations for all users, not just this user, and the user can see tables, which contradicts a blanket deny on queries.

Full explanation →

599

Multi-Selecteasy

A data engineering team is operationalizing a machine learning model for real-time fraud detection. The model must process transactions with sub-100ms latency and be highly available. Which TWO strategies should the team implement?

Select 2 answers

A.Deploy the model to multiple Google Cloud regions for failover.

B.Deploy the model to a single zone to minimize cross-zone latency.

C.Use Cloud Batch for asynchronous prediction.

D.Optimize the model by pruning or quantizing to reduce size.

E.Store the model in Cloud Storage and load it on each request.

AnswersA, D

Why this answer

Deploying the model to multiple Google Cloud regions ensures high availability and failover capability. If one region becomes unavailable, traffic can be routed to another region, maintaining sub-100ms latency by using regional load balancing and Cloud DNS. This aligns with the requirement for a highly available, low-latency fraud detection system.

Exam trap

Google Cloud often tests the misconception that single-zone deployment minimizes latency, but the real trade-off is between availability and negligible intra-region latency, making multi-region deployment the correct choice for high availability.

Full explanation →

600

MCQmedium

A company uses Google Ads and wants to automatically load their advertising data into BigQuery daily. They also need to transform the data with SQL and schedule a recurring query. Which combination of services meets these requirements with minimal operational overhead?

A.Cloud Functions triggered by Cloud Scheduler to call Google Ads API and load into BigQuery

B.Cloud Composer to extract Google Ads API and Dataflow to transform

C.Storage Transfer Service to move CSV files to GCS, then load into BigQuery

D.BigQuery Data Transfer Service for Google Ads and scheduled queries

AnswerD

Direct integration with scheduled queries for transformation.

Why this answer

BigQuery Data Transfer Service can automatically load Google Ads data; scheduled queries handle transformation.

Full explanation →

Page 8 of 12

All pages

1 2 3 4 5 6 7 8 9 10 11 12

Practice PDE by domain

Target a specific domain to shore up weak areas.

Designing Data Processing Systems Ingesting and Processing the Data Storing the Data Preparing and Using Data for Analysis Maintaining and Automating Data Workloads

See all domains with question counts →