Google Professional Data Engineer PDE Questions 76–150 | Page 2/10

Multi-Selecteasy

A data engineer needs to load data from CSV files in Cloud Storage into BigQuery. The CSV files have a header row and some columns contain nested JSON strings. Which TWO methods can they use to load this data into BigQuery?

Select 2 answers

A.Use Datastream to load CSV files

B.Use the Storage Write API to write rows from a custom application

C.Create a federated query using an external table

D.Create a BigQuery load job with the CSV format

E.Use gsutil to copy files into BigQuery

AnswersB, D

The Storage Write API can be used to stream data from CSV after parsing.

Why this answer

Option D is correct because a BigQuery load job natively supports the CSV format, including a header row via the --skip_leading_rows parameter, and can ingest Cloud Storage files directly into a native BigQuery table. Option B is correct because the Storage Write API lets a custom application parse the CSV and nested JSON strings itself and then stream the resulting rows into BigQuery, giving full control over transformation during load. Option A is wrong because Datastream is a change data capture (CDC) and replication service for databases such as MySQL, PostgreSQL, Oracle, and SQL Server, not a CSV file loader.

Option C is wrong because an external table (federated query) only queries data in place in Cloud Storage; it does not load the data into BigQuery. Option E is wrong because gsutil only copies objects between Cloud Storage locations and cannot write data into BigQuery tables.

Exam trap

Google often tests the distinction between loading data into BigQuery (permanent storage) versus querying external data sources (federated queries), causing candidates to mistakenly choose Option C as a valid loading method.

Full explanation →

Multi-Selectmedium

You are designing a BigQuery data lake for a healthcare organization. The data includes patient records that must be access-controlled at the row level. Which TWO features should you use to meet this requirement?

Select 2 answers

A.Row-level security using row access policies

B.Authorised views with row filters

C.Dataset-level IAM roles

D.Clustering on patient_id

E.Materialised views

AnswersA, B

Row access policies enforce row-level filtering directly in BigQuery by attaching a filter expression to a table, so each user sees only permitted patient records. This satisfies the stem's row-level access-control constraint natively, without duplicating data or maintaining separate views per role.

Why this answer

Row-level security using row access policies (A) is correct because BigQuery row access policies let you define a filter predicate (e.g., based on a patient_id or facility column) that restricts which rows each user or group can see, enforcing row-level access control directly on the table. Authorised views with row filters (B) are also correct because you can create a view that applies a WHERE clause to expose only permitted rows, then authorise that view on the dataset so users query the view without direct table access, achieving row-level control. Dataset-level IAM roles (C) only grant access at the dataset, table, or view level, not per row, so they cannot satisfy row-level requirements.

Clustering on patient_id (D) is a physical storage optimization that improves query performance and reduces scanned bytes but provides no access control. Materialised views (E) cache precomputed query results for performance and do not enforce row-level security.

Exam trap

PDE often tests the distinction between row-level and dataset-level access controls, causing candidates to overlook authorised views as a valid row-level security method.

Full explanation →

Multi-Selecthard

A company runs a Dataflow pipeline that processes a high-volume data stream. They notice that the pipeline's worker CPU utilisation is near 100% and the system lag is increasing. Which three actions can improve performance? (Choose three.)

Select 3 answers

A.Increase the worker disk size.

B.Increase the number of workers.

C.Use batch processing instead of streaming.

D.Enable Dataflow Streaming Engine.

E.Use higher-CPU machine types (e.g., n2-highcpu).

AnswersB, D, E

More workers distribute the load and reduce CPU per worker.

Why this answer

Increasing the number of workers distributes the processing load across more parallel workers, reducing CPU utilization per worker and allowing the pipeline to keep up with the incoming data stream. This directly addresses both high CPU usage and increasing system lag by scaling out horizontally.

Exam trap

A common trap is assuming that increasing disk size (Option A) improves CPU-related performance issues. In Dataflow, increasing disk size only helps with storage bottlenecks (e.g., shuffle disk overflow), not with high CPU utilization or system lag.

Full explanation →

MCQmedium

Your team runs a Cloud Composer 2 environment to orchestrate BigQuery ELT jobs. A DAG that loads a critical fact table must only run after an upstream ingestion DAG completes, and you want Composer itself to trigger it automatically without any external scheduler. Which mechanism should you use?

A.A TimeDeltaSensor with a fixed delay that matches the upstream DAG's typical runtime.

B.A Pub/Sub push subscription that invokes a Cloud Function to call the downstream DAG's trigger endpoint.

C.An ExternalTaskSensor that watches the upstream DAG's task in the same Composer environment.

D.A Cloud Scheduler job that calls the Airflow REST API to trigger the downstream DAG on a cron schedule.

AnswerC

ExternalTaskSensor is designed precisely for cross-DAG dependencies within the same Airflow deployment. It polls the metadata database for the specified external DAG and task run in the matching logical date, then releases downstream tasks only when that upstream task reaches the expected state. This gives a real completion dependency instead of guessing at timing.

Why this answer

Airflow's ExternalTaskSensor exists specifically to express a dependency on a task in another DAG within the same Airflow deployment. It checks the external DAG run for the matching execution date and only allows downstream tasks to proceed after the upstream task succeeds. Time-based waits, external cron triggers, and message-based glue all fail to guarantee that the ingestion actually finished successfully before the load begins.

Exam trap

The trap here is assuming any scheduling mechanism that can trigger a DAG also enforces a true completion dependency, when only a sensor that inspects the upstream task state does.

Full explanation →

MCQmedium

A financial company processes transactions in real-time and requires exactly-once processing semantics. They also need to reprocess historical data for backtesting. Which Google Cloud service should they use?

A.Cloud Pub/Sub

B.Cloud Functions

C.Cloud Dataproc

D.Cloud Dataflow

AnswerD

Cloud Dataflow provides exactly-once processing through its streaming engine and supports replaying bounded historical datasets via the same pipeline code, so backtesting and real-time transaction processing share one model. Pub/Sub alone lacks the reprocessing semantics required here.

Why this answer

Cloud Dataflow (D) is correct because it provides exactly-once processing semantics via its distributed snapshot mechanism (based on the MillWheel paper) and supports both real-time streaming and batch processing for historical backtesting under a unified programming model. This allows the company to reprocess historical data using the same pipeline code, ensuring consistency across real-time and batch modes.

Exam trap

Google Cloud often tests the misconception that Cloud Pub/Sub (A) provides exactly-once delivery, but in reality it offers at-least-once delivery, and candidates overlook Dataflow's unified batch/streaming model for reprocessing historical data.

How to eliminate wrong answers

Option A is wrong because Cloud Pub/Sub is a messaging service that offers at-least-once delivery by default, not exactly-once processing, and it lacks built-in capabilities for reprocessing historical data in a unified batch/streaming manner. Option B is wrong because Cloud Functions is an event-driven serverless compute service that does not provide exactly-once processing guarantees or native support for reprocessing large historical datasets; it is designed for lightweight, stateless functions. Option C is wrong because Cloud Dataproc is a managed Hadoop/Spark service that does not natively guarantee exactly-once processing semantics and requires manual handling of state and reprocessing logic, unlike Dataflow's automatic checkpointing.

Full explanation →

MCQmedium

You are using Cloud Composer to orchestrate a data pipeline that runs a Dataproc job to process data, followed by a BigQuery load. You notice that the Dataproc job sometimes takes longer than expected, causing the BigQuery load to start before the Dataproc job finishes, resulting in incomplete data. Which Airflow feature should you use to ensure the BigQuery load only runs after the Dataproc job completes successfully?

A.Set a dependency between the Dataproc job task and the BigQuery load task using the >> operator.

B.Use a TimeSensor to wait for the Dataproc job to finish.

C.Use the trigger_rule parameter to set the BigQuery load task to 'all_done'.

D.Set the Dataproc job task's retries to a high number.

AnswerA

In Airflow, task dependencies are defined using the bitshift operators, such as >>, to specify that one task must complete successfully before another starts. By setting the Dataproc job task upstream of the BigQuery load task, you ensure that the load only runs after the Dataproc job finishes successfully. This is the fundamental way to control execution order in a DAG and directly addresses the issue of premature BigQuery loads due to timing.

Why this answer

The correct way to enforce that the BigQuery load runs only after the Dataproc job completes successfully is to define a task dependency using the >> operator. This creates a directed edge in the DAG, ensuring the downstream task waits for the upstream task to succeed. Other options either do not control ordering or do not enforce success.

Task dependencies are the core mechanism in Airflow for sequencing tasks in a data pipeline.

Exam trap

The trap here is confusing task dependencies with sensors or retries; only explicit dependencies guarantee that one task waits for another's successful completion.

Full explanation →

MCQmedium

Your organization runs a batch Dataflow pipeline that reads from BigQuery, transforms records, and writes Parquet files to Cloud Storage partitioned by event date. The pipeline currently writes all files into a single directory and downstream Hive-style queries scan the entire dataset. You need to restructure the output so that queries scan only the relevant date partitions, while keeping the pipeline idempotent on re-runs. What should you do?

A.Use AvroIO with a custom filename policy that includes the event date, and enable the pipeline's --diskSizeGb option to improve write throughput.

B.Use FileIO.writeDynamic with a destination function that maps each record to a path like gs://bucket/events/event_date=YYYY-MM-DD/, and write to a temporary location before atomically moving files into place.

C.Write all files to a single directory and create a BigQuery external table over the bucket, then rely on BigQuery's automatic partition pruning from the file names.

D.Add a GroupByKey transform keyed by event date before writing, then use TextIO to write one file per date into a flat directory with the date embedded in the file name.

AnswerB

writeDynamic lets each record choose its output directory based on the event date, producing Hive-style partition paths that downstream engines prune. Writing to a temporary location and then moving files into the final partition directory ensures that a failed re-run does not leave partial data visible to queries, which preserves idempotency. This combination directly satisfies both the partition-pruning and idempotency requirements.

Why this answer

Hive-style partition pruning depends on directory structure, not file naming. FileIO.writeDynamic with a destination function that emits paths containing event_date=YYYY-MM-DD creates the directories that query engines use to skip irrelevant data. Writing first to a temporary location and then moving completed files into the final partition directory prevents partially written partitions from being visible if the pipeline fails and is re-run, preserving idempotency.

Exam trap

The trap here is assuming that embedding the date in a file name or file extension is enough for partition pruning, when engines actually prune on directory paths such as event_date=YYYY-MM-DD.

Full explanation →

MCQmedium

Your organization deploys multiple versions of the same model to Vertex AI Endpoint for A/B testing. You have a production model (v1) serving 90% of traffic and a candidate model (v2) serving 10%. After one week, you observe that v2 has a slightly lower AUC but significantly higher business metrics like click-through rate. The product team wants to gradually increase v2's traffic. However, you need to ensure that the overall prediction latency remains under 200 ms. Currently, the endpoint has 10 replicas for v1 and 2 replicas for v2. What is the best approach to roll out v2 while maintaining latency SLO?

A.Merge v2's model into v1 by retraining v1 with v2's architecture and deploy as a single model.

B.Immediately set v2 to serve 100% traffic and monitor latency; if it exceeds 200 ms, roll back.

C.Increase v2's traffic split by 10% each day while also adding replicas for v2 based on CPU utilization.

D.Use a separate endpoint for v2 and route traffic at the load balancer level.

AnswerC

Raising the split incrementally limits risk, while autoscaling v2 replicas on CPU utilisation preserves the 200 ms latency SLO as v2's share grows. Static replica counts would let v2 traffic overwhelm its two replicas, breaching the latency constraint the stem imposes.

Why this answer

The best approach is to gradually increase v2's traffic while dynamically scaling replicas based on CPU utilization. This allows controlled rollout, monitoring of latency, and ensures sufficient resources to maintain the latency SLO. Adding replicas based on CPU utilization helps handle increased load as traffic to v2 grows, preventing latency degradation.

Exam trap

PDE often tests the trade-off between rapid rollout and maintaining SLOs; candidates may choose immediate 100% traffic or separate endpoints without considering autoscaling and gradual increase.

How to eliminate wrong answers

Option A is wrong because merging models is complex, may degrade performance, and does not allow separate monitoring of v2's business metrics. Option B is wrong because immediately shifting 100% traffic risks violating the latency SLO if v2 cannot handle the load, and rollback may be too late. Option D is wrong because using a separate endpoint and routing at the load balancer adds complexity, may not integrate with Vertex AI's traffic split features, and does not automatically scale replicas based on load.

Full explanation →

MCQeasy

A data engineer needs to ingest data from a Cloud Storage bucket into BigQuery. The data is in CSV format and is updated daily with new files. The engineer wants to minimize manual intervention and ensure that new files are automatically loaded into BigQuery. Which Google Cloud service should be used to orchestrate this?

A.Cloud Functions

B.Cloud Scheduler

C.Cloud Data Fusion

D.Cloud Composer

AnswerA

Cloud Functions can be triggered by object finalize events in Cloud Storage, allowing automatic execution of code when a new file is uploaded. This enables a serverless, event-driven approach to load new CSV files into BigQuery with minimal intervention. It is simple, cost-effective, and directly addresses the need for automatic ingestion upon file arrival.

Why this answer

Cloud Functions provides an event-driven, serverless way to respond to new files in Cloud Storage. By configuring a Cloud Function to trigger on object finalize events, the function can execute a BigQuery load job for each new CSV file. This approach requires no polling, minimal code, and automatically scales.

Other services like Cloud Scheduler or Cloud Composer could work but involve more overhead and are less direct for this specific requirement.

Exam trap

The trap here is overlooking event-driven options and defaulting to scheduled or orchestrated solutions, which add unnecessary complexity for simple file arrival triggers.

Full explanation →

MCQmedium

A data engineer needs to enforce that all datasets in a project expire after 90 days to reduce storage costs. They want to automate this without manual intervention. Which approach should they use?

A.Use Cloud Storage lifecycle rules to delete tables after 90 days

B.Create an IAM policy that revokes access after 90 days

C.Use BigQuery scheduled queries to delete tables older than 90 days

D.Set a default table expiration on each BigQuery dataset to 90 days

AnswerD

A default table expiration set on each dataset automatically deletes tables 90 days after creation, requiring no manual intervention. This satisfies the stem's automation constraint while enforcing the 90-day retention across all datasets in the project.

Why this answer

BigQuery datasets support a default table expiration setting that automatically deletes tables after a specified number of days. This enforces a 90-day lifecycle for all tables in the dataset without requiring manual intervention or external automation, directly addressing the cost reduction goal.

Exam trap

A common mistake is applying Cloud Storage lifecycle rules to BigQuery tables. BigQuery has its own table expiration settings at the dataset level, which are distinct from Cloud Storage object lifecycle management.

How to eliminate wrong answers

Option A is wrong because Cloud Storage lifecycle rules apply to objects in Cloud Storage buckets, not to BigQuery tables; BigQuery tables are stored separately and cannot be managed by Cloud Storage lifecycle policies. Option B is wrong because IAM policies control access permissions, not data lifecycle; revoking access does not delete tables or reduce storage costs. Option C is wrong because BigQuery scheduled queries can delete tables but require writing and maintaining custom SQL scripts and scheduling logic, introducing complexity and potential failure points, whereas a default table expiration is a declarative, server-managed setting that requires no ongoing maintenance.

Full explanation →

MCQmedium

A retail analytics team runs a Cloud Composer DAG that starts a Dataproc job, waits for it, and then runs a BigQuery load. The Dataproc job sometimes fails due to a transient YARN resource shortage, and the team wants the DAG to automatically retry the Dataproc submission a few times before alerting. Which Airflow configuration on the Dataproc task best meets this need?

A.Set the Dataproc operator's cluster_name parameter to a different cluster and rely on Dataproc to resubmit the job.

B.Add a downstream EmailOperator that notifies the team on failure and manually reruns the DAG.

C.Set retries to 3 and retry_delay to 5 minutes on the Dataproc job task.

D.Set max_active_runs to 3 on the DAG so multiple DAG runs can absorb the failure.

AnswerC

Task-level retries cause Airflow to re-run the same operator up to the specified count after a failure, with the given delay between attempts. Because the failure is transient, a small number of retries with a short delay gives the cluster time to free resources, and only after all attempts fail does the task enter a failed state that triggers alerting.

Why this answer

Transient failures are best handled with operator-level retries, which re-execute the same task after a delay and only surface an alert once the configured attempts are exhausted. Setting retries to 3 with a short retry_delay on the Dataproc task gives the cluster room to recover from resource pressure while keeping the DAG's dependency chain and alerting behavior unchanged.

Exam trap

The trap here is reaching for DAG-level concurrency settings to solve a single-task reliability problem, when retries belong on the failing task itself.

Full explanation →

MCQhard

A logistics company writes shipment telemetry to a Cloud Bigtable table using a row key of shipment_id, where shipment_id values are monotonically increasing and roughly sequential over time. Write throughput has become uneven, with a small number of tablet servers overloaded while others sit idle. The engineer must improve write distribution without changing the schema of the column families. What should the engineer do?

A.Create a Cloud Bigtable replication cluster in a second zone and write half the traffic to each cluster.

B.Increase the number of column families in the table to spread the write load across more storage structures.

C.Enable automatic region rebalancing so Bigtable redistributes the hot rows across the cluster.

D.Prepend a hashed or reversed component to the row key so writes spread across the key space instead of concentrating at the high end.

AnswerD

Sequential, monotonically increasing keys concentrate writes on the tablet holding the highest key range, creating a hotspot. Prepending a hash bucket or reversing the key distributes incoming writes across many tablets, letting all tablet servers share load. This preserves the column family schema and keeps keys unique, which is the standard fix for write hotspots caused by sequential row keys.

Why this answer

Bigtable distributes rows by row key ranges, so a monotonically increasing key sends every new write to the tablet owning the highest range, overloading one node while others idle. Adding a hash bucket prefix or reversing the key scatters writes across many ranges and tablets, restoring even load without altering column families. Automatic rebalancing, extra column families, and replication clusters all leave the underlying key concentration in place, so the hotspot persists.

Exam trap

The trap here is assuming Bigtable's automatic tablet rebalancing will fix a hotspot, when rebalancing cannot compensate for a row key that always targets the same key range.

Full explanation →

MCQmedium

Your team has implemented a CI/CD pipeline using Cloud Composer (Apache Airflow) to retrain a model every day. The pipeline reads new data from BigQuery, trains a model using Vertex AI Training, evaluates it, and if the accuracy improves, deploys it to a Vertex AI Endpoint. For the past week, the pipeline has been running successfully but no new model has been deployed because the evaluation accuracy never exceeds the previous model's accuracy. The training data volume has been consistent. You suspect that the model is not learning from the new data. What should you do?

A.Deploy the new model anyway and run an A/B test in production to see if it performs better online.

B.Examine the training data for any data quality issues such as missing values or label leakage.

C.Increase the training budget or number of training steps to allow the model to converge better.

D.Change the evaluation metric to a different one that may show improvement, such as F1 score instead of accuracy.

AnswerB

Consistent data volume with no accuracy improvement points to data quality problems such as missing values or label leakage, which prevent the model learning new patterns. Inspecting the training data directly identifies whether inputs are corrupted before tuning hyperparameters.

Why this answer

When a model trained on consistent data volume fails to improve accuracy over a week, the most likely root cause is a data quality problem rather than a training capacity problem. Missing values, label leakage, or corrupted features can prevent the model from learning meaningful patterns, causing accuracy to plateau or degrade. Examining the training data for these issues is the correct diagnostic step before changing hyperparameters or deployment strategy.

Exam trap

PDE often tests the misconception that more training or a different metric will fix a model that is not learning, when the real issue is usually data quality or feature engineering.

How to eliminate wrong answers

Option A is wrong because deploying a model that has not demonstrated improvement in offline evaluation introduces risk without evidence of benefit, and A/B testing does not address the underlying learning failure. Option C is wrong because increasing training budget or steps will not help if the data itself is flawed; more compute on bad data yields the same or worse results. Option D is wrong because changing the evaluation metric to one that shows improvement is a form of metric shopping that masks the real problem and does not fix the model's inability to learn from new data.

Full explanation →

MCQmedium

An organization needs to continuously replicate change data from a MySQL database to BigQuery with sub-minute latency. The database is running on-premises. Which Google Cloud service should they use?

A.Cloud Pub/Sub with a custom connector to MySQL

B.BigQuery Data Transfer Service for MySQL

C.Cloud Dataflow with a JDBC source

D.Cloud Datastream

AnswerD

Datastream provides serverless change data capture, reading MySQL binlogs and streaming inserts, updates and deletes into BigQuery with sub-minute latency. It supports on-premises sources through private connectivity, satisfying the continuous replication and low-latency constraints without managing replication infrastructure.

Why this answer

Cloud Datastream is a serverless change data capture (CDC) and replication service that supports continuous replication from MySQL (including on-premises) to BigQuery with sub-second latency. It handles schema conversion, and integrates with BigQuery for real-time analytics. This is the purpose-built service for the requirement.

Exam trap

PDE often tests the misconception that Dataflow or Pub/Sub are the default for CDC, when Datastream is the managed service specifically designed for low-latency database replication to BigQuery.

How to eliminate wrong answers

Option A is wrong because Cloud Pub/Sub with a custom connector requires building and managing a custom CDC solution, which is complex and may not achieve sub-minute latency reliably. Option B is wrong because BigQuery Data Transfer Service for MySQL is for batch transfers, not continuous CDC with sub-minute latency. Option C is wrong because Cloud Dataflow with a JDBC source can be used for CDC but requires custom development and is not as streamlined as Datastream; it may also not guarantee sub-minute latency without significant tuning.

Full explanation →

MCQmedium

You have a BigQuery table that is partitioned by ingestion time and clustered on user_id. The table stores event logs and is queried frequently by user_id to analyze user behavior over the last 30 days. Queries are still scanning too many partitions. Which optimization should you apply first?

A.Create a materialized view that pre-aggregates data by user_id and date

B.Remove partitioning and rely solely on clustering

C.Change the partition column to a DATE column based on event_timestamp and keep clustering on user_id

D.Add clustering on a second column like event_type

AnswerC

Ingestion-time partitioning groups rows by load time, so a 30-day event query scans every partition loaded in that period regardless of event dates. Partitioning on a DATE derived from event_timestamp enables partition pruning to only the relevant event dates, while clustering on user_id still accelerates per-user filtering.

Why this answer

Ingestion-time partitioning creates partitions based on when data arrives, not the event timestamp, so a query filtering on the last 30 days of events may still scan partitions that contain older events ingested recently. Changing the partition column to a DATE derived from event_timestamp aligns partition pruning with the query filter, and keeping clustering on user_id preserves the benefit for user-based filtering. This is the highest-impact first optimization.

Exam trap

PDE often tests whether candidates recognize that ingestion-time partitioning does not align with event-time filters — many assume any partitioning enables pruning, missing that the partition column must match the query predicate.

How to eliminate wrong answers

Option A is wrong because a materialized view adds cost and complexity and does not fix the root cause — the partition scheme still mismatches the query predicate, so the view may still scan too much. Option B is wrong because removing partitioning eliminates partition pruning entirely, which would make scans worse, not better. Option D is wrong because adding a second clustering column can help some queries but does not address the fundamental mismatch between ingestion-time partitions and event-time filters, so it is not the first fix.

Full explanation →

Multi-Selecthard

Which TWO statements about designing a data processing pipeline on Google Cloud are correct? (Choose 2.)

Select 2 answers

A.Pub/Sub guarantees message ordering across all subscribers globally.

B.Cloud Bigtable is ideal for data warehousing and SQL analytics.

C.Dataproc is the best choice for fully managed data warehousing and analytics.

D.Cloud Data Fusion allows you to build and manage data pipelines visually without writing code.

E.Dataflow supports both batch and streaming modes in a single pipeline model.

AnswersD, E

Cloud Data Fusion provides a graphical, code-free interface for building and managing pipelines, satisfying the stem's requirement for visual design without coding. Its underlying CDAP engine handles orchestration and execution, so teams can create repeatable data integration workflows through drag-and-drop rather than hand-written code.

Why this answer

Option D is correct because Cloud Data Fusion is a fully managed, code-free ETL/ELT service built on the open-source CDAP framework, providing a visual drag-and-drop pipeline designer and a broad library of connectors and transformations, so pipelines can be built and managed without writing code. Option E is correct because Dataflow, based on Apache Beam, uses a unified programming model in which the same pipeline code can run in batch or streaming mode, with the runner handling windowing, triggers, and watermarks for both. Option A is wrong because Pub/Sub does not guarantee global ordering across all subscribers; ordering is only provided per ordering key within a region when message ordering is explicitly enabled.

Option B is wrong because Cloud Bigtable is a wide-column NoSQL database optimized for high-throughput, low-latency reads and writes at scale, not for data warehousing or SQL analytics. Option C is wrong because Dataproc is a managed Apache Hadoop/Spark service for running clusters and jobs, whereas BigQuery is the fully managed data warehousing and analytics service.

Exam trap

Google Cloud often tests the distinction between fully managed services (like BigQuery for warehousing) and managed cluster services (like Dataproc), as well as the limitations of Pub/Sub ordering guarantees, to see if candidates confuse operational databases with analytical systems.

Full explanation →

Multi-Selectmedium

A data engineer needs to restrict access to BigQuery datasets such that only data from approved VPC networks can query them. They also need to audit data access. Which two security controls should they implement? (Choose two.)

Select 2 answers

A.Cloud Audit Logs

B.Customer-managed encryption keys (CMEK)

C.Data Loss Prevention (DLP)

D.IAM roles

E.VPC Service Controls

AnswersA, E

Cloud Audit Logs record BigQuery API calls and data access events, capturing who queried which datasets and when. This satisfies the auditing half of the requirement, providing the access visibility needed alongside network-based restrictions to demonstrate compliance and investigate suspicious queries.

Why this answer

VPC Service Controls (E) is correct because it creates a service perimeter around BigQuery that restricts access to only approved VPC networks, blocking requests from outside the perimeter and mitigating data exfiltration. Cloud Audit Logs (A) is correct because it records BigQuery data access and administrative activity (Data Access audit logs must be explicitly enabled), providing the required audit trail. CMEK (B) only controls encryption key ownership, not network-based access or auditing.

DLP (C) discovers and classifies sensitive data but does not restrict network access or provide access auditing. IAM roles (D) grant permissions to identities but cannot enforce VPC network origin restrictions on BigQuery.

Exam trap

Google Cloud often tests the distinction between identity-based controls (IAM) and network-based controls (VPC Service Controls), leading candidates to mistakenly choose IAM roles when the requirement explicitly specifies restricting access by VPC network rather than by user identity.

Full explanation →

MCQeasy

A data engineer needs to monitor model performance over time for drift detection. What tool is specifically designed for this?

A.Vertex AI Model Monitoring

B.Cloud Monitoring

C.Cloud Logging

D.BigQuery ML

AnswerA

Vertex AI Model Monitoring is purpose-built to track deployed model performance against a training baseline, computing drift and skew metrics on a schedule. It satisfies the drift-detection constraint directly, unlike general logging or dashboards that surface raw metrics without distributional comparison.

Why this answer

Vertex AI Model Monitoring is specifically designed to detect prediction drift and feature skew in deployed machine learning models. It continuously analyzes serving data against training data distributions and alerts when statistical metrics (e.g., Jensen-Shannon divergence, L-infinity distance) exceed configured thresholds, making it the correct tool for drift detection in the context of operationalizing ML models.

Exam trap

Google Cloud often tests the distinction between general-purpose monitoring tools (Cloud Monitoring, Cloud Logging) and ML-specific monitoring services (Vertex AI Model Monitoring), trapping candidates who assume any monitoring tool can handle drift detection.

How to eliminate wrong answers

Option B (Cloud Monitoring) is wrong because it is a general-purpose infrastructure and application monitoring service for metrics, uptime, and alerting, not specialized for ML model drift detection. Option C (Cloud Logging) is wrong because it is a centralized log management and analysis service for storing and querying log data, not designed to compute statistical drift between training and serving distributions. Option D (BigQuery ML) is wrong because it is a service for creating and executing machine learning models using SQL queries in BigQuery, not a monitoring tool for detecting drift in already-deployed models.

Full explanation →

MCQhard

A financial services company uses Cloud Bigtable to store trade data. They are experiencing hot-spotting on a single node, causing high latency. The row key format is [trade_id]#[timestamp]. Which row key design change would BEST distribute writes across tablets?

A.Use a hashed prefix of the trade_id, e.g., [hash(trade_id)]#[trade_id]#[timestamp]

B.Use a single row key of [timestamp]

C.Increase the number of Bigtable nodes to 20

D.Change row key to [timestamp]#[trade_id]

AnswerA

Hashing the trade_id prefix scatters sequential writes across the entire key space, so consecutive trades land on different tablets rather than concentrating on one node. This directly resolves the hot-spotting constraint, since Cloud Bigtable distributes rows lexicographically by row key and a uniform hash prefix balances write load.

Why this answer

Adding a hashed prefix of the trade_id ensures that writes are evenly distributed across all Bigtable tablets. Bigtable partitions data by row key lexicographic order; without a hash, sequential trade IDs or timestamps cause all recent writes to land on a single tablet, creating a hotspot. The hash spreads the write load uniformly, regardless of the underlying key pattern.

Exam trap

A common trap in the Google Professional Data Engineer exam is confusing the order of key components. Simply reversing the order (e.g., putting the timestamp first) is often incorrectly thought to solve hot-spotting, but any monotonically increasing value at the start of the key will still cause a hotspot because Bigtable stores rows in lexicographic order.

How to eliminate wrong answers

Option B is wrong because using a single row key of [timestamp] would cause all writes with the same timestamp to collide on one row, creating an extreme hotspot and violating Bigtable's requirement for unique, distributed row keys. Option C is wrong because increasing the number of nodes does not fix a row key design flaw; Bigtable cannot rebalance writes if the row key pattern forces all traffic to a single tablet, and adding nodes only helps if the load is already distributed. Option D is wrong because reversing the order to [timestamp]#[trade_id] still places all writes with the same timestamp adjacent in lexicographic order, so recent timestamps will still hotspot on a single tablet; it does not introduce the randomness needed for distribution.

Full explanation →

MCQmedium

A data engineer is building a Dataflow pipeline in Python that reads from BigQuery, transforms data, and writes to Cloud Storage. The pipeline will be deployed in production. Which approach should they use to ensure the pipeline is reusable across environments with different configuration parameters?

A.Create a separate pipeline for each environment with hardcoded values

B.Use a Dataflow Classic Template

C.Use a Dataflow Flex Template

D.Run the pipeline using the DirectRunner for each environment

AnswerC

Flex Templates package the pipeline as a Docker image with a metadata file, so runtime parameters such as input, output and project values are supplied at job launch. This satisfies the reusability requirement across environments without editing code, unlike classic templates with fixed dependencies.

Why this answer

Dataflow Flex Templates allow you to package a Docker container with your pipeline code and dependencies, enabling parameterization at runtime via the Dataflow UI, CLI, or API. This makes the pipeline reusable across environments (e.g., dev, staging, prod) by passing different configuration parameters (like project IDs, table names, or output paths) without modifying the code. Flex Templates support custom container images and are the recommended approach for production pipelines that need environment-agnostic deployment.

Exam trap

Google often tests the distinction between Classic Templates and Flex Templates, where candidates mistakenly choose Classic Templates because they are simpler, but Flex Templates are required for custom environments and parameterized production reuse.

How to eliminate wrong answers

Option A is wrong because creating separate pipelines with hardcoded values violates the principle of reusability and introduces maintenance overhead; any change requires updating multiple pipeline copies, increasing the risk of configuration drift. Option B is wrong because Classic Templates are limited to the Apache Beam SDK's built-in I/O transforms and do not support custom container images or complex dependencies, making them less flexible for production pipelines that may require custom code or third-party libraries. Option D is wrong because the DirectRunner is intended for local testing and development only; it runs the pipeline in a single JVM process and cannot handle the scalability, distributed execution, or environment-specific configuration needed for production deployment.

Full explanation →

MCQmedium

A company has a Cloud SQL for PostgreSQL instance and wants to create a read replica to offload read traffic from the primary. They also need to ensure the replica is in a different region for disaster recovery. Which Cloud SQL feature should they use?

A.External replica

B.Automatic backups with PITR

C.Cross-region read replica

D.High availability (HA) configuration

AnswerC

A cross-region read replica is created in a different region from the primary, replicating asynchronously over Google's network. This offloads read traffic while providing regional isolation for disaster recovery, which a same-region replica or standard read replica cannot deliver.

Why this answer

Cross-region read replicas in Cloud SQL for PostgreSQL allow you to create a replica in a different region from the primary instance. This offloads read traffic from the primary while also providing disaster recovery capabilities by maintaining a standby copy in a geographically separate location. The replica uses asynchronous replication to stay up-to-date with the primary.

Exam trap

A common mistake is to confuse high availability (HA) configuration, which only protects against zone-level failures, with cross-region read replicas that provide regional disaster recovery. HA does not replicate data to another region.

How to eliminate wrong answers

Option A is wrong because an external replica refers to a replica running outside of Cloud SQL, such as on a self-managed instance or on-premises, which does not meet the requirement for a managed Cloud SQL replica in a different region. Option B is wrong because automatic backups with point-in-time recovery (PITR) are used for data recovery from failures or corruption, not for offloading read traffic or providing a separate regional replica for disaster recovery. Option D is wrong because a high availability (HA) configuration uses synchronous replication within the same region (typically across zones) to provide failover, not a cross-region replica for disaster recovery or read offloading.

Full explanation →

MCQeasy

A data engineering team needs to run SQL analytics on a large BigQuery dataset from a Looker Studio dashboard. The dashboard must return results quickly, and the underlying data changes only once per day through a batch load. The team wants to minimize query cost and latency for repeated dashboard queries. What should they do?

A.Enable the BigQuery BI Engine reservation and rely on it to cache all dashboard queries automatically.

B.Export the dataset to Cloud Storage as CSV and have Looker Studio query the files directly.

C.Partition the table by ingestion date and require all dashboard queries to filter on that column.

D.Create a materialized view that aggregates the dataset and configure it to refresh on a daily schedule.

AnswerD

A materialized view stores precomputed results and can be refreshed automatically or on a schedule. Because the dashboard queries repeat and the source data changes daily, the view serves cached results, reducing bytes scanned and latency while lowering cost. BigQuery can also use the materialized view for compatible queries, making it well suited to this batch-refresh analytics pattern.

Why this answer

Materialized views precompute and store query results, and with a daily refresh schedule they match the batch update cadence. Repeated dashboard queries hit the stored results, cutting bytes scanned and latency. This directly targets the cost and performance goals, unlike caching layers or partitioning that do not precompute aggregations.

Exam trap

The trap here is assuming an in-memory acceleration layer replaces precomputation, when a materialized view is what actually reduces scanned bytes for repeated aggregate queries.

Full explanation →

MCQmedium

An application requires a globally distributed, strongly consistent database with 99.999% availability SLA. The workload is OLTP with high throughput across continents. Which service fits best?

A.Cloud SQL with cross-region replicas

B.Firestore

C.Cloud Bigtable

D.Cloud Spanner

AnswerD

Cloud Spanner provides external consistency globally through TrueTime, and its multi-region configurations carry a 99.999% availability SLA. That combination of strong consistency, cross-continent distribution and five-nines uptime matches the OLTP high-throughput requirement, unlike eventually consistent alternatives.

Why this answer

Cloud Spanner is the only service that provides globally distributed, strongly consistent (external consistency via TrueTime) OLTP with 99.999% availability SLA. It supports high-throughput ACID transactions across continents using synchronous replication and atomic clocks, meeting all stated requirements.

Exam trap

Candidates often confuse Cloud SQL cross-region replicas as providing strong consistency, but those replicas are eventually consistent due to asynchronous replication, and they do not meet the 99.999% SLA.

How to eliminate wrong answers

Option A is wrong because Cloud SQL with cross-region replicas uses asynchronous replication, which cannot guarantee strong consistency across regions and offers only a 99.95% SLA for regional instances, not 99.999%. Option B is wrong because Firestore is a NoSQL document database that provides strong consistency only within a single region; its multi-region mode uses eventual consistency for global reads, and it lacks the ACID transaction support needed for high-throughput OLTP across continents. Option C is wrong because Cloud Bigtable is a wide-column NoSQL database designed for analytical workloads with high throughput, but it does not support SQL queries, ACID transactions, or strong consistency across regions; it offers only single-row transactions and eventual consistency for multi-cluster replication.

Full explanation →

MCQmedium

A company wants to run hybrid transactional and analytical workloads on a PostgreSQL-compatible database with high performance. Which service should they choose?

A.Cloud Spanner

B.Cloud SQL for PostgreSQL

C.BigQuery

D.AlloyDB

AnswerD

AlloyDB is a PostgreSQL-compatible, Google-managed database engine with a columnar acceleration engine that runs hybrid transactional and analytical workloads at high performance, satisfying both the PostgreSQL compatibility and HTAP constraints. Cloud SQL and standard PostgreSQL lack this integrated analytical acceleration.

Why this answer

AlloyDB is the correct choice because it is a fully managed PostgreSQL-compatible database service specifically designed for high-performance hybrid transactional and analytical workloads. It combines transactional processing with built-in columnar analytics, delivering up to 4x faster transactional performance and up to 100x faster analytical queries than standard PostgreSQL, without requiring any schema changes or ETL.

Exam trap

Google often tests the distinction between 'PostgreSQL-compatible' and 'PostgreSQL-based' — candidates mistakenly choose Cloud SQL for PostgreSQL because it is a managed PostgreSQL service, but they overlook the specific requirement for hybrid transactional and analytical workloads, which AlloyDB uniquely addresses with its integrated columnar engine.

How to eliminate wrong answers

Option A is wrong because Cloud Spanner is a globally distributed, strongly consistent relational database that is not PostgreSQL-compatible (it uses GoogleSQL or standard SQL with Spanner-specific extensions) and is optimized for horizontal scaling across regions, not for hybrid transactional/analytical workloads with PostgreSQL compatibility. Option B is wrong because Cloud SQL for PostgreSQL is a fully managed PostgreSQL service but is designed primarily for transactional (OLTP) workloads and lacks the built-in columnar engine and analytical acceleration needed for hybrid workloads, resulting in significantly slower analytical queries. Option C is wrong because BigQuery is a serverless, highly scalable data warehouse for analytical (OLAP) workloads, not a transactional database, and it is not PostgreSQL-compatible (it uses BigQuery SQL).

Full explanation →

100

MCQeasy

A data engineer needs to design a data pipeline that ingests streaming data from Cloud Pub/Sub, performs real-time aggregations, and loads the results into BigQuery for dashboarding. Which Google Cloud service should they use for the streaming aggregation step?

A.Cloud Functions

B.Dataflow

C.Cloud Dataproc

D.Cloud Composer

AnswerB

Dataflow provides serverless, autoscaling Apache Beam processing with exactly-once semantics, satisfying the stem's real-time aggregation requirement. Its native Pub/Sub source and BigQuery sink connectors handle windowed streaming aggregations without managing compute infrastructure, unlike Dataproc's cluster-based Spark or BigQuery's load-only ingestion.

Why this answer

Dataflow is a fully managed service for stream and batch processing that integrates with Pub/Sub and BigQuery. It supports exactly-once processing and low-latency streaming.

Full explanation →

101

MCQhard

Your MLOps pipeline uses Vertex AI Pipelines. You want to ensure that model training uses a consistent environment with specific Python package versions. Which approach best achieves this?

A.Include a requirements.txt file in the pipeline step and let Vertex AI install them

B.Use a pre-built deep learning container from Deep Learning Containers and install packages at runtime

C.Specify the Python version and package versions in the training job configuration

D.Build a custom container image with all dependencies and use it in the training step

AnswerD

Building a custom container image bakes exact Python package versions into an immutable artefact, so every pipeline run pulls the identical environment. This directly satisfies the stem's requirement for consistent, specific dependency versions, unlike runtime pip installs that can resolve differently between executions.

Why this answer

Building a custom container image with all dependencies ensures a fully deterministic and reproducible environment for model training. Vertex AI Pipelines executes each step as a container, so by pre-installing specific Python package versions into a custom image, you eliminate any risk of version drift or network issues during package installation at runtime. This approach aligns with MLOps best practices for environment consistency and is the most reliable method when exact package versions are critical.

Exam trap

Google Cloud often tests the distinction between runtime configuration (options A, B, C) and pre-built containerization (option D), trapping candidates who think specifying versions in a config file or installing at runtime is sufficient for full environment consistency in a pipeline context.

How to eliminate wrong answers

Option A is wrong because including a requirements.txt file and letting Vertex AI install them at runtime introduces variability; the installation may fail due to network issues, dependency conflicts, or changes in package repositories, and it does not guarantee the same environment across pipeline retries. Option B is wrong because using a pre-built deep learning container and installing packages at runtime still relies on runtime installation, which can lead to inconsistent environments if the installation process fails or if package versions are not pinned correctly. Option C is wrong because specifying Python version and package versions in the training job configuration only applies to AI Platform Training jobs, not to Vertex AI Pipelines; Vertex AI Pipelines runs steps as containers and does not natively support specifying package versions in the pipeline step configuration—the environment must be defined within the container image.

Full explanation →

102

MCQhard

A company uses BigQuery and wants to reduce query costs by using BI Engine for Looker Studio dashboards. The data is stored in a BigQuery dataset with 5 TB of frequently accessed tables. The dashboards run dozens of concurrent queries. What is the recommended approach to enable BI Engine acceleration?

A.Enable BI Engine by setting the dataset option 'enable_bi_engine=TRUE' in the dataset metadata.

B.Grant all Looker Studio users the 'biengine.user' IAM role on the project.

C.Create a reservation in the Administration panel and assign it to the project.

D.Create materialized views of the tables and connect Looker Studio to the views.

AnswerC

BI Engine acceleration requires a reservation assigned to the project; the 5 TB dataset and dozens of concurrent dashboard queries exceed the free per-project capacity, so a reservation guarantees dedicated in-memory cache for Looker Studio.

Why this answer

BI Engine is a reserved capacity service that caches data in memory. You must reserve capacity (amount of memory) for a specific BigQuery region, and then BI Engine automatically accelerates queries from Looker Studio and other BI tools. It does not require you to grant specific IAM roles to users (they just need BigQuery permissions) or to create materialized views.

You do not need to enable it per dataset; it works at the project/region level.

Full explanation →

103

MCQeasy

You want to quickly estimate the number of distinct visitors to your website from a large BigQuery table. Which function provides an approximate count with low latency?

A.APPROX_COUNT_DISTINCT

B.HyperLogLog++

C.COUNT(DISTINCT)

D.APPROX_QUANTILES

AnswerA

APPROX_COUNT_DISTINCT uses HyperLogLog++ sketches to estimate cardinality within a small, bounded memory footprint, returning results in seconds rather than the minutes an exact COUNT(DISTINCT) would need on a large table. This directly satisfies the stem's low-latency requirement for estimating distinct website visitors.

Why this answer

APPROX_COUNT_DISTINCT is a built-in BigQuery function that returns an approximate count of distinct values using a HyperLogLog++ sketch. It is designed for large-scale data, providing fast, low-latency results with a small error rate (typically <1%). This makes it ideal for quickly estimating distinct visitors without the overhead of exact counting.

Exam trap

PDE often tests the confusion between approximate and exact functions, leading candidates to choose COUNT(DISTINCT) for performance despite its high latency on large datasets.

How to eliminate wrong answers

Option B is wrong because HyperLogLog++ is the underlying algorithm that APPROX_COUNT_DISTINCT uses, not a directly callable function in BigQuery. Option C is wrong because COUNT(DISTINCT) computes an exact count, which requires shuffling all distinct values and can be very slow and resource-intensive on large tables. Option D is wrong because APPROX_QUANTILES is used to estimate quantiles (e.g., median, percentiles) of a numeric column, not to count distinct values.

Full explanation →

104

MCQeasy

You are using AI Platform Prediction (now Vertex AI) for online predictions. You notice that some requests are failing with a 503 status code. Which is the most likely cause?

A.The model is experiencing high traffic and the underlying nodes are still scaling up

B.The input data format does not match the model's expected schema

C.The project has exceeded its prediction requests quota

D.The service account used for prediction does not have the required permissions

AnswerA

High traffic triggers autoscaling, but new nodes take time to provision and load the model. During that window, existing nodes cannot absorb the load, so the service returns 503. This satisfies the stem's constraint: transient failures under sudden load, rather than a permanent configuration fault.

Why this answer

A 503 status code in Vertex AI (formerly AI Platform Prediction) indicates that the prediction service is temporarily unavailable, most commonly due to autoscaling latency. When a model receives a sudden spike in traffic, the underlying nodes (compute instances) may still be provisioning and initializing, causing requests to be rejected until the new nodes are ready to serve. This is a transient condition that resolves once scaling completes.

Exam trap

Google Cloud often tests the distinction between HTTP 503 (service unavailable, transient) and HTTP 429 (quota exceeded) or HTTP 400 (bad request), so candidates mistakenly attribute scaling issues to quota exhaustion or permission errors.

How to eliminate wrong answers

Option B is wrong because a mismatch in input data format (e.g., wrong tensor shape or feature names) would result in a 400 Bad Request error, not a 503. Option C is wrong because exceeding prediction request quota would return a 429 Too Many Requests error, not a 503. Option D is wrong because insufficient permissions (e.g., missing `aiplatform.predict` role) would cause a 403 Forbidden error, not a 503.

Full explanation →

105

MCQhard

A company processes large volumes of GPS sensor data stored in Cloud Storage. Each hour, they run an Apache Spark job that aggregates the data by geohash region. The job must be cost-effective and scale automatically. Currently, they are using a Dataproc cluster with preemptible workers. Which improvement would best reduce costs while maintaining performance?

A.Use a larger Dataproc cluster with standard workers

B.Migrate the job to BigQuery scheduled queries

C.Switch to Dataflow batch pipeline with Apache Beam

D.Use Dataproc Serverless Spark

AnswerD

Dataproc Serverless Spark bills per-second for actual workload consumption and provisions capacity automatically, eliminating idle cluster time and manual sizing. For hourly aggregation jobs, this removes the cost of preemptible workers sitting idle between runs while still scaling to demand.

Why this answer

Dataproc Serverless Spark (Option D) eliminates the need to manage a cluster, automatically scaling resources to match job demand and charging only for the resources consumed during execution. This removes the overhead of preemptible worker management and idle cluster costs, directly reducing expenses while maintaining performance for the hourly aggregation job.

Exam trap

Google Cloud often tests the misconception that migrating to a different processing engine (like Dataflow or BigQuery) is always the best cost-saving move, when in fact reusing existing Spark code on a serverless platform avoids migration costs and leverages the same API.

How to eliminate wrong answers

Option A is wrong because using a larger cluster with standard workers increases costs due to higher per-hour instance pricing and potential idle time, without addressing the cost inefficiency of preemptible workers. Option B is wrong because BigQuery scheduled queries are designed for SQL-based analytics on data already in BigQuery, not for processing large volumes of GPS sensor data stored in Cloud Storage with Apache Spark aggregations; migrating would require rewriting the Spark logic and may incur high BigQuery slot costs. Option C is wrong because while Dataflow batch pipelines with Apache Beam can process data cost-effectively, they require rewriting the existing Spark job into Beam, introducing development overhead and potential performance differences, whereas Dataproc Serverless Spark directly runs the existing Spark code without migration.

Full explanation →

106

MCQeasy

A company needs to stream real-time user click events from a web application to BigQuery for analysis. Which Google Cloud architecture is most suitable?

A.App Engine -> Pub/Sub -> Dataflow -> BigQuery

B.Cloud Scheduler -> BigQuery

C.Compute Engine -> Cloud Storage -> BigQuery

D.Cloud Functions -> BigQuery

AnswerA

Pub/Sub decouples ingestion from processing, absorbing bursty click streams without loss, while Dataflow provides windowed, exactly-once streaming transforms into BigQuery. This satisfies the real-time streaming requirement, unlike batch-only pipelines. App Engine hosts the web tier that publishes events, completing an end-to-end managed architecture.

Why this answer

It provides a fully managed, scalable, and decoupled architecture for ingesting real-time click events. Pub/Sub acts as a durable, asynchronous message buffer that can handle high-throughput streams, Dataflow (Apache Beam) processes the events in near real-time with exactly-once semantics, and BigQuery serves as the analytics warehouse. This pattern is the recommended Google Cloud approach for streaming analytics, as it decouples producers from consumers and supports auto-scaling.

Exam trap

The trap here is that candidates often choose Cloud Functions (Option D) thinking it is sufficient for real-time ingestion, but they overlook its execution timeout and lack of built-in streaming semantics, which makes it unsuitable for sustained high-throughput event pipelines.

How to eliminate wrong answers

Option B is wrong because Cloud Scheduler is a cron job service for triggering actions on a schedule, not a real-time event ingestion mechanism; it cannot stream continuous click events. Option C is wrong because Compute Engine and Cloud Storage are batch-oriented; writing events directly to Cloud Storage introduces latency and requires additional batch processing to load into BigQuery, making it unsuitable for real-time streaming. Option D is wrong because Cloud Functions has a 9-minute timeout and is designed for short-lived, event-driven compute, not for continuous, high-throughput streaming; it would also require custom code to buffer and batch writes to BigQuery, losing the managed streaming capabilities of Dataflow.

Full explanation →

107

MCQeasy

A healthcare analytics team must store patient records in BigQuery and prove to auditors that no one can read or modify the data without an explicit, logged authorization. They want to enforce column-level access so that only the billing team can see insurance_id, while the research team sees only de-identified columns. Which BigQuery feature should they implement?

A.Policy tags in a data taxonomy applied to the insurance_id column, with column-level access granted to the billing team.

B.Authorized views that join the patient table with a lookup table to mask insurance_id.

C.Row-level security policies that filter rows based on the SESSION_USER() function.

D.Customer-managed encryption keys stored in Cloud KMS and rotated every 90 days.

AnswerA

Policy tags let you classify sensitive columns and grant access through Data Catalog column-level IAM. Only principals with the fine-grained reader role on the policy tag can query insurance_id, and all access is logged. This enforces column-level access at the table level, so the research team is blocked from that column even if they can query the table, meeting the audit requirement.

Why this answer

Policy tags in a data taxonomy provide column-level access control in BigQuery. Applying a policy tag to insurance_id and granting the fine-grained reader role only to the billing team ensures the research team cannot query that column even though they can access the table. BigQuery logs policy tag enforcement, supporting the auditor's requirement for explicit, logged authorization.

Other features address rows, encryption, or views but not column-level authorization.

Exam trap

The trap here is equating encryption key management with access control, when keys protect data at rest rather than restricting column reads.

Full explanation →

108

MCQeasy

A data engineer runs this Dataflow template to load CSV files from Cloud Storage into BigQuery. The job fails with a 'File pattern not matching any files' error. What is the most likely cause?

A.The bucket name is incorrectly spelled

B.The CSV files are stored in a subdirectory that is not matched by the pattern

C.The template has a bug

D.The output table does not exist

AnswerB

The pattern only matches files at the specified path level, so files nested in a subdirectory are never enumerated. Broadening the pattern to include the subdirectory prefix, or using a recursive wildcard, resolves the 'no files matched' failure.

Why this answer

The error 'File pattern not matching any files' indicates that the file pattern specified in the Dataflow template does not resolve to any existing objects in Cloud Storage. If the CSV files are stored in a subdirectory (e.g., gs://bucket/subdir/*.csv) but the pattern only references the root (e.g., gs://bucket/*.csv), no files will be matched. This is the most likely cause because the pattern must explicitly include the subdirectory path.

Exam trap

Google Cloud often tests the distinction between file pattern matching errors and bucket-level errors, trapping candidates who confuse a missing subdirectory in the pattern with a misspelled bucket name.

How to eliminate wrong answers

Option A is wrong because an incorrectly spelled bucket name would result in a 'bucket not found' or 'access denied' error, not a 'file pattern not matching any files' error. Option C is wrong because the template is a well-tested Google-provided template; a bug is unlikely and would typically cause different errors (e.g., runtime exceptions). Option D is wrong because the output table not existing would cause a BigQuery table creation or write error, not a file pattern matching error in Cloud Storage.

Full explanation →

109

Multi-Selectmedium

A company is designing a data lake on Cloud Storage with BigLake tables for unified governance. Which TWO statements about BigLake are correct? (Choose 2.)

Select 2 answers

A.BigLake requires data to be loaded into BigQuery storage.

B.BigLake tables allow querying data in Cloud Storage using BigQuery without loading into BigQuery storage.

C.BigLake only supports structured data in BigQuery storage.

D.BigLake supports row-level and column-level security.

E.BigLake automatically converts CSV files to Parquet.

AnswersB, D

BigLake's external table capability lets BigQuery query Cloud Storage data in place, so no ingestion into BigQuery-managed storage is needed. This directly satisfies the stem's data lake design, where unified governance across open formats matters more than duplicating data, and it underpins the federated query pattern the scenario requires.

Why this answer

Option B is correct because BigLake tables are external tables that let BigQuery query data residing in Cloud Storage (Parquet, ORC, Avro, JSON, CSV, Iceberg, Delta) directly, without ingesting it into BigQuery's native storage. Option D is correct because BigLake extends BigQuery's fine-grained governance to external data, supporting row-level security and column-level security (via policy tags in Data Catalog) so access controls apply uniformly across the lakehouse. Option A is wrong because BigLake's defining feature is avoiding the load into BigQuery storage; that describes native BigQuery tables.

Option C is wrong because BigLake handles structured and semi-structured formats in Cloud Storage, not only structured data in BigQuery storage. Option E is wrong because BigLake does not perform automatic format conversion of CSV to Parquet; it queries files in their existing format.

Exam trap

A common misconception is that BigLake requires data to be loaded into BigQuery storage, when in fact it is designed to query external data in Cloud Storage directly.

Full explanation →

110

MCQeasy

You need to schedule a Dataproc Spark job to run at 2 AM every day, and upon completion, trigger a BigQuery load job. Which Cloud Composer operator should you use to run the Spark job?

A.DataflowPythonOperator

B.BigQueryOperator

C.DataprocClusterCreateOperator

D.DataprocSubmitJobOperator

AnswerD

DataprocSubmitJobOperator submits a Spark job to a Dataproc cluster and waits for completion, so it fits the scheduled 2 AM run. Its downstream task can then trigger the BigQuery load job within the same DAG.

Why this answer

The DataprocSubmitJobOperator is specifically designed to submit a job (e.g., a Spark job) to an existing Dataproc cluster. In this scenario, you need to run a Spark job on a scheduled basis, and Cloud Composer (Airflow) provides this operator to submit the job to Dataproc. After the Spark job completes, you can chain a BigQuery load operator to trigger the load, matching the requirement exactly.

Exam trap

The trap here is that candidates confuse operators that manage cluster lifecycle (like DataprocClusterCreateOperator) with operators that submit jobs, or they mistakenly think DataflowPythonOperator can run Spark jobs because both are data processing frameworks.

How to eliminate wrong answers

Option A is wrong because DataflowPythonOperator is used to run Apache Beam pipelines on Dataflow, not Spark jobs on Dataproc. Option B is wrong because BigQueryOperator is used to execute BigQuery SQL queries or load jobs, not to run Spark jobs. Option C is wrong because DataprocClusterCreateOperator is used to create a new Dataproc cluster, not to submit a job to an existing cluster; the question assumes the cluster already exists or is managed separately, and the focus is on submitting the Spark job.

Full explanation →

111

MCQhard

A company uses Cloud Pub/Sub to ingest events from multiple sources. They need to guarantee that each event is processed exactly once by downstream consumers. However, Pub/Sub guarantees at-least-once delivery. Which additional steps should they implement to achieve exactly-once processing?

A.Set the subscription's acknowledgment deadline to 0.

B.Enable message deduplication on the subscription.

C.Store each message's unique ID in a database and ignore duplicates.

D.Use a dead letter topic to capture duplicates.

AnswerC

Pub/Sub delivers at-least-once, so the same message can arrive repeatedly. Persisting each message's unique ID and discarding already-seen IDs gives idempotent deduplication at the consumer, converting at-least-once delivery into effectively exactly-once processing as the stem requires.

Why this answer

Pub/Sub's at-least-once delivery means a message can be redelivered if the ack isn't received in time or if the subscriber crashes after processing but before acking. To achieve exactly-once processing, the consumer must implement idempotency by tracking processed message IDs and discarding duplicates. Storing each message's unique ID in a database and ignoring duplicates ensures that even if the same message is delivered multiple times, it is only processed once.

This is the standard pattern for exactly-once semantics on top of at-least-once infrastructure.

Exam trap

PDE often tests the misconception that enabling a Pub/Sub feature (like deduplication or dead letter topics) can achieve exactly-once processing, when in fact exactly-once requires application-level idempotency.

How to eliminate wrong answers

Option A is wrong because setting the acknowledgment deadline to 0 would cause messages to be redelivered immediately, making duplicates more likely, not less. Option B is wrong because Pub/Sub does not have a subscription-level deduplication feature; deduplication is only available on the publish side when using ordering keys or message IDs, and it does not guarantee exactly-once processing on the subscriber side. Option D is wrong because a dead letter topic captures messages that cannot be processed after a certain number of attempts; it does not prevent duplicate processing and is unrelated to exactly-once semantics.

Full explanation →

112

MCQeasy

Which BigQuery function can be used to retrieve the value of a column from the previous row within a partition, ordered by a timestamp?

A.LAG()

B.FIRST_VALUE()

C.ROW_NUMBER()

D.LEAD()

AnswerA

LAG() is a navigation window function that returns a column's value from a preceding row within the same partition, evaluated by the ORDER BY timestamp. It satisfies the stem's requirement to retrieve the previous row's value, unlike LEAD(), which looks forward.

Why this answer

LAG() is a window function that returns the value of a column from a row that is a specified number of rows before the current row within the partition. LEAD() retrieves from a following row.

Full explanation →

113

MCQmedium

A data engineering team manages BigQuery datasets across multiple projects. They want to automatically detect and respond when a scheduled query fails, and they want the response to create an incident in their existing ticketing system. The team prefers minimal custom infrastructure and wants to use native Google Cloud tooling. Which approach should they use?

A.Schedule a Cloud Scheduler job that polls the BigQuery INFORMATION_SCHEMA.JOBS view and emails the team when errors are found.

B.Configure BigQuery to send failure notifications to a Cloud Storage bucket and use a Dataproc job to parse them.

C.Enable BigQuery audit logs, create a log-based alerting policy in Cloud Monitoring, and route the alert to a Pub/Sub topic that triggers a Cloud Function to open the ticket.

D.Use the BigQuery REST API from a long-running Compute Engine VM to poll for failed jobs and call the ticketing API.

AnswerC

BigQuery writes job failure information to Cloud Audit Logs. A log-based alerting policy in Cloud Monitoring can match failed query jobs and publish to a Pub/Sub topic, which invokes a Cloud Function that calls the ticketing API. This uses native tooling with minimal custom infrastructure and reliably captures scheduled query failures.

Why this answer

Cloud Audit Logs capture BigQuery job failures, and log-based alerting policies in Cloud Monitoring can trigger Pub/Sub notifications. A Cloud Function subscribed to the topic can call the ticketing API, creating incidents automatically. This event-driven chain uses managed services, avoids polling, and requires little custom code compared with the other options.

Exam trap

The trap here is choosing polling-based approaches, which add latency and maintenance, instead of event-driven log-based alerting that natively captures BigQuery failures.

Full explanation →

114

MCQeasy

A data scientist wants to test a new model version on a small percentage of traffic before full rollout. Which Vertex AI feature allows this?

A.A/B testing

B.Endpoint traffic splitting

C.Model monitoring

D.Model versioning with canary deployments

AnswerB

Endpoint traffic splitting lets you assign percentage weights to multiple deployed model versions behind one endpoint, routing a small share of requests to the new version. This directly satisfies the gradual-rollout constraint before promoting it to full traffic.

Why this answer

Vertex AI Endpoint traffic splitting allows you to route a specified percentage of inference requests to different model versions deployed on the same endpoint. This enables gradual rollout by directing a small fraction of traffic (e.g., 5%) to the new model while the rest goes to the current version, without needing separate endpoints or manual routing logic.

Exam trap

The trap here is that candidates confuse the conceptual practice of 'canary deployments' (Option D) with the specific Vertex AI feature 'endpoint traffic splitting' (Option B), but the exam expects the exact feature name as defined in the Google Cloud documentation.

How to eliminate wrong answers

Option A is wrong because A/B testing in Vertex AI is a feature for comparing model performance metrics (like accuracy or latency) by splitting traffic, but it is not the feature that directly enables traffic splitting itself—traffic splitting is the underlying mechanism, and A/B testing is a higher-level evaluation tool built on top of it. Option C is wrong because Model monitoring is used to detect data drift, feature skew, and prediction anomalies on deployed models, not to control traffic distribution between versions. Option D is wrong because model versioning with canary deployments is a conceptual practice, not a specific Vertex AI feature; the actual feature that implements canary-style traffic routing is endpoint traffic splitting, which is the correct answer.

Full explanation →

115

Multi-Selecteasy

A data engineer needs to schedule a recurring transfer of data from a partner's Amazon S3 bucket to a Cloud Storage bucket for further processing. Which THREE components or configurations are necessary? (Choose 3)

Select 3 answers

A.A VPC network configuration

B.Specification of the source S3 bucket and destination GCS bucket

C.A scheduled transfer job in Storage Transfer Service

D.A Pub/Sub topic to notify completion

E.Authentication credentials for AWS (e.g., access key and secret)

AnswersB, C, E

Storage Transfer Service requires the source and destination locations to be defined so it knows which S3 bucket to read from and which Cloud Storage bucket to write into. Specifying both endpoints is the fundamental configuration the transfer job acts upon.

Why this answer

Storage Transfer Service is the Google Cloud service designed to move data from external sources like Amazon S3 into Cloud Storage, so the transfer job must be defined as a scheduled transfer job (option C), which controls the recurring execution. Every transfer job requires the source and destination locations to be identified, so specifying the source S3 bucket and destination GCS bucket (option B) is mandatory for the job to know what to copy and where to place it. Because the source is an external AWS S3 bucket, Storage Transfer Service must authenticate to AWS, which requires AWS credentials such as an access key ID and secret access key (option E).

A VPC network configuration (option A) is not required because Storage Transfer Service is a fully managed service that does not need customer VPC networking for S3-to-GCS transfers. A Pub/Sub topic (option D) is optional, used only if you want notifications about transfer completion, not a necessary component for the transfer itself.

Exam trap

PDE often tests the misconception that additional components like VPC or Pub/Sub are required for Storage Transfer Service, when only the source/destination, job, and credentials are necessary.

Full explanation →

116

MCQeasy

A startup runs an operational analytics dashboard on a Cloud SQL for MySQL instance. The dashboard issues many short read queries that are slowing down the write-heavy transactional workload on the primary instance. The team wants to offload reads without changing application write logic and without re-architecting to a different database engine. What should the data engineer configure?

A.Create one or more read replicas of the Cloud SQL instance and direct the dashboard's read queries to the replica connection.

B.Enable high availability on the primary instance so a standby instance can absorb the dashboard's read queries.

C.Migrate the dashboard to query a BigQuery federated external table that reads directly from the Cloud SQL instance.

D.Increase the number of vCPUs and memory on the primary instance to handle the combined read and write load.

AnswerA

Cloud SQL read replicas serve read-only traffic from a separate instance that receives asynchronous replication from the primary. Pointing the dashboard at the replica endpoint removes those read queries from the primary, restoring write throughput, while the application's write path continues to target the primary unchanged. This is the standard, minimal-change way to scale reads for Cloud SQL.

Why this answer

Read replicas are the intended Cloud SQL feature for scaling read traffic: the dashboard connects to a replica endpoint, the transactional application keeps writing to the primary, and no write-path code changes are required. A high-availability standby is passive and cannot serve reads, vertical scaling keeps both workloads on the same instance, and federated BigQuery queries still hit the source database. The replica approach isolates the dashboard's short read queries from the write-heavy workload.

Exam trap

The trap here is confusing a Cloud SQL high-availability standby, which only activates for failover, with a read replica, which actively serves read-only queries.

Full explanation →

117

MCQmedium

A retail company uses a Vertex AI endpoint to serve product recommendations. The model is a TensorFlow model deployed with a custom container. Recently, users have reported that recommendations are stale. The model is retrained daily using Vertex AI Pipelines. The pipeline completes successfully, but the endpoint continues to serve the old model. The team checks the pipeline logs and sees that the new model is uploaded to the Vertex AI Model Registry. The endpoint has traffic split set to 100% for the old model. The team needs to update the endpoint to serve the new model version. What should they do?

A.Check the pipeline for errors in the deployment step

B.Re-upload the model with a different version ID

C.Redeploy the same model to the endpoint

D.Update the endpoint to deploy the new model version from the registry and adjust traffic split

AnswerD

The endpoint still routes 100% of traffic to the old model, so uploading to the registry alone changes nothing. Deploying the new version to the endpoint and shifting the traffic split directs live inference to it.

Why this answer

The pipeline successfully uploaded the new model to the Vertex AI Model Registry, but the endpoint still has its traffic split configured to 100% for the old model. To serve the new model, the team must explicitly update the endpoint to deploy the new model version from the registry and adjust the traffic split to route 100% of traffic to it. This is a standard operational step in Vertex AI: uploading a model does not automatically update the endpoint's deployment or traffic allocation.

Exam trap

Google Cloud often tests the misconception that uploading a new model version to the registry automatically updates the endpoint's serving configuration, when in fact the traffic split must be explicitly adjusted to route requests to the new model.

How to eliminate wrong answers

Option A is wrong because the pipeline logs show no errors in the deployment step; the model was successfully uploaded to the registry, so checking for errors is unnecessary and misdiagnoses the issue. Option B is wrong because re-uploading the model with a different version ID does not change the endpoint's deployment or traffic split; the endpoint still points to the old model version. Option C is wrong because redeploying the same model (the old version) to the endpoint would not serve the new model; the team needs to deploy the new model version from the registry, not redeploy the old one.

Full explanation →

118

Multi-Selecthard

Which THREE factors should be considered when designing a Vertex AI Pipeline for continuous training?

Select 3 answers

A.Cost of training and infrastructure

B.Debugging tools like Cloud Debugger

C.Trigger mechanism (time-based or event-based)

D.Number of model versions to keep

E.Data freshness and staleness tolerance

AnswersA, C, E

Budget impacts resource selection.

Why this answer

Cost of training and infrastructure (A) is correct because Vertex AI Pipelines incur compute costs for each pipeline run, including training, data processing, and orchestration. Continuous training amplifies these costs, so you must consider budget constraints, resource optimization (e.g., using preemptible VMs), and cost monitoring to avoid unexpected bills.

Exam trap

Google Cloud often tests the distinction between operational pipeline design factors (triggers, cost, data freshness) and peripheral management tasks (versioning, debugging tools), leading candidates to incorrectly select options like D or B that are valid but not core to pipeline design.

Full explanation →

119

MCQhard

A media company streams playback telemetry through Pub/Sub into a Dataflow pipeline that writes to BigQuery. During prime-time peaks, the pipeline's BigQuery write step shows growing latency and the job repeatedly reports that it is backing off on insert retries. The team wants to reduce write pressure without changing the downstream table schema or losing exactly-once semantics. What should they do?

A.Enable Dataflow Shuffle and set the pipeline's disk size larger so the write stage can buffer more rows before retrying.

B.Change the pipeline to use a BigQuery load job triggered every 15 minutes by writing micro-batches to Cloud Storage first.

C.Switch the sink to the BigQuery Storage Write API with the exactly-once stream type and group records into batches before writing.

D.Increase the number of BigQuery Streaming Inserts API calls by adding more Dataflow workers so each record is inserted individually.

AnswerC

The Storage Write API supports an exactly-once stream type that uses offsets to deduplicate on retry, preserving exactly-once semantics while allowing high-throughput batched appends. Batching amortizes request overhead and dramatically lowers the per-row cost and quota pressure that caused the backoff. Because it writes to the same table schema, no downstream change is required, and it is the recommended high-volume ingestion path for Dataflow into BigQuery.

Why this answer

Backoff on insert retries signals that the write path is saturating BigQuery's streaming ingestion limits. The Storage Write API's exactly-once stream type preserves exactly-once semantics through offset-based deduplication, and batching rows into larger appends cuts request volume and quota pressure. This keeps the existing table schema and the streaming model intact while removing the write bottleneck that was throttling the pipeline during peak load.

Exam trap

The trap here is scaling out workers or tuning Dataflow shuffle and disk settings to fix a problem that actually originates in the BigQuery ingestion API's quotas and retry behavior.

Full explanation →

120

MCQeasy

A mobile app needs an offline-first NoSQL database that syncs data across devices when connectivity is available. Which Google Cloud database meets these requirements?

A.Memorystore

B.Cloud SQL

C.Bigtable

D.Firestore

AnswerD

Firestore provides offline persistence through local caching, letting mobile apps read and write without connectivity, then synchronises changes automatically once the device reconnects. Its real-time listeners and multi-device sync satisfy the offline-first NoSQL requirement directly, unlike Cloud SQL or Bigtable, which demand continuous network access.

Why this answer

Firestore is a NoSQL, serverless, offline-first database that automatically syncs data across devices when connectivity is restored. It provides built-in offline persistence and real-time synchronization, making it ideal for mobile apps that need to work offline and sync later.

Exam trap

The trap here is that candidates may confuse Google Bigtable's NoSQL label with a mobile-friendly NoSQL database, overlooking that Bigtable is designed for high-throughput analytical workloads, not for offline-first mobile sync with real-time listeners.

How to eliminate wrong answers

Option A is wrong because Memorystore is a fully managed in-memory cache (Redis/Memcached) designed for caching and session storage, not a persistent NoSQL database with offline sync capabilities. Option B is wrong because Cloud SQL is a relational (SQL) database, not a NoSQL database, and it does not offer offline-first or cross-device sync features. Option C is wrong because Bigtable is a wide-column NoSQL database optimized for large analytical workloads, not for mobile app offline sync with real-time updates.

Full explanation →

121

MCQeasy

You are loading 10 GB of daily CSV files from a GCS bucket into a BigQuery table. The files contain some malformed rows that you want to skip. Which BigQuery load configuration should you use?

A.Use the 'skip_leading_rows' option.

B.Use the 'ignore_unknown_values' option.

C.Use the 'max_bad_records' option set to a value like 10.

D.Use the 'allow_jagged_rows' option.

AnswerC

Setting max_bad_records to 10 lets the load job tolerate up to ten malformed rows per file, skipping them while importing the valid data. Without it, any single bad row aborts the entire 10 GB load.

Why this answer

The 'max_bad_records' option allows you to specify the maximum number of bad records that BigQuery can ignore during a load job. Setting it to a value like 10 means that up to 10 malformed rows will be skipped; if more than 10 bad records are encountered, the job fails. This is the correct configuration to skip a limited number of malformed rows while ensuring data quality.

Exam trap

PDE often tests the confusion between options that handle different types of CSV parsing issues, such as skipping headers versus skipping malformed rows, and candidates may incorrectly choose 'ignore_unknown_values' or 'allow_jagged_rows' for general malformed data.

How to eliminate wrong answers

Option A is wrong because 'skip_leading_rows' is used to skip header rows in CSV files, not malformed data rows. Option B is wrong because 'ignore_unknown_values' ignores extra columns in the data that do not match the schema, but it does not skip rows with malformed data (e.g., type mismatches). Option D is wrong because 'allow_jagged_rows' allows rows with missing trailing columns, but it does not handle other types of malformed data like invalid values.

Full explanation →

122

MCQhard

A retail analytics team uses BigQuery to store point-of-sale transactions in a partitioned table. They need to optimize query performance for dashboards that filter on store_id and product_category, and they want to minimize the amount of data scanned. The table is partitioned by transaction_date and has a clustered column of store_id. Queries frequently filter on product_category as well. Which change should the data engineer make to improve performance?

A.Create a materialized view that pre-aggregates sales by store_id and product_category

B.Add product_category as a second clustering column after store_id

C.Change the partitioning column to product_category and keep store_id as the clustered column

D.Convert the table to use time-unit partitioning on transaction_date with a daily granularity and remove clustering

AnswerB

BigQuery supports up to four clustering columns, and the order matters: data is sorted by the first column, then the second, and so on. Adding product_category as a second clustering column allows BigQuery to prune blocks more effectively when queries filter on both store_id and product_category. This reduces the amount of data scanned and improves dashboard performance without changing the partitioning strategy, which remains optimal for date filters.

Why this answer

BigQuery clustering sorts data within partitions by the specified columns, and queries that filter on those columns can skip blocks. Adding product_category as a second clustering column after store_id lets the engine prune more effectively when both columns appear in filters. The partitioning column remains transaction_date, which is ideal for date-range dashboards.

This change reduces bytes scanned and improves latency without altering the partitioning scheme.

Exam trap

The trap here is thinking that clustering columns must be unique or that you can only cluster on one column, when BigQuery actually supports up to four clustering columns and order affects pruning efficiency.

Full explanation →

123

MCQmedium

A data pipeline processes JSON files from Cloud Storage, transforms them using Apache Beam, and writes the output to BigQuery. Some records are malformed and cause the pipeline to fail. How should the engineer handle these errors to ensure the pipeline continues processing while preserving the malformed records for analysis?

A.Set the pipeline to retry malformed records indefinitely until they succeed.

B.Use a side input to send malformed records to a dead letter queue in Pub/Sub for later reprocessing.

C.Log the malformed records to Stackdriver and skip them in the pipeline.

D.Catch exceptions in a DoFn and write the malformed records to a separate Cloud Storage bucket using a FileIO sink.

AnswerD

Wrapping parsing in a DoFn try/except lets the pipeline emit valid records downstream while routing malformed ones to a separate Cloud Storage bucket via FileIO. This satisfies both constraints: the pipeline continues processing, and the bad records are preserved for later analysis.

Why this answer

It allows the pipeline to continue processing by catching exceptions within a DoFn and writing malformed records to a separate Cloud Storage bucket using a FileIO sink. This preserves the malformed records for later analysis without blocking the main data flow, which is a standard pattern in Apache Beam for handling dead-letter records. The approach ensures fault tolerance while maintaining data integrity for debugging.

Exam trap

A common misconception in Google Cloud data pipelines is that error handling should involve retrying indefinitely or simply logging errors to Cloud Logging, but the correct approach is to isolate and persist malformed records to a durable sink like Cloud Storage for later analysis.

How to eliminate wrong answers

Option A is wrong because retrying malformed records indefinitely would cause the pipeline to hang or exhaust resources, as malformed records will never succeed due to inherent data issues. Option B is wrong because using a side input to send malformed records to a Pub/Sub dead letter queue is not a direct pattern in Apache Beam; side inputs are for broadcasting data to all elements, not for error handling, and Pub/Sub would require additional setup and does not inherently preserve the records for analysis without a separate sink. Option C is wrong because logging malformed records to Stackdriver and skipping them loses the data permanently, as logs are not designed for structured storage or reprocessing of the original records.

Full explanation →

124

MCQmedium

A company uses Vertex AI to serve a model. They notice that some predictions are incorrect due to data drift. What is the best way to detect and retrain the model automatically?

A.Store predictions in BigQuery and run scheduled queries

B.Create a Cloud Monitoring dashboard

C.Set up Cloud Logging metrics to monitor predictions

D.Use Vertex AI Model Monitoring with alerts and retraining pipeline

AnswerD

Vertex AI Model Monitoring compares serving input distributions against training baselines and fires alerts on drift, which can trigger a retraining pipeline automatically. This satisfies the requirement to both detect drift and retrain without manual intervention.

Why this answer

Vertex AI Model Monitoring is specifically designed to detect data drift and feature skew in production models. It can be configured to send alerts and trigger an automated retraining pipeline via Cloud Functions or Vertex AI Pipelines, enabling continuous model improvement without manual intervention. This directly addresses the need for automatic detection and retraining in response to data drift.

Exam trap

The trap here is that candidates may confuse general monitoring tools (Cloud Monitoring, Cloud Logging) with the specialized drift detection and automated retraining capabilities of Vertex AI Model Monitoring, assuming any monitoring solution can trigger retraining without native integration.

How to eliminate wrong answers

Option A is wrong because storing predictions in BigQuery and running scheduled queries is a manual, batch-oriented approach that does not provide real-time drift detection or automated retraining; it requires custom code and lacks native integration with Vertex AI's monitoring capabilities. Option B is wrong because Cloud Monitoring dashboards visualize metrics but do not inherently detect data drift or trigger retraining pipelines; they are for observability, not automated action. Option C is wrong because Cloud Logging metrics can track prediction logs but are not designed for statistical drift analysis (e.g., distribution comparisons) and cannot directly initiate retraining workflows without additional custom logic.

Full explanation →

125

Multi-Selectmedium

A data engineer is designing a streaming pipeline with Cloud Pub/Sub and Cloud Dataflow. They need to guarantee at-least-once delivery and handle occasional duplicates. Which TWO configurations should they implement?

Select 2 answers

A.Use idempotent sinks

B.Use global windows with triggers

C.Use fixed windows

D.Use at-least-once Pub/Sub subscription

E.Enable Dataflow Streaming Engine

AnswersA, D

Idempotent sinks allow safe duplicate writes, ensuring exactly-once effect despite duplicates.

Why this answer

Idempotent sinks (e.g., BigQuery with insertId, Cloud Storage with object generation numbers) allow the pipeline to safely process duplicate records without causing data corruption or double-counting. In a streaming pipeline with at-least-once semantics, duplicates are inevitable, and idempotent sinks ensure that repeated writes produce the same result as a single write, maintaining data consistency.

Exam trap

Google Cloud often tests the misconception that windowing strategies (global or fixed) or execution engine features (Streaming Engine) can substitute for explicit delivery guarantees and idempotent sinks, when in fact they address entirely different concerns.

Full explanation →

126

MCQmedium

Your Looker dashboard uses a BigQuery connection. You notice that some queries take over a minute. Which service can you enable to cache results in memory for sub-second Looker queries?

A.BigQuery BI Engine

B.Cloud SQL

C.Cloud Bigtable

D.Cloud Memorystore

AnswerA

BigQuery BI Engine reserves memory for sub-second query acceleration and caches frequently accessed data, so repeated Looker queries against the BigQuery connection return in under a second. It satisfies the stated requirement of in-memory caching rather than relying on disk-based results cache.

Why this answer

BigQuery BI Engine is an in-memory analysis service that caches frequently accessed BigQuery data and accelerates queries, delivering sub-second response times for Looker dashboards. It integrates natively with BigQuery and Looker, so enabling BI Engine on the relevant project or dataset is the correct fix. Other options are separate database services that do not accelerate BigQuery queries.

Exam trap

The trap is assuming any caching or in-memory service (Memorystore, Bigtable) will accelerate BigQuery — only BI Engine is purpose-built to cache BigQuery data in memory for Looker.

How to eliminate wrong answers

Option B is wrong because Cloud SQL is a managed relational database (MySQL/PostgreSQL/SQL Server) and does not cache or accelerate BigQuery query results. Option C is wrong because Cloud Bigtable is a NoSQL wide-column store for high-throughput workloads, not an in-memory accelerator for BigQuery. Option D is wrong because Cloud Memorystore is a managed Redis/Memcached service; while it can cache application data, it does not transparently accelerate BigQuery SQL executed by Looker.

Full explanation →

127

Drag & Dropmedium

Drag and drop the steps to create a Cloud Function triggered by Cloud Storage events into the correct order.

Drag or tap steps into the slots.

Steps

Order

1Step 1

2Step 2

3Step 3

4Step 4

Why this order

The correct sequence for creating a Cloud Function triggered by Cloud Storage events is: first ensure the bucket exists (create if needed), then write the function code, deploy the function with the appropriate event trigger (e.g., google.storage.object.finalize), and finally test by uploading a file. Common mistakes include deploying before creating the bucket, testing before deployment, or deploying without code.

Full explanation →

128

MCQmedium

You have deployed a classification model on Vertex AI Endpoints. The model's training data had a balanced class distribution, but over time, the production data has shifted such that one class appears 90% of the time. The model's overall accuracy remains high, but the recall for the minority class has dropped significantly. What is the best approach to detect and address this issue?

A.Retrain the model daily on the entire historical dataset

B.Set up Vertex AI Model Monitoring to detect skew and drift, and retrain using a sliding window of recent data

C.Increase the number of replicas on the endpoint to reduce latency

D.Adjust the decision threshold to improve minority class recall

AnswerB

Vertex AI Model Monitoring detects training-serving skew and drift against a baseline, surfacing the 90/10 class shift. Retraining on a sliding window of recent data rebalances the learned decision boundary, restoring minority-class recall that aggregate accuracy masks.

Why this answer

Vertex AI Model Monitoring is specifically designed to detect skew and drift between training and serving data. In this scenario, the production data has shifted to 90% of one class, which is a clear case of data drift. By setting up monitoring, you can be alerted to this drift and then retrain the model using a sliding window of recent data, which adapts to the new distribution without requiring full retraining on the entire historical dataset.

This approach directly addresses the root cause—the shift in class distribution—rather than just treating symptoms.

Exam trap

Google Cloud often tests the distinction between monitoring/detection (Model Monitoring) and reactive fixes (threshold tuning), where candidates mistakenly choose a quick fix like adjusting the decision threshold instead of addressing the root cause of data drift.

How to eliminate wrong answers

Option A is wrong because retraining daily on the entire historical dataset is computationally expensive and does not prioritize recent data; it would still include the old balanced distribution, potentially diluting the model's ability to adapt to the new skewed production data. Option C is wrong because increasing the number of replicas on the endpoint reduces latency and improves throughput, but it does not address data drift or the drop in minority class recall; it is a scaling solution, not a monitoring or retraining solution. Option D is wrong because adjusting the decision threshold can improve recall for the minority class in the short term, but it does not fix the underlying model's inability to generalize to the shifted data distribution; it is a band-aid that may hurt precision and overall model performance.

Full explanation →

129

MCQhard

You manage a BigQuery reservation with 500 baseline slots and autoscaling up to 2000 slots. Your team runs a mix of interactive queries and batch load jobs. During peak hours, you notice that interactive queries are throttled when autoscaling slots are consumed by long-running batch loads. How can you ensure interactive queries get priority access to slots?

A.Create a separate reservation for interactive queries with a higher priority assignment.

B.Reduce the baseline slots to 200 and rely solely on autoscaling.

C.Switch to on-demand pricing to eliminate slot contention.

D.Set the autoscaling max to 1000 slots for batch jobs.

AnswerA

Separate reservations isolate slot pools, so batch load jobs cannot consume the interactive reservation's baseline or autoscaled slots. Assigning interactive queries higher priority within their own reservation guarantees they are scheduled ahead of batch work.

Why this answer

BigQuery reservations allow you to create separate reservations for different workloads (e.g., interactive queries vs. batch loads) and assign them different priority levels. By creating a dedicated reservation for interactive queries with a higher priority, you ensure that interactive queries get preferential access to slots, even when autoscaling slots are consumed by long-running batch jobs. This directly addresses the contention issue without reducing overall capacity.

Exam trap

Google often tests the misconception that autoscaling alone or reducing baseline slots can solve priority issues, but the key is that without separate reservations and explicit priority assignments, all jobs compete equally for the same pool of slots.

How to eliminate wrong answers

Option B is wrong because reducing baseline slots to 200 and relying solely on autoscaling does not solve the priority issue; autoscaling slots are shared and batch jobs could still consume them, leading to the same throttling of interactive queries. Option C is wrong because switching to on-demand pricing eliminates slot reservations entirely, meaning you lose the ability to guarantee capacity or prioritize workloads, and you may face unpredictable performance and higher costs. Option D is wrong because setting the autoscaling max to 1000 slots for batch jobs does not prevent batch jobs from consuming all available slots; it only limits the maximum they can use, but without priority assignment, interactive queries can still be throttled if batch jobs fill the reservation.

Full explanation →

130

Multi-Selectmedium

You are building a BigQuery table that contains nested and repeated fields (e.g., order with line items). You need to write a query that counts the number of line items per order. Which TWO SQL functions/techniques can you use?

Select 2 answers

A.Window function ROW_NUMBER

B.UNNEST with COUNT

C.STRUCT with aggregation

D.ARRAY_LENGTH

E.SELECT * EXCEPT

AnswersB, D

UNNEST flattens the repeated line_items array into individual rows, letting COUNT aggregate them per order. This directly satisfies the stem's requirement to count line items within nested, repeated fields, since COUNT alone cannot traverse array elements without first unnesting them into a queryable row set.

Why this answer

UNNEST flattens the repeated line items array into individual rows, allowing COUNT to aggregate the number of line items per order. Option D is correct because ARRAY_LENGTH directly returns the number of elements in the repeated field array, which corresponds to the line item count.

Exam trap

Google often tests the distinction between functions that operate on arrays directly (like ARRAY_LENGTH) versus those that require row-level expansion (like UNNEST), and candidates may mistakenly choose window functions or STRUCT-based aggregation that do not directly count array elements.

Full explanation →

131

MCQmedium

A data engineer needs to migrate 200 TB of on-premises Oracle data to BigQuery. The network bandwidth is limited to 100 Mbps, and the data must be loaded within 2 weeks. Which Google Cloud service is most appropriate for the initial data transfer?

A.Transfer Appliance

B.BigQuery Data Transfer Service for Oracle

C.Datastream

D.Storage Transfer Service

AnswerA

At 100 Mbps, 200 TB needs roughly 185 days of continuous transfer, far beyond the two-week deadline. Transfer Appliance ships data physically, bypassing the bandwidth constraint entirely, then uploads to Cloud Storage for loading into BigQuery.

Why this answer

Transfer Appliance is correct because it is a physical device designed for large-scale offline data transfers when network bandwidth is insufficient. With 200 TB at 100 Mbps, the theoretical transfer time exceeds 190 days, far beyond the 2-week window. Transfer Appliance allows shipping the data directly to Google, bypassing network constraints entirely.

Exam trap

The trap here is that candidates may assume online services like Storage Transfer Service or BigQuery Data Transfer Service can handle large volumes if given enough time, ignoring the hard bandwidth calculation that proves 200 TB at 100 Mbps is impossible within 2 weeks.

How to eliminate wrong answers

Option B is wrong because BigQuery Data Transfer Service for Oracle is a scheduled, incremental transfer service that relies on network connectivity and cannot handle the initial bulk load of 200 TB within the bandwidth limit. Option C is wrong because Datastream is a real-time change data capture (CDC) service for streaming changes, not designed for initial bulk transfers of large datasets. Option D is wrong because Storage Transfer Service is an online transfer tool that moves data over the network, which would be bottlenecked by the 100 Mbps link and cannot complete 200 TB within 2 weeks.

Full explanation →

132

MCQmedium

Your streaming Dataflow pipeline reads from Pub/Sub, enriches data with a side input, and writes to BigQuery. You need to update the enrichment logic without draining the pipeline, to minimize data loss and maintain exactly-once semantics. What should you do?

A.Cancel the pipeline and create a new one with the updated code.

B.Stop the pipeline, update the code, and restart from the latest snapshot.

C.Use the Dataflow job update mechanism to replace the pipeline with a new version.

D.Drain the pipeline, update the code, and restart with the same job ID.

AnswerC

Dataflow allows updating a streaming pipeline with a new job graph, preserving state and exactly-once processing.

Why this answer

The Dataflow job update mechanism allows you to replace a running pipeline's code with a new version without draining or stopping it, preserving the existing state and minimizing data loss. This mechanism supports exactly-once semantics by ensuring that all in-flight elements are processed exactly once, even after the update, by maintaining the pipeline's checkpoint and watermark state.

Exam trap

The trap here is that candidates often confuse the Dataflow job update mechanism with draining or snapshot-based restarts, not realizing that Dataflow's update feature is specifically designed to allow in-place code changes without data loss or reprocessing.

How to eliminate wrong answers

Option A is wrong because canceling the pipeline would discard all in-flight data and state, leading to data loss and violating exactly-once semantics. Option B is wrong because stopping the pipeline and restarting from a snapshot is not a supported operation in Dataflow; snapshots are used for draining or saving state, but restarting from a snapshot does not guarantee exactly-once processing and can cause data duplication or loss. Option D is wrong because draining the pipeline would allow it to finish processing all existing data before stopping, but then you must create a new pipeline with a new job ID; restarting with the same job ID is not possible after draining, and the drain process itself can cause data loss if not handled correctly.

Full explanation →

133

Multi-Selecteasy

A company is designing a data processing system that must handle both batch and streaming workloads with unified pipeline code. Which two Google Cloud services are most suitable for implementing a unified batch and streaming pipeline? (Choose TWO.)

Select 2 answers

A.Cloud Data Fusion

B.BigQuery

C.Apache Beam SDK

D.Cloud Dataflow

E.Cloud Dataproc

AnswersC, D

Beam is the unified model; Dataflow is one runner.

Why this answer

Apache Beam SDK (C) provides a unified programming model that allows developers to write a single pipeline that can execute in both batch and streaming modes without code changes. It abstracts the underlying execution engine, making it the correct choice for unified pipeline code.

Exam trap

Google Cloud often tests the misconception that Cloud Data Fusion or Cloud Dataproc can achieve unified batch and streaming with a single codebase, but only Apache Beam SDK combined with Cloud Dataflow provides the native programming model and execution engine for this requirement.

Full explanation →

134

MCQeasy

An organization needs to store transactional data for a global e-commerce platform with strong consistency across regions and an SLA of 99.999% availability. The application requires SQL semantics with horizontal scaling. Which Google Cloud database should they choose?

A.Firestore

B.Cloud SQL

C.Cloud Spanner

D.Cloud Bigtable

AnswerC

Cloud Spanner is the only Google Cloud database delivering external strong consistency alongside horizontal scaling via TrueTime, satisfying the 99.999% SLA and SQL semantics required. Regional Cloud SQL or Firestore cannot scale horizontally while preserving global transactional consistency.

Why this answer

Cloud Spanner is the correct choice because it provides globally distributed, strongly consistent SQL semantics with horizontal scaling and a 99.999% availability SLA. It uses synchronous replication and the TrueTime API to ensure external consistency across regions, meeting the strict consistency and uptime requirements of a global e-commerce platform.

Exam trap

The trap here is that candidates often confuse Cloud Spanner with Cloud SQL, assuming any SQL database can scale horizontally, but Cloud SQL is a single-region, vertically scaled service, while Cloud Spanner is the only Google Cloud database that combines SQL, horizontal scaling, and global strong consistency with a 99.999% SLA.

How to eliminate wrong answers

Option A is wrong because Firestore is a NoSQL document database that does not support SQL semantics; it offers strong consistency only within a single region and lacks the global consistency and 99.999% SLA required. Option B is wrong because Cloud SQL is a traditional relational database that supports SQL but cannot horizontally scale across regions; it is limited to a single region and provides up to 99.95% availability, not 99.999%. Option D is wrong because Cloud Bigtable is a NoSQL wide-column database that does not support SQL semantics and provides only eventual consistency, not the strong consistency required for transactional data.

Full explanation →

135

MCQeasy

A data pipeline using Dataflow processes streaming data. Late-arriving events are currently being dropped. How should the team modify the pipeline to ensure late data is processed correctly?

A.Use side inputs to join late data with the main stream.

B.Use streaming inserts into BigQuery and ignore late data.

C.Configure triggers with allowed lateness and accumulation of late firings.

D.Increase the window duration to cover late data.

AnswerC

Allowed lateness extends the window's lifetime so late-arriving events still fall within it, while accumulation of late firings emits updated results as each late element arrives. Together these satisfy the requirement to process rather than drop late data.

Why this answer

Dataflow's triggers allow you to configure allowed lateness and accumulation of late firings, which ensures that late-arriving events are still processed within the correct window. By setting `withAllowedLateness` and specifying accumulation mode (e.g., `ACCUMULATING`), the pipeline will emit additional panes for late data and merge them with the existing window results, preventing data loss.

Exam trap

A common misconception is that simply increasing the window duration or using side inputs can handle late data in Dataflow, when in fact only trigger-based mechanisms with allowed lateness and accumulation provide the precise control needed for streaming late-arriving events.

How to eliminate wrong answers

Option A is wrong because side inputs are used to enrich the main stream with static or slowly changing reference data, not to handle late-arriving events; they do not provide a mechanism to re-process or accumulate late data within a window. Option B is wrong because streaming inserts into BigQuery do not inherently handle late data; ignoring late data contradicts the requirement to ensure it is processed correctly, and BigQuery's streaming buffer has no built-in late-data handling for windowed aggregations. Option D is wrong because increasing the window duration only shifts the problem by making the window larger, but late data beyond the new window boundary will still be dropped; it does not provide a configurable lateness threshold or accumulation of late firings.

Full explanation →

136

MCQmedium

A retail analytics team stores transaction data in a BigQuery table partitioned by day on a DATE column named transaction_date. Analysts repeatedly run queries that filter on transaction_date and also on a high-cardinality STRING column named store_id. These queries scan far more data than expected, and slot usage is high. The team wants to reduce bytes processed without changing query results. What should they do?

A.Create a clustered table on store_id, so BigQuery can prune blocks within each transaction_date partition.

B.Create a materialized view that precomputes all transactions grouped by store_id and transaction_date.

C.Export the table to Cloud Storage as Avro and query it with an external table that applies store_id predicates.

D.Change the table to require a partition filter, forcing analysts to always specify transaction_date.

AnswerA

Clustering on store_id sorts data within each partition into storage blocks, and BigQuery uses the cluster column to skip blocks that cannot match the store_id filter. Because queries already filter on transaction_date, partition pruning still applies, and adding clustering further reduces scanned bytes. This matches the requirement of lowering bytes processed without changing results, and clustering can be added to an existing partitioned table.

Why this answer

Clustering a partitioned table on the frequently filtered high-cardinality column lets BigQuery prune storage blocks within each partition. Partition pruning on transaction_date still happens, and cluster pruning further limits scanned data. This directly reduces bytes processed and slot consumption for the described queries without altering results, making it the appropriate storage optimization.

Exam trap

The trap here is assuming that partitioning alone handles every filter predicate, when partitioning only prunes by the partition column and a second high-cardinality filter needs clustering.

Full explanation →

137

MCQmedium

A data engineer is designing a pipeline that reads from Cloud Pub/Sub, aggregates events into 5-minute windows, and writes the results to BigQuery. The engineer wants to ensure that late-arriving data (up to 2 minutes late) is included in the correct window. Which Dataflow feature should they configure?

A.Use a sliding window of 5 minutes with 2-minute slide

B.Set the window duration to 7 minutes to account for lateness

C.Set the allowed lateness to 2 minutes with a trigger that fires on late data

D.Use a global window and watermark

AnswerC

Allowed lateness keeps each 5-minute window's state alive for 2 minutes past the watermark, and a late-data trigger emits updated results when those events arrive. This ensures events up to 2 minutes late land in the correct window rather than being dropped.

Why this answer

Dataflow's allowed lateness feature (set to 2 minutes) ensures that late-arriving data within that threshold is still assigned to the correct 5-minute window. Combined with a trigger that fires on late data, the pipeline can emit updated results for the window after the watermark passes, which is exactly what the engineer needs to handle late-arriving events up to 2 minutes late.

Exam trap

The trap here is that candidates confuse window duration adjustments (Option B) or sliding windows (Option A) with the proper late-data handling mechanism, not realizing that allowed lateness and triggers are the correct Dataflow primitives for including late-arriving data in the correct event-time window.

How to eliminate wrong answers

Option A is wrong because a sliding window of 5 minutes with a 2-minute slide creates overlapping windows that emit results every 2 minutes, not a single 5-minute window with late data handling; it would double-count events and not solve the late-arrival problem. Option B is wrong because setting the window duration to 7 minutes does not account for lateness—it simply shifts the window boundaries, causing data to be assigned to a different time range, which is incorrect for the intended 5-minute aggregation. Option D is wrong because a global window and watermark would aggregate all data into a single unbounded window, losing the per-5-minute grouping required by the pipeline.

Full explanation →

138

MCQeasy

A data engineer needs to transfer 500 TB of archival data from an on-premises NAS to Cloud Storage. The on-premises network has limited bandwidth (100 Mbps). Which transfer method should they recommend?

A.Storage Transfer Service for on-premises

B.gsutil rsync

C.Transfer Appliance

D.Dataflow pipeline reading from NAS

AnswerC

At 100 Mbps, transferring 500 TB over the network would take months, far exceeding any practical window. Transfer Appliance ships physical storage to Google, sidestepping the bandwidth constraint entirely, whereas online methods remain bottlenecked by the same limited link.

Why this answer

The Transfer Appliance is a physical device designed for large-scale data transfers (up to petabytes) when network bandwidth is insufficient. With 500 TB of data and only 100 Mbps bandwidth, the theoretical transfer time would be over 500 days, making any online transfer method impractical. The Transfer Appliance bypasses network constraints entirely by shipping the data physically to Google Cloud.

Exam trap

The exam often tests the misconception that any cloud-native tool (like Storage Transfer Service or gsutil) can handle large data volumes regardless of bandwidth, ignoring the physical reality of network transfer times for archival-scale data.

How to eliminate wrong answers

Option A is wrong because Storage Transfer Service for on-premises requires network connectivity and is designed for smaller, incremental transfers, not for 500 TB over a 100 Mbps link. Option B is wrong because gsutil rsync is a command-line tool that relies on network bandwidth and would take an impractical amount of time (over 500 days) to transfer 500 TB at 100 Mbps. Option D is wrong because a Dataflow pipeline reading from NAS would still need to stream data over the limited 100 Mbps network, resulting in the same bandwidth bottleneck and excessive transfer time.

Full explanation →

139

Multi-Selectmedium

You need to create a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle malformed messages by writing them to a dead-letter table in BigQuery. Which two Apache Beam transforms or patterns should you use to achieve this? (Choose two.)

Select 2 answers

A.Use a ParDo transform with a side output for failed messages.

B.Write the failed messages to a separate BigQuery table using the side output from the ParDo.

C.Write all messages to a BigQuery table and use a view to filter out malformed records.

D.Use a ParDo transform that throws an exception for malformed messages, and configure a dead-letter queue in Dataflow.

E.Use a Filter transform to separate valid and invalid messages.

AnswersA, B

A ParDo transform can process each message and, upon detecting a malformed record, output it to a side output (using MultiOutputReceiver or withOutputTags). The main output continues with valid records. This is a standard pattern for dead-letter handling in Apache Beam, allowing separate processing of errors.

Why this answer

To handle malformed messages, you need to detect them and route them separately. A ParDo transform with a side output allows you to process each message and emit malformed ones to a secondary output. Then, you can write that side output to a BigQuery dead-letter table.

The combination of a ParDo with side output and a separate BigQueryIO.Write for that output achieves the required dead-letter pattern.

Exam trap

The trap here is thinking that a Filter transform alone can route failed messages to a different sink, but it only splits the PCollection and still requires additional transforms to write the failed messages.

Full explanation →

140

MCQhard

A gaming company uses Pub/Sub to ingest player events and Dataflow for real-time analytics. They notice that the Pub/Sub subscription backlog is growing despite the Dataflow pipeline running continuously. The pipeline has a 1-hour window for aggregations. What is the most effective way to reduce the backlog?

A.Increase the Dataflow pipeline's worker count via autoscaling.

B.Use a push subscription instead of pull.

C.Decrease the window duration to 10 minutes.

D.Enable Pub/Sub topic retention.

AnswerA

Scaling worker count lets Dataflow parallelise more Pub/Sub partitions concurrently, raising throughput so the pipeline drains the backlog faster. Since the stem confirms the pipeline runs continuously, the bottleneck is processing capacity rather than stalled jobs, making horizontal autoscaling the direct fix for the growing subscription backlog.

Why this answer

Increasing the Dataflow pipeline's worker count via autoscaling directly addresses the backlog by adding more parallel processing capacity to consume messages from the Pub/Sub subscription faster. Since the pipeline is continuously running but the backlog grows, the bottleneck is processing throughput, not pipeline availability. Autoscaling allows Dataflow to dynamically allocate more workers based on the backlog size, matching consumption rate to the incoming message rate.

Exam trap

Google Cloud often tests the misconception that changing window duration or subscription type can fix a throughput bottleneck, when the real solution is scaling compute resources to match the consumption rate.

How to eliminate wrong answers

Option B is wrong because switching from pull to push subscription does not inherently increase throughput; push subscriptions have their own limitations (e.g., endpoint capacity, HTTP timeouts) and the backlog growth is a processing capacity issue, not a delivery mechanism issue. Option C is wrong because decreasing the window duration to 10 minutes does not reduce the backlog; it changes the aggregation granularity but does not affect the rate at which messages are consumed from the subscription. Option D is wrong because enabling Pub/Sub topic retention controls how long unacknowledged messages are kept, not the rate of consumption; it would only extend the time messages remain available, not reduce the backlog.

Full explanation →

141

MCQhard

You are designing a data pipeline that processes streaming events with late-arriving data (up to 2 hours late). The pipeline must compute hourly aggregations and emit results as soon as possible, but must also accurately update results when late data arrives. You want to minimize overall processing cost. Which Dataflow windowing and trigger configuration should you use?

A.Fixed windows of 1 hour with allowed lateness of 2 hours and trigger every 5 minutes (early) and on watermark (late) with accumulating fired panes

B.Global window with triggers every 5 minutes

C.Sliding windows of 1 hour with 30-minute offset

D.Session windows with 10-minute gap duration

AnswerA

Fixed one-hour windows with two hours of allowed lateness retain late events, while early periodic triggers plus a watermark trigger emit results promptly and accumulating panes revise prior output. This satisfies both low-latency emission and accurate late-data correction without over-provisioning resources.

Why this answer

Fixed windows of 1 hour with allowed lateness of 2 hours and triggers every 5 minutes (early) and on watermark (late) with accumulating fired panes is the correct configuration. This setup computes hourly aggregations, emits early results every 5 minutes, and updates results when late data arrives within the 2-hour allowed lateness. Accumulating panes ensure that late data updates the previous results.

This minimizes cost by using fixed windows and appropriate triggers.

Exam trap

The trap is misunderstanding triggers and allowed lateness; candidates may choose global windows or sliding windows without considering the need for hourly aggregations and late data handling.

How to eliminate wrong answers

Option B is wrong because a global window with triggers every 5 minutes does not provide hourly aggregations; it would aggregate all data into one window, which is not the requirement. Option C is wrong because sliding windows of 1 hour with a 30-minute offset would produce overlapping windows and may not align with hourly aggregations; also, it does not specify allowed lateness or triggers for late data. Option D is wrong because session windows with a 10-minute gap are for grouping events based on activity gaps, not for fixed hourly aggregations, and they do not handle late data as specified.

Full explanation →

142

MCQeasy

A retail company uploads daily sales CSV files to a Cloud Storage bucket. The analytics team wants BigQuery to automatically detect the schema and make new files queryable without running load jobs. The files are added with date-based prefixes, and the team wants to minimize operational overhead. Which BigQuery feature should they use?

A.A Cloud Dataflow streaming pipeline that writes rows into BigQuery.

B.A BigQuery native table loaded with a recurring scheduled query.

C.A BigQuery table partitioned by ingestion time with a load job per file.

D.A BigQuery external table with schema autodetect over the Cloud Storage prefix.

AnswerD

BigQuery external tables can point at a Cloud Storage URI prefix, and with autodetect BigQuery infers column names and types from the files. New files matching the prefix become queryable without load jobs, which matches the low-overhead requirement and the date-prefixed upload pattern.

Why this answer

An external table over the Cloud Storage prefix with schema autodetect lets BigQuery read the CSV files directly. New files that match the prefix appear in query results without any load jobs, which precisely fits the requirement to minimize operational overhead while keeping daily files queryable.

Exam trap

The trap here is reaching for a load job or pipeline when an external table with autodetect already reads new files in place with no ingestion operations.

Full explanation →

143

MCQeasy

Your company is building a real-time anomaly detection system for financial transactions. The system must process streams of transactions and flag anomalies within seconds. The volume is moderate (5000 transactions per second). You want a fully managed solution that integrates with BigQuery for historical analysis. Which service should you use for stream processing?

A.Cloud Dataflow

B.Cloud Pub/Sub with push subscriptions

C.Cloud Dataproc with Spark Streaming

D.Cloud Data Fusion

AnswerA

Cloud Dataflow provides fully managed, autoscaling stream processing with exactly-once semantics and low-latency windowing, satisfying the sub-second anomaly flagging requirement. Its native BigQuery sink connector streams results directly into BigQuery for historical analysis, meeting the integration constraint without custom code or cluster management.

Why this answer

Cloud Dataflow is a fully managed, serverless stream and batch processing service based on Apache Beam. It natively supports real-time streaming with low latency (sub-second to seconds), integrates seamlessly with Pub/Sub for ingestion and BigQuery for output, and auto-scales to handle moderate throughput like 5000 transactions per second. Its managed nature eliminates cluster operations, making it ideal for this use case.

Exam trap

PDE often tests the distinction between messaging (Pub/Sub) and stream processing (Dataflow), and candidates may confuse fully managed services with self-managed ones like Dataproc.

How to eliminate wrong answers

Option B is wrong because Cloud Pub/Sub with push subscriptions is a messaging service, not a stream processing engine; it can deliver messages but cannot perform anomaly detection logic. Option C is wrong because Cloud Dataproc with Spark Streaming requires managing a cluster and is not fully managed, adding operational overhead. Option D is wrong because Cloud Data Fusion is a managed ETL tool for batch and micro-batch data integration, not designed for low-latency stream processing.

Full explanation →

144

MCQeasy

A marketing team needs to run ad-hoc SQL queries on terabytes of clickstream data stored in Parquet files in Cloud Storage. They want a serverless solution with no cluster management and the ability to query external data without loading. Which service should they use?

A.Cloud SQL

B.AlloyDB

C.Dataproc with Spark SQL

D.BigQuery with external tables

AnswerD

BigQuery external tables query Parquet in Cloud Storage directly, so no data loading occurs and no cluster is managed — fully serverless. This satisfies both stated constraints: ad-hoc SQL over terabytes and querying external data in place.

Why this answer

BigQuery with external tables allows querying data stored in Cloud Storage (including Parquet files) without loading it into BigQuery storage, providing a serverless, fully managed solution with no cluster management. This matches the requirement for ad-hoc SQL queries on terabytes of clickstream data in Parquet format, as BigQuery automatically scales compute and storage.

Exam trap

The trap here is that candidates may choose Dataproc with Spark SQL (Option C) because it can query Parquet files, but they overlook the 'serverless' and 'no cluster management' requirement, which BigQuery satisfies natively without any cluster provisioning.

How to eliminate wrong answers

Option A is wrong because Cloud SQL is a fully managed relational database for OLTP workloads, not designed for petabyte-scale analytical queries on external Parquet files, and requires data to be loaded into its storage. Option B is wrong because AlloyDB is a PostgreSQL-compatible database optimized for transactional and hybrid workloads, not a serverless query engine for external data in Cloud Storage, and it requires data to be imported. Option C is wrong because Dataproc with Spark SQL requires cluster management (even if ephemeral) and is not serverless; it also involves provisioning and scaling clusters, contradicting the 'no cluster management' requirement.

Full explanation →

145

Multi-Selecteasy

A company uses Dataproc for transient clusters. Which TWO actions can reduce costs?

Select 2 answers

A.Increase master node size

B.Set cluster autoscaling to minimize idle resources

C.Use standard VMs for all nodes

D.Use persistent clusters to avoid creation overhead

E.Use preemptible VMs for worker nodes

AnswersB, E

Autoscaling reduces resource waste, lowering cost.

Why this answer

Dataproc cluster autoscaling automatically adjusts the number of worker nodes based on the YARN memory and CPU utilization metrics. By scaling down during idle periods, you avoid paying for unused compute capacity, directly reducing costs for transient clusters that have variable workloads.

Exam trap

The trap here is that candidates often think 'persistent clusters' are cheaper because they avoid re-creation overhead, but they overlook the continuous compute cost of idle persistent clusters versus the pay-per-use model of transient clusters.

Full explanation →

146

MCQhard

A financial services firm is designing a Dataflow pipeline that reads from Pub/Sub and writes enriched transactions to BigQuery. The pipeline must guarantee exactly-once processing semantics for the BigQuery sink, even during pipeline updates and worker restarts. The team plans to use the Apache Beam Java SDK with the BigQueryIO connector. Which combination of configurations should they use?

A.Use BigQueryIO.write() with withMethod(STREAMING_INSERTS) and set withFailedInsertRetryPolicy to RETRY_NEVER. This ensures that failed inserts are not retried, preventing duplicates.

B.Use BigQueryIO.write() with withMethod(FILE_LOADS), withTriggeringFrequency set to a fixed interval, and enable withAutoSharding. Rely on the default insertion semantics.

C.Use BigQueryIO.write() with withMethod(STREAMING_INSERTS) and provide a deterministic insertId derived from the transaction ID. Set withFailedInsertRetryPolicy to RETRY_ALWAYS.

D.Use BigQueryIO.write() with withMethod(FILE_LOADS) and set withWriteDisposition(WRITE_TRUNCATE) for each window. This ensures that each window's data replaces the previous contents of the table.

AnswerC

STREAMING_INSERTS with a deterministic insertId enables BigQuery to deduplicate retried inserts, providing exactly-once semantics. RETRY_ALWAYS ensures that transient failures are retried, and because the insertId is stable, duplicates are suppressed. This combination is the recommended way to achieve exactly-once writes to BigQuery from a streaming Dataflow pipeline, even across worker restarts.

Why this answer

Exactly-once semantics for BigQuery writes in Dataflow require deduplication on the BigQuery side. With STREAMING_INSERTS, BigQuery can use the insertId to discard duplicate rows within a short time window. A deterministic insertId derived from the transaction ID ensures that retries after worker restarts or pipeline updates do not create duplicates.

RETRY_ALWAYS preserves data by retrying transient errors, while deduplication prevents duplicates.

Exam trap

The trap here is believing that FILE_LOADS or RETRY_NEVER alone can guarantee exactly-once semantics, when the real mechanism is a deterministic insertId with STREAMING_INSERTS and retries enabled.

Full explanation →

147

Multi-Selectmedium

A media company is designing a data processing system on Google Cloud to analyze video streaming logs. The logs are generated continuously and stored in Cloud Storage. The company wants to use Cloud Dataflow to process these logs, but they need to ensure the pipeline can handle late-arriving data and provide accurate results for both real-time dashboards and historical analysis. Which two features of Cloud Dataflow should they use? (Choose two.)

Select 2 answers

A.Side inputs for enrichment

B.Triggers to emit early results

C.Exactly-once processing

D.Session windows

E.Windowing with allowed lateness

AnswersB, E

Triggers in Dataflow control when to emit aggregated results for a window. Using triggers, such as early triggers or late triggers, allows the pipeline to produce speculative results before the window closes and then update them as late data arrives. This enables real-time dashboards to show near-instantaneous insights while still refining results for historical accuracy. Triggers are a key mechanism for balancing latency and completeness in streaming pipelines.

Why this answer

To handle late-arriving data and provide accurate results for both real-time dashboards and historical analysis, the pipeline should use windowing with allowed lateness and triggers to emit early results. Windowing with allowed lateness ensures that late data is incorporated into the correct window, while triggers allow early results to be emitted for real-time dashboards and updated as more data arrives. Together, they balance latency and completeness.

Exam trap

The trap here is focusing on data integrity features like exactly-once processing instead of the timing features that actually manage late data and result emission.

Full explanation →

148

MCQhard

A Dataflow batch pipeline reads CSV files from Cloud Storage, joins them with a slowly changing dimension stored in BigQuery, and writes the enriched output to BigQuery. The dimension table is large and the join is causing excessive shuffle and worker memory pressure. The team wants to reduce shuffle while keeping the join logic in the pipeline. Which approach should they use?

A.Increase the number of workers and raise the disk size per worker to absorb the shuffle.

B.Use a CoGroupByKey on the two collections and process the grouped results.

C.Write the CSV data to BigQuery first, then run a SQL join in BigQuery and export the result.

D.Load the dimension side into a side input and use it in a ParDo to enrich each record.

AnswerD

A side input broadcasts the dimension data to every worker so the join happens locally without a shuffle of the main dataset. For a large but manageable dimension table, this eliminates the shuffle that causes memory pressure and speeds up enrichment. It keeps the join logic in the pipeline as required.

Why this answer

Using the dimension as a side input lets each worker enrich records locally, avoiding the shuffle and hot-key concentration that CoGroupByKey or a grouped join would introduce. It preserves the in-pipeline join logic and relieves memory pressure. Provisioning more workers or moving the join to BigQuery does not reduce the shuffle as requested.

Exam trap

The trap here is reaching for a grouped join such as CoGroupByKey by default, when a side input avoids the shuffle entirely for a broadcastable dimension.

Full explanation →

149

MCQmedium

Refer to the exhibit. This log entry was generated by Vertex AI Model Monitoring for a production model. What should the data engineer do to address this issue?

A.Increase the drift threshold to 0.9 to suppress alerts

B.Retrain the model with more recent data

C.Deploy a new model version trained on the original dataset

D.Disable monitoring for the 'age' feature

AnswerB

The monitoring log indicates training-serving skew or drift, where live feature distributions diverge from those the model learned. Retraining on recent production data realigns the model with current patterns, restoring prediction accuracy without altering the serving pipeline.

Why this answer

Vertex AI Model Monitoring detected a drift in the 'age' feature, indicating that the production data distribution has shifted from the training data. Retraining the model with more recent data aligns the model with the current data distribution, mitigating the drift and maintaining prediction accuracy. This is the standard remediation for model drift in production ML systems.

Exam trap

Google Cloud often tests the misconception that adjusting thresholds or disabling monitoring is a valid fix for drift, when the correct action is always to retrain the model with current data.

How to eliminate wrong answers

Option A is wrong because increasing the drift threshold to 0.9 would suppress alerts without addressing the underlying data drift, allowing the model to continue making inaccurate predictions. Option C is wrong because deploying a new model version trained on the original dataset would not resolve the drift; it would reuse the same outdated training data that no longer represents the current production distribution. Option D is wrong because disabling monitoring for the 'age' feature would hide the drift issue rather than fixing it, leaving the model vulnerable to degraded performance due to a drifted feature.

Full explanation →

150

MCQeasy

A data engineer needs to run a recurring SQL transformation in BigQuery every night at 02:00 and, if it fails, retry automatically and send a notification. The team wants the least operational overhead and no external orchestrator. What should they use?

A.A Cloud Composer DAG that runs a BigQueryInsertJobOperator on a cron schedule.

B.A Cloud Scheduler job that calls the BigQuery jobs.insert API with a service account.

C.A BigQuery scheduled query configured with a schedule, a destination table, and notification settings.

D.A Dataflow batch pipeline that reads the source table and writes the transformed result.

AnswerC

BigQuery scheduled queries natively support a recurring schedule, a SQL statement, a destination table, and optional Pub/Sub notification on failure. They run entirely inside BigQuery with no cluster or orchestrator to manage, which matches the requirement for minimal operational overhead. This is the purpose-built feature for recurring SQL transformations on a schedule.

Why this answer

BigQuery scheduled queries are the native mechanism for running SQL on a recurring schedule. They accept a schedule, a query, an optional destination table, and configuration for failure notification through Pub/Sub, all without any infrastructure to operate. Composer, Cloud Scheduler with API calls, and Dataflow all can run SQL, but each introduces components to manage, which conflicts with the requirement for the least operational overhead.

Exam trap

The trap here is assuming a general-purpose orchestrator is always the right answer for scheduled work, when BigQuery's built-in scheduled query already covers a single recurring SQL statement.

Full explanation →

Google Professional Data Engineer (PDE) — Questions 76–150