PDE Maintaining and Automating Data Workloads — All Questions With Answers

Question 1mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company uses Dataflow streaming pipelines to process real-time events. They notice increasing system lag over time. Which two Cloud Monitoring metrics should be examined to diagnose the cause?

Question 2hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data team needs to share a BigQuery dataset with another business unit. They want to provide a point-in-time snapshot of the data without incurring additional storage costs for the copy. Which BigQuery feature should they use?

Question 3mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company runs a Dataproc cluster for ETL jobs that process data nightly. They want to reduce costs while maintaining performance. Which strategy is MOST effective?

Question 4mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company uses Cloud Composer for pipeline orchestration. They need to define task dependencies where Task B and Task C can run in parallel after Task A, and Task D must run after both B and C complete. How should they define the DAG?

Question 5mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer notices that BigQuery queries are slower than expected. They want to identify the most expensive stages in the query execution. Which tool or command should they use?

Question 6mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to migrate a schema from BigQuery where a column is currently REQUIRED and needs to become NULLABLE. Which TWO statements are correct? (Choose 2)

Question 7hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

A company runs BigQuery workloads with varying demand. They want to use flat-rate pricing with baseline slots and the ability to burst during peak times. Which TWO actions should they take? (Choose 2)

Question 8mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A company uses Cloud Composer (Airflow) to orchestrate pipelines. They want to implement a pattern where a task polls for a file arrival in Cloud Storage and then triggers subsequent tasks. Which THREE Airflow concepts are essential? (Choose 3)

Question 9hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company runs a streaming Dataflow pipeline that reads from Pub/Sub, enriches data with a side input from BigQuery, and writes to BigQuery. After updating the pipeline code (adding a new field to the output), the engineer notices that the new pipeline version is not picking up the updated code because the job was started from a template. The engineer wants to update the streaming pipeline without draining it. What should the engineer do?

Question 10mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are monitoring a streaming Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. In Cloud Monitoring, you notice that the 'system_lag' metric is increasing over time and now exceeds 10 minutes. The 'data_watermark' metric shows a steady lag. What is the most likely cause of the increasing system lag?

Question 11easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company wants to share a large BigQuery dataset with a partner for analysis. The partner needs read-only access to a specific snapshot of the data as of a certain point in time, and the company wants to avoid additional storage costs for the partner. What is the most cost-effective approach?

Question 12mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your organization has a BigQuery flat-rate reservation with 2000 slots. During peak hours, query performance degrades because concurrent queries exceed the available slots. You want to handle these bursts without changing the base reservation. What should you do?

Question 13hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer is designing a Dataflow pipeline that reads from a Kafka topic (using Pub/Sub for Kafka) and writes to BigQuery. The data schema may change over time, with new fields appearing. The engineer wants to handle schema drift automatically without failing the pipeline. Which approach should the engineer use?

Question 14mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are troubleshooting a Dataproc cluster that runs nightly Spark jobs. The jobs are failing with out-of-memory errors. You want to reduce costs while fixing the issue. Which combination of actions should you take? (Select the BEST answer.)

Question 15mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to set up a Dataplex data quality scan to run weekly on a BigQuery table. The scan should check that: (1) the 'email' column is not null, (2) the 'age' column is between 0 and 120, and (3) the 'country_code' column matches a list of valid ISO codes. Which TWO Dataplex features should the engineer use?

Question 16hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer is building a Cloud Workflows workflow that orchestrates multiple Cloud Functions and API calls. The workflow should handle transient failures with retries and send a notification to a Pub/Sub topic if the workflow ultimately fails. Which THREE steps should the engineer include in the workflow definition?

Question 17hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company uses BigQuery flat-rate pricing with 500 slots purchased as a committed use discount. During peak hours, they need additional capacity but do not want to buy more committed slots. They have a secondary project used for ad-hoc queries by analysts. How can they provide burst capacity to the primary project during peak times without increasing committed spend?

Question 18mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team uses Cloud Composer to run Apache Airflow DAGs. One DAG uses a BigQueryInsertJobOperator to run a query and then uses BigQueryCheckOperator to verify the results. The DAG is failing intermittently because the query result is not ready when the check operator runs. How should you modify the DAG to ensure the check operator runs only after the query completes successfully?

Question 19hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to share a large BigQuery table with a different team, but wants to minimize storage costs. The table is 1 TB in size and is updated daily. The other team only needs read access to the data as of a specific point in time (e.g., end of each day). Which BigQuery feature should be used to provide a read-only copy without duplicating the entire table?

Question 20mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are designing a Dataflow pipeline that reads from Pub/Sub, performs transformations, and writes to BigQuery. The pipeline must handle schema changes in the incoming data (e.g., new fields appearing). The BigQuery schema should evolve automatically to accept new fields without failing. Which approach should you use?

Question 21hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to set up a BigQuery reservation that provides a baseline of 500 slots for daily workloads and can automatically scale up to 1000 slots during peak times. You want to pay only for the slots used beyond the baseline. Which reservation configuration should you choose?

Question 22mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are configuring Dataplex data quality rules for a BigQuery table. Which three types of rules can be defined using Dataplex's SQL-based rule engine? (Choose three.)

Question 23mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to create a reusable Dataflow pipeline for transforming CSV files in Cloud Storage into Avro files in another bucket. The pipeline should be configurable via runtime parameters (e.g., input and output paths). Which approach should you use?

Question 24mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your Dataflow streaming job is experiencing high system lag. You want to identify the root cause. Which Cloud Monitoring metrics should you examine first? (Choose the best option.)

Question 25hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You want to create a cost-efficient snapshot of a large BigQuery table that can be used by other teams for read-only analytics without incurring additional storage costs for the base table data. What should you use?

Question 26hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are designing a data pipeline that ingests streaming data from Pub/Sub, processes it with Dataflow, and writes to BigQuery. You need to ensure that schema changes in the incoming data (new fields) are handled without pipeline failure. Which THREE steps should you take? (Choose THREE.)

Question 27hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your Dataflow streaming pipeline is experiencing increasing system lag over time. You have enabled autoscaling and the pipeline is using the default streaming engine. Which metric should you monitor in Cloud Monitoring to determine if the pipeline is falling behind due to slow processing or due to a bottleneck in the output sink?

Question 28hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are running a Dataproc cluster for batch processing. The job is not latency-sensitive and you want to minimize cost. You notice that the cluster is underutilized during the job. Which configuration change would reduce costs most effectively?

Question 29mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

Your streaming Dataflow pipeline reads from Pub/Sub and writes to BigQuery. You need to update the pipeline to add a new transformation step without losing any messages or causing duplicate processing. Which TWO actions should you take? (Choose 2)

Question 30mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

You need to monitor the health of a Pub/Sub subscription that feeds into a Dataflow pipeline. Which TWO Cloud Monitoring metrics are most relevant to detect if messages are not being acknowledged promptly? (Choose 2)

Question 31hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You have a BigQuery table with a REQUIRED column that you now need to allow NULL values. You also need to add two new nullable columns. Which TWO steps are required to achieve this schema evolution? (Choose 2)

Question 32mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a nightly Cloud Dataflow batch pipeline that reads from Cloud Storage, transforms data, and writes to BigQuery. The pipeline must execute after an upstream Cloud Composer DAG completes. You need to ensure the Dataflow job runs only after the DAG finishes and that the pipeline uses the latest template version stored in Cloud Storage. What should you do?

Question 33mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. You need to deploy a new version of the pipeline with a modified transformation. You want to minimize disruption to the running pipeline and ensure that the new version starts processing new messages immediately. Which deployment approach should you use?

Question 34mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A retail company runs a Cloud Composer (Airflow) DAG that ingests files from Cloud Storage into BigQuery every night. The DAG has failed three nights in a row with a task-level error on the load task, but the engineer only learns about it after business users complain. The team wants to be automatically notified by email whenever any task in any DAG in the environment enters a failed state, with minimal ongoing effort. What should the engineer do?

Question 35mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are using Cloud Dataproc to run a Spark job that processes data stored in Cloud Storage. You want to enable autoscaling for the cluster to handle variable workloads. Which two configurations are required to enable autoscaling? (Choose two.)

Question 36hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. You need to update the pipeline with a new transformation that changes the output schema of the BigQuery table. The pipeline is currently running in production and must not lose data during the update. Which update strategy should you use?

Question 37mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A retail company runs a Dataflow streaming pipeline that reads events from Pub/Sub and writes enriched records into BigQuery. Over a weekend the on-call engineer receives an alert that the pipeline's system lag keeps climbing and never drains, even though the upstream publisher volume is unchanged. The pipeline uses a fixed 10-second window with a 5-minute allowed lateness and writes to BigQuery with no load job frequency override. What is the MOST likely cause of the sustained system lag?

Question 38mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to automate a BigQuery ELT workflow so that a scheduled SQL transformation runs only when new data lands in a Cloud Storage bucket, and so that failures notify the on-call team. The team already uses Cloud Composer. Which two approaches satisfy these requirements? (Choose two.)

Question 39mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Cloud Composer 2 environment (Airflow 2.6) to orchestrate BigQuery ELT. A nightly DAG ingests data from Cloud Storage, runs a BigQuery load job, then triggers a Dataflow pipeline. The DAG recently started failing intermittently at the BigQuery load task with 'Quota exceeded: Your project exceeded quota for imports per project'. You need to reduce the frequency of these failures without changing the DAG's schedule. What should you do?

Question 40hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are designing a data pipeline that uses Cloud Dataflow to process streaming data from Pub/Sub and write to BigQuery. You need to ensure that the pipeline handles late-arriving data correctly and provides exactly-once processing semantics. Which two actions should you take? (Choose two.)

Question 41hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You operate a Cloud Composer 2 environment whose DAGs trigger BigQuery jobs. A new compliance rule requires that every DAG run's BigQuery queries execute under a dedicated identity with its own audit trail, separate from the Composer environment's service account. Which configuration should you apply?

Question 42hardmultiple choice

Review the full routing breakdown →

A logistics company has a Cloud Composer DAG with a task that calls a third-party routing API. The API occasionally returns HTTP 503 during peak hours, and the DAG then fails the whole run, forcing a manual restart. The engineer wants the task to automatically retry transient 503 responses with increasing delays, but not to retry on a 401 authentication error, which indicates a misconfigured credential that must be fixed by a human. What should the engineer do?

Question 43hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You have a Dataflow batch pipeline that processes a large dataset stored in Cloud Storage and writes results to BigQuery. The pipeline uses a custom DoFn that makes API calls to an external service. You notice that the pipeline is running slowly and some workers are idle while others are overloaded. You want to improve performance and ensure even distribution of work. What should you do?

Question 44mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your company uses Cloud Composer to orchestrate a data pipeline that loads data into BigQuery. A new compliance requirement mandates that all data access be logged and that any changes to the DAGs be auditable. You need to ensure that DAG file changes are tracked and that you can identify who modified a DAG and when. What should you do?

Question 45hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are designing automated monitoring for a Cloud Composer 2 environment that runs critical ingestion DAGs. You need to detect conditions that indicate the scheduler or workers are unhealthy before DAG runs silently stop. Which two Cloud Monitoring metrics should you alert on? (Choose two.)

Question 46hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are using Cloud Composer 2 to orchestrate a data pipeline that includes tasks that interact with BigQuery and Cloud Storage. You notice that some tasks are failing due to insufficient permissions, and you want to ensure that the Composer environment has the necessary IAM roles to execute these tasks. Which two IAM roles should you grant to the Composer service account to allow it to manage BigQuery jobs and access Cloud Storage objects? (Choose two.)

Question 47mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer uses Cloud Composer to orchestrate a daily batch pipeline. A downstream task should only start after an upstream BigQuery load job finishes successfully and a specific file appears in Cloud Storage. Which combination of operators should the engineer use in the Airflow DAG?

Question 48easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

An engineer needs to create a reusable Dataflow pipeline that can be executed with different parameters without modifying code. Which Dataflow feature should they use?

Question 49hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to alert when Pub/Sub subscription has messages older than 1 hour. Which Cloud Monitoring metric and filter should they use?

Question 50mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A team wants to enforce data quality rules on BigQuery tables using Dataplex. They need to run column-level checks for null values and row-level checks for value ranges on a schedule. Which Dataplex feature should they use?

Question 51easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

An organization uses BigQuery on-demand pricing. To control costs, they want to estimate the bytes processed by a query before running it. Which command or method should they use?

Question 52easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A streaming Dataflow pipeline needs to be updated without draining the existing pipeline. Which update strategy should be used?

Question 53hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company wants to use Cloud DLP to inspect data in BigQuery for sensitive information and de-identify it by masking credit card numbers. They want to perform this on a schedule. Which approach should they take?

Question 54mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer is building a batch pipeline that runs daily using Cloud Composer. The pipeline has three tasks: extract data from Cloud Storage, transform data using Dataflow, and load the transformed data into BigQuery. The engineer wants to ensure that the Dataflow job only starts after the extraction task completes successfully, and the load task only starts after the Dataflow job finishes. How should the engineer define the task dependencies in the Airflow DAG?

Question 55easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to schedule a simple workflow that fetches data from an API every hour, transforms it using Cloud Functions, and writes the result to Cloud Storage. The workflow has no complex branching or retry logic beyond basic retries. Which orchestration service is the MOST cost-effective and simplest to implement?

Question 56easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to inspect a BigQuery table for sensitive data such as credit card numbers and email addresses before sharing it with a third party. The engineer also wants to de-identify the data by masking the sensitive columns. Which Google Cloud service should be used?

Question 57mediummultiple choice

Study the full Python automation breakdown →

A company runs a critical batch pipeline using Cloud Dataflow. The pipeline processes financial transactions and runs every hour. Recently, some runs have failed due to transient errors (e.g., network timeouts). The engineer wants to automatically retry failed runs without manual intervention. The pipeline is launched from a Cloud Composer DAG using DataflowPythonOperator. What is the BEST way to handle retries?

Question 58hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are optimizing a BigQuery query that scans 1 TB of data every day. The query joins a large fact table (partitioned by date) with a small dimension table. You notice that the query always scans the entire fact table, even though you only need the last 7 days of data. Which optimization will MOST reduce the bytes scanned?

Question 59mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company uses Cloud Pub/Sub for a real-time data pipeline. The subscription has a backlog of millions of messages that are not being processed quickly enough. In Cloud Monitoring, you observe that the 'subscription/num_undelivered_messages' metric is high and growing, while 'subscription/oldest_unacked_message_age' is also increasing. Which action is MOST likely to reduce the backlog?

Question 60mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A company is migrating their on-premises data warehouse to BigQuery. They have a mix of batch and streaming ingestion. The data team wants to optimize query costs. Which THREE practices should they adopt?

Question 61easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to orchestrate a complex data pipeline that involves multiple steps including data extraction from Cloud Storage, transformation using Dataflow, and loading into BigQuery. The pipeline has dependencies between tasks and requires monitoring and retries. Which Google Cloud service should be used for orchestration?

Question 62mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are running a streaming pipeline with Dataflow that reads from Pub/Sub and writes to BigQuery. You notice that the system lag metric is increasing over time, indicating that messages are taking longer to process. What is the most likely cause and how should you address it?

Question 63easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Which Dataflow feature allows you to package a pipeline into a reusable template that can be deployed with different parameters at runtime?

Question 64hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A company runs a Dataflow streaming pipeline that processes financial transactions. They need to apply a new transformation that enriches the data with a lookup from Cloud Bigtable without stopping the pipeline. The pipeline must be updated in a way that minimises data loss and preserves exactly-once semantics. What is the recommended approach?

Question 65mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are monitoring a Dataproc cluster and notice that the cluster utilisation is high, but jobs are running slowly. The cluster uses preemptible workers for cost savings. What is the most likely cause of the performance degradation?

Question 66easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Which BigQuery feature allows you to estimate the cost of a query before running it, by returning the number of bytes that would be processed?

Question 67mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your company stores sensitive customer data in Cloud Storage. You need to inspect the data for personally identifiable information (PII) and de-identify it before sharing with a third party. Which Google Cloud service should you use?

Question 68mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to monitor a Pub/Sub-based streaming pipeline. Which two Cloud Monitoring metrics should be used to detect a backlog of unprocessed messages? (Choose two.)

Question 69hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

A company runs a Dataflow pipeline that processes a high-volume data stream. They notice that the pipeline's worker CPU utilisation is near 100% and the system lag is increasing. Which three actions can improve performance? (Choose three.)

Question 70mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are building a data pipeline that runs daily batch jobs on Dataproc, then loads results into BigQuery. You want to orchestrate the entire workflow, including dependencies between steps, retries, and monitoring. Which Google Cloud service is most appropriate?

Question 71mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your streaming Dataflow pipeline reads from Pub/Sub, enriches data with a side input, and writes to BigQuery. You need to update the enrichment logic without draining the pipeline, to minimize data loss and maintain exactly-once semantics. What should you do?

Question 72hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a BigQuery reservation with 500 baseline slots and autoscaling up to 2000 slots. Your team runs a mix of interactive queries and batch load jobs. During peak hours, you notice that interactive queries are throttled when autoscaling slots are consumed by long-running batch loads. How can you ensure interactive queries get priority access to slots?

Question 73easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You want to monitor the latency of messages in a Pub/Sub subscription. Which Cloud Monitoring metric should you use to see the age of the oldest unacknowledged message?

Question 74mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team uses Cloud Dataproc for Spark ML training jobs. You want to reduce costs for non-critical, fault-tolerant training jobs. Which Dataproc feature should you use for worker nodes?

Question 75hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A BigQuery table has a REQUIRED column 'user_id' that now needs to accept NULL values due to upstream data changes. You want to alter the schema with minimal downtime and no data loss. What should you do?

Question 76easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to estimate the cost of a BigQuery query before running it. Which command or feature should you use?

Question 77mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are designing a data quality pipeline that must inspect PII in BigQuery tables and de-identify sensitive columns before sharing with analysts. Which GCP service should you use?

Question 78easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to schedule a Dataproc Spark job to run at 2 AM every day, and upon completion, trigger a BigQuery load job. Which Cloud Composer operator should you use to run the Spark job?

Question 79mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

Your company uses Cloud Composer to orchestrate a data pipeline that includes Dataproc Spark jobs and BigQuery load operations. You need to pass the output file path from the Spark job to the next BigQuery task in the DAG. Which two mechanisms can you use to share data between tasks? (Choose TWO.)

Question 80easymulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are using Cloud Workflows to orchestrate a series of API calls. You need to handle errors and retries. Which THREE features of Cloud Workflows can you use? (Choose THREE.)

Question 81mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are designing a Cloud Composer workflow that loads data from Cloud Storage into BigQuery, runs a Dataflow job to transform the data, and then triggers a Dataproc Spark job. After each step, you need to conditionally branch based on success or failure. Which Airflow feature allows you to pass messages between tasks to enable dynamic branching?

Question 82mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You have a BigQuery table that is used by multiple teams. To save costs, you want to provide a consistent view of the data as of a specific point in time without creating full copies. Which BigQuery feature should you use?

Question 83easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to orchestrate a simple, linear workflow that calls several Cloud Functions and API endpoints sequentially with conditional logic. The workflow should be defined as code and have minimal overhead. Which GCP service should you use?

Question 84mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your organization has a BigQuery flat-rate reservation with 500 slots. During peak hours, queries are queued and you need additional capacity temporarily. You want to add slots for a burst of activity without committing to a long-term purchase. What should you do?

Question 85easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer wants to quickly estimate the cost of running a BigQuery query before executing it. Which command-line tool or command should they use?

Question 86mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to inspect a BigQuery table for sensitive data such as credit card numbers and apply masking. Which GCP service should you use to identify and de-identify the data?

Question 87hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are setting up Dataplex data quality rules for a BigQuery table. You want to define rules that check for non-null values in key columns and also validate that a column's values fall within a certain range. Which TWO rule types must you use? (Choose 2)

Question 88mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

You want to optimize BigQuery costs for a large dataset that is frequently queried by time range. You also need to ensure that predictable workloads have dedicated slot capacity. Which TWO strategies should you combine? (Choose 2)

Question 89easymulti select

Read the full Maintaining and Automating Data Workloads explanation →

You need to deploy a reusable Dataflow pipeline that can be executed with different parameters from Cloud Composer. Which TWO components should you use? (Choose 2)

Question 90mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

You are building a data pipeline that ingests data from on-premises into Cloud Storage, then processes it with Dataproc, and finally loads into BigQuery. You need to schedule the pipeline to run daily. The pipeline must handle occasional failures gracefully. Which THREE Google Cloud services should you use together to achieve this? (Choose 3)

Question 91easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer schedules a Cloud Composer 2 environment to run a DAG that triggers a Dataflow batch job every night. The DAG sometimes fails because the Dataflow job takes longer than the default task timeout. The engineer wants the DAG to wait for the Dataflow job to finish rather than timing out. Which change should the engineer make?

Question 92hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a Cloud Composer environment that runs a critical DAG every hour. The DAG includes a task that calls a Cloud Function to process data. Recently, the Cloud Function started taking longer than expected, causing the DAG to exceed its SLA. You need to detect this delay and automatically retry the task if it fails due to timeout, while minimizing changes to the DAG. What should you do?

Question 93hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your organization runs a Cloud Composer 2 environment that executes dozens of DAGs. Several DAGs share a connection to an external REST API that enforces a rate limit of 100 requests per minute. During peak hours, DAGs fail with HTTP 429 errors. You want to prevent these failures without changing the external API's limits. What should you do?

Question 94mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineering team manages BigQuery datasets across multiple projects. They want to automatically detect and respond when a scheduled query fails, and they want the response to create an incident in their existing ticketing system. The team prefers minimal custom infrastructure and wants to use native Google Cloud tooling. Which approach should they use?

Question 95mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Cloud Composer 2 environment (composer-2.1.0-airflow-2.6.3) that executes a daily BigQuery ETL workflow. The workflow must not run on weekends. You want to implement this with minimal code and without modifying the DAG's task logic. What should you do?

Question 96hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are designing the deployment process for a Dataflow streaming pipeline that processes financial transactions. The pipeline must be updated without losing in-flight state, such as open windows and timers, and without downtime. Your team uses the Apache Beam Java SDK and deploys from a CI/CD pipeline. Which update strategy should you use?

Question 97hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a Cloud Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline uses the BigQueryIO write transform with STREAMING_INSERTS. You need to ensure exactly-once processing semantics for the BigQuery writes. What should you do?

Question 98mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your organization runs a Cloud Composer (Apache Airflow) environment that executes a DAG every night to move data from Cloud Storage into BigQuery. The DAG has been succeeding for months, but last week the nightly load silently produced a BigQuery table with zero rows while the Airflow task still reported success. You need to add a safeguard that fails the DAG task whenever the loaded row count is zero before downstream tasks run. What should you do?

Question 99mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Cloud Composer (Airflow) environment that executes a nightly BigQuery ELT DAG. A downstream task must only run after an upstream task that loads a partitioned table completes, and you want the downstream task to wait for the load's completion signal without polling BigQuery repeatedly. Which Airflow mechanism should you implement to coordinate these tasks within the DAG?

Question 100easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to schedule a recurring BigQuery query that aggregates data from a partitioned table and writes the results to a new table every day at 03:00 UTC. You want a fully managed solution with minimal operational overhead. What should you use?

Question 101easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineering team maintains several Cloud Composer DAGs that load files from Cloud Storage into BigQuery. They want the DAG to start only after a new file has landed in the source bucket, rather than polling on a fixed schedule and often finding nothing. Which Airflow construct should they use to trigger the DAG based on the arrival of the object?

Question 102mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Cloud Composer 2 environment in project `analytics-prod`. A nightly DAG loads Cloud Storage files into BigQuery, but a recent Cloud Storage outage caused several tasks to fail after exhausting their retries. You want failed task instances to automatically re-run without manual intervention once the upstream dependency recovers. What should you do?

Question 103hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer manages a Cloud Composer 2 environment. A DAG that downloads a large reference dataset each night occasionally exceeds the default task timeout because the source API is slow. The engineer wants the task to fail fast and be retried automatically rather than hanging for hours, and wants failed runs to be visible for alerting. Which configuration should be applied to the task?

Question 104hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A financial services company runs a Dataflow streaming pipeline that reads from Pub/Sub and writes enriched records to BigQuery. Compliance requires that the raw Pub/Sub messages be retained for seven years so that any record can be reprocessed if the enrichment logic is later found to be incorrect. The pipeline currently has no archival step. What should the data engineer do to satisfy the retention requirement with the least operational overhead?

Question 105mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must be updated to add a new transformation that enriches each message with data from Cloud SQL. You need to minimize downtime and ensure the pipeline continues processing without data loss. What should you do?

Question 106hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your company uses Cloud Composer to orchestrate a complex data pipeline. You need to ensure that the pipeline can recover from failures and that tasks are retried automatically with exponential backoff. You also want to be alerted if a task fails after all retries. Which combination of features should you implement?

Question 107easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer must give a Dataproc Serverless for Spark batch workload permission to read objects from a specific Cloud Storage bucket and write to a BigQuery dataset, following least privilege. The workload runs as a custom service account. Which approach should be used?

Question 108mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A retail analytics team runs a Cloud Composer DAG that starts a Dataproc job, waits for it, and then runs a BigQuery load. The Dataproc job sometimes fails due to a transient YARN resource shortage, and the team wants the DAG to automatically retry the Dataproc submission a few times before alerting. Which Airflow configuration on the Dataproc task best meets this need?

Question 109easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer has a BigQuery SQL script that must run every day at 06:00, load its results into a reporting table, and retry automatically if the query fails due to transient errors. The team has no existing orchestration tooling, wants the lowest operational overhead, and needs the schedule and the SQL to be managed entirely inside Google Cloud. Which approach should the engineer use?

Question 110hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

A media company runs a Cloud Composer environment whose DAGs trigger Dataflow batch jobs and BigQuery loads. The operations team reports that Composer costs are rising and that DAGs occasionally stall because workers are saturated. You review the environment and find that several tasks are long-running sensors that hold worker slots while waiting on external conditions. Which TWO changes should you make to reduce worker saturation and cost? (Choose two.)

Question 111mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You are using Cloud Composer to orchestrate a data pipeline that runs a Dataproc job to process data, followed by a BigQuery load. You notice that the Dataproc job sometimes takes longer than expected, causing the BigQuery load to start before the Dataproc job finishes, resulting in incomplete data. Which Airflow feature should you use to ensure the BigQuery load only runs after the Dataproc job completes successfully?

Question 112easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer must ensure that a Cloud Composer DAG which loads a BigQuery table runs every day at 02:00 UTC and that a dependent downstream report DAG runs only after the load succeeds. The report DAG lives in the same Composer environment but is a separate DAG file. Which feature should be used to coordinate the two DAGs?

Question 113mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team runs a Cloud Composer 2 environment to orchestrate BigQuery ELT jobs. A DAG that loads a critical fact table must only run after an upstream ingestion DAG completes, and you want Composer itself to trigger it automatically without any external scheduler. Which mechanism should you use?

Question 114easymultiple choice

Study the full Python automation breakdown →

A data engineer needs to run a recurring nightly extract-transform-load job that pulls data from a REST API, applies Python transformations, and writes the output to a Cloud Storage bucket. The team wants a fully managed, serverless scheduler that can retry failed runs and send notifications, and they do not want to maintain any cluster or VM. Which Google Cloud service should they use to define and run this job?

Question 115mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A financial services firm stores customer transaction data in BigQuery. Compliance requires that a nightly Cloud Composer DAG verify that the previous day's partition is complete before downstream reporting DAGs run, and that the reporting DAG never start if the verification fails. The engineer wants the dependency expressed inside orchestration rather than by polling from the reporting DAG. What should the engineer do?

Question 116easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your company uses Cloud Composer to run a daily ETL workflow. The workflow consists of several tasks that must run in a specific order. You want to receive an alert if any task fails. Which Cloud Monitoring feature should you use?

Question 117mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your team uses Cloud Composer to orchestrate a daily ETL workflow that extracts data from an on-premises database, transforms it in Dataproc, and loads it into BigQuery. The workflow occasionally fails due to transient network issues. You want to automatically retry failed tasks without manual intervention. What should you do?

Question 118hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You manage a Cloud Composer 2 environment that runs a DAG with a task using the BigQueryInsertJobOperator. The task occasionally fails with 'rateLimitExceeded' when submitting many jobs in parallel. You want to limit the number of concurrent BigQuery jobs submitted by this DAG without affecting other DAGs in the same environment. What should you do?

Question 119hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer is migrating a Composer 1 environment to Cloud Composer 2 and notices that a DAG relying on a legacy operator for a deprecated service no longer works. The engineer wants a durable fix that keeps the pipeline running and avoids repeating this problem in future upgrades. Which action should be taken?

Question 120hardmulti select

Read the full Maintaining and Automating Data Workloads explanation →

You manage several Cloud Composer 2 environments that run production DAGs. You must define an alerting strategy that detects when a DAG run fails and when a task is stuck retrying for an unusually long time, using Cloud Monitoring. (Choose two.)

Question 121mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

Your Dataflow streaming pipeline writes to BigQuery and occasionally fails with `QuotaExceededError` on streaming inserts during peak hours. You want to reduce insert-driven quota pressure without changing the pipeline's output schema or losing exactly-once semantics. Which change should you make?

Question 122hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You have a Dataflow batch pipeline that processes data from Cloud Storage and writes to BigQuery. The pipeline uses a custom DoFn that sometimes throws exceptions due to malformed input records. You want to ensure that the pipeline continues processing valid records while logging the malformed ones for later analysis, without failing the entire job. Which Dataflow feature should you use?

Question 123hardmultiple choice

Study the full Python automation breakdown →

A financial services firm runs an Apache Airflow workload on Cloud Composer 2 that ingests market data, runs dbt transformations, and loads curated tables into BigQuery. The DAG currently uses a single PythonOperator that runs a long shell command, and the team wants to make failures easier to diagnose and retries more granular. They also want to avoid rerunning already-successful upstream steps. Which change BEST meets these goals?

Question 124easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data engineer needs to run a recurring SQL transformation in BigQuery every night at 02:00 and, if it fails, retry automatically and send a notification. The team wants the least operational overhead and no external orchestrator. What should they use?

Question 125mediummulti select

Read the full Maintaining and Automating Data Workloads explanation →

A team runs a Dataflow batch pipeline that writes to BigQuery. They want to reduce cost and improve reliability of the write step. The pipeline currently uses BigQueryIO with the default write method. Which two changes should they make to achieve these goals? (Choose two.)

Question 126hardmultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A Dataflow streaming pipeline writes to BigQuery and has run in production for months. The team wants to add a transformation and deploy the change with zero data loss and no interruption to the running pipeline. Which deployment approach should they use?

Question 127mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data platform team uses Cloud Composer 2 to orchestrate a DAG that runs a Dataproc Serverless batch job producing a partitioned BigQuery table. The DAG passes the output location to a downstream task that runs a dbt model. The team wants failures in the dbt task to automatically trigger a retry of only that task, and they want the DAG to expose the Dataproc job ID in the Airflow UI for troubleshooting. Which approach BEST satisfies both requirements?

Question 128mediummultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

A data platform team wants to grant a service account the ability to run BigQuery jobs and read data in a specific dataset, while ensuring it cannot create or delete datasets. Which IAM approach satisfies this with least privilege?

Question 129easymultiple choice

Read the full Maintaining and Automating Data Workloads explanation →

You need to schedule a BigQuery query to run every day at 6:00 AM and write the results to a new table. The query is simple and does not require complex dependencies. You want a low-maintenance, serverless solution with minimal configuration. What should you do?