PDE Maintaining and Automating Data Workloads Practice Test 5 — 15 Questions

Question 1

You are designing a Cloud Composer workflow that loads data from Cloud Storage into BigQuery, runs a Dataflow job to transform the data, and then triggers a Dataproc Spark job. After each step, you need to conditionally branch based on success or failure. Which Airflow feature allows you to pass messages between tasks to enable dynamic branching?

Accepted Answer

XComs. XComs (cross-communications) in Airflow allow tasks to exchange small amounts of data, such as status or metadata. This data can be used by BranchPythonOperator to conditionally choose downstream tasks.

Answer

Sensors

Answer

TaskFlow API

Answer

DAG dependencies

Question 2

Your Dataflow streaming pipeline is experiencing increasing system lag over time. You have enabled autoscaling and the pipeline is using the default streaming engine. Which metric should you monitor in Cloud Monitoring to determine if the pipeline is falling behind due to slow processing or due to a bottleneck in the output sink?

Accepted Answer

System lag. In Dataflow streaming pipelines, 'System lag' represents the maximum time that an item has been waiting to be processed. If system lag is high, it indicates that the pipeline cannot keep up with the input rate. However, to pinpoint if the bottleneck is the sink, you should also monitor 'Data freshness' (time since last output) or write to a staging area temporarily. But the question specifically asks for a metric that helps determine if the pipeline is falling behind due to slow processing or a sink bottleneck. The correct answer is 'System lag' as it directly reflects processing delay; if it increases, the pipeline is not processing fast enough regardless of sink.

Answer

Worker CPU utilization

Answer

Element count

Answer

Data freshness

Question 3

You have a BigQuery table that is used by multiple teams. To save costs, you want to provide a consistent view of the data as of a specific point in time without creating full copies. Which BigQuery feature should you use?

Accepted Answer

Table snapshot. BigQuery table snapshots provide a point-in-time copy of a table that incurs only storage costs for the snapshot (no additional slot usage). They are read-only and can be used to share data without duplicating the base table.

Answer

Authorized views

Answer

Materialized views

Answer

Table clone

Question 4

You need to orchestrate a simple, linear workflow that calls several Cloud Functions and API endpoints sequentially with conditional logic. The workflow should be defined as code and have minimal overhead. Which GCP service should you use?

Accepted Answer

Workflows. Workflows is a serverless orchestration service that uses YAML/JSON to define workflows. It is ideal for simpler, linear or conditional orchestrations without the need for full Airflow infrastructure.

Answer

Cloud Tasks

Answer

Dataflow

Answer

Cloud Composer

Question 5

Your organization has a BigQuery flat-rate reservation with 500 slots. During peak hours, queries are queued and you need additional capacity temporarily. You want to add slots for a burst of activity without committing to a long-term purchase. What should you do?

Accepted Answer

Use flex slots. Flex slots are short-term, hourly commitments that can be added to an existing flat-rate reservation to handle bursts. They are ideal for temporary capacity needs.

Answer

Switch to on-demand pricing

Answer

Create a secondary reservation with autoscaling

Answer

Purchase additional committed use reservations

Question 6

You are running a Dataproc cluster for batch processing. The job is not latency-sensitive and you want to minimize cost. You notice that the cluster is underutilized during the job. Which configuration change would reduce costs most effectively?

Accepted Answer

Use preemptible workers for the worker nodes. Using preemptible workers in Dataproc reduces cost by about 60-80% compared to standard VMs. They are suitable for fault-tolerant batch jobs because they can be terminated at any time.

Answer

Resize the cluster to use larger machines

Answer

Switch to single-node cluster

Answer

Enable autoscaling

Question 7

A data engineer wants to quickly estimate the cost of running a BigQuery query before executing it. Which command-line tool or command should they use?

Accepted Answer

bq query --dry_run. The `bq query --dry_run` command parses the query and reports the number of bytes processed without executing it, allowing cost estimation based on on-demand pricing.

Answer

gcloud logging read

Answer

bq query --use_cache=false

Answer

gcloud bigtable queries run

Question 8

You need to inspect a BigQuery table for sensitive data such as credit card numbers and apply masking. Which GCP service should you use to identify and de-identify the data?

Accepted Answer

Cloud Data Loss Prevention (DLP). Cloud DLP can automatically inspect BigQuery tables for sensitive data types (credit card numbers, etc.) and apply de-identification transforms like masking, tokenization, etc.

Answer

IAM Recommender

Answer

Cloud KMS

Answer

Dataplex

Question 9

Your streaming Dataflow pipeline reads from Pub/Sub and writes to BigQuery. You need to update the pipeline to add a new transformation step without losing any messages or causing duplicate processing. Which TWO actions should you take? (Choose 2)

Accepted Answer

Use the Dataflow update command with the same pipeline name and the new template.. To update a streaming pipeline without draining, you can use the `--update` flag with the new pipeline. The snapshot feature allows you to restore state if needed. Draining would stop the pipeline; canceling would lose messages.

Answer

Stop the pipeline, modify it, then restart with a new name.

Answer

Drain the pipeline before making changes.

Answer

Cancel the pipeline and create a new one.

Question 10

You are setting up Dataplex data quality rules for a BigQuery table. You want to define rules that check for non-null values in key columns and also validate that a column's values fall within a certain range. Which TWO rule types must you use? (Choose 2)

Accepted Answer

Row rule (e.g., not null). Dataplex data quality rules include row rules (for null checks) and column rules (for range or value checks). Table rules apply to the entire table; custom SQL can be used but row and column rules are the standard.

Answer

Table rule (e.g., row count)

Answer

Partition rule

Answer

Custom SQL rule

Question 11

You need to monitor the health of a Pub/Sub subscription that feeds into a Dataflow pipeline. Which TWO Cloud Monitoring metrics are most relevant to detect if messages are not being acknowledged promptly? (Choose 2)

Accepted Answer

subscription/oldest_unacked_message_age. The metric `subscription/num_undelivered_messages` indicates the number of messages not yet delivered/acknowledged. `subscription/oldest_unacked_message_age` shows the age of the oldest unacknowledged message, indicating backlogs.

Answer

subscription/num_outstanding_messages

Answer

topic/send_request_count

Answer

subscription/ack_message_count

Question 12

You want to optimize BigQuery costs for a large dataset that is frequently queried by time range. You also need to ensure that predictable workloads have dedicated slot capacity. Which TWO strategies should you combine? (Choose 2)

Accepted Answer

Partition the table by date. Partitioned tables reduce bytes scanned by time-range queries. Committed use reservations provide predictable slot capacity at a discount. Autoscaling does not provide dedicated capacity; caching is automatic.

Answer

Use query caching

Answer

Create a materialized view for the entire table

Answer

Enable autoscaling slots

Question 13

You have a BigQuery table with a REQUIRED column that you now need to allow NULL values. You also need to add two new nullable columns. Which THREE steps are required to achieve this schema evolution? (Choose 3)

Accepted Answer

Use the ALTER TABLE SET OPTIONS statement to change the column mode to NULLABLE.. BigQuery requires exporting data, modifying the schema, and importing data or using SQL statements. The schema can be updated via `bq update` or ALTER TABLE. Simply adding columns is possible, but changing from REQUIRED to NULLABLE requires recreation or ALTER TABLE CHANGE COLUMN.

Answer

Add the new columns using ALTER TABLE ADD COLUMN.

Answer

Update the IAM permissions on the table.

Question 14

You need to deploy a reusable Dataflow pipeline that can be executed with different parameters from Cloud Composer. Which TWO components should you use? (Choose 2)

Accepted Answer

Dataflow Flex Template. Dataflow Flex Templates allow you to create custom templates that can accept runtime parameters. Cloud Composer can trigger these templates using the DataflowStartFlexTemplateOperator. Direct runner options are not needed.

Answer

Direct runner

Answer

Dataflow Classic Template

Answer

Cloud Scheduler

Question 15

You are building a data pipeline that ingests data from on-premises into Cloud Storage, then processes it with Dataproc, and finally loads into BigQuery. You need to schedule the pipeline to run daily. The pipeline must handle occasional failures gracefully. Which THREE Google Cloud services should you use together to achieve this? (Choose 3)

Accepted Answer

Cloud Storage. Cloud Composer orchestrates the whole pipeline. Cloud Storage is the staging area. Dataproc processes data. BigQuery is the destination. Dataflow is not needed. Pub/Sub is for messaging, not scheduling.

Answer

Dataflow

Answer

Pub/Sub