PDE Maintaining and Automating Data Workloads Practice Test 1 — 15 Questions

Question 1

Your team uses Cloud Dataproc for Spark ML training jobs. You want to reduce costs for non-critical, fault-tolerant training jobs. Which Dataproc feature should you use for worker nodes?

Accepted Answer

Use preemptible instances for worker nodes.. Preemptible instances are short-lived, lower-cost VMs that Cloud Dataproc can use for worker nodes. Because the training jobs are non-critical and fault-tolerant (e.g., they can handle node failures via Spark's built-in resilience), preemptible instances significantly reduce costs while still completing the workload. This directly addresses the requirement to reduce costs for fault-tolerant jobs.

Answer

Use custom machine types with more memory.

Answer

Use SSDs instead of HDDs for persistent disks.

Answer

Use committed use discounts for 1-year or 3-year terms.

Question 2

You have a BigQuery table with a REQUIRED column that you now need to allow NULL values. You also need to add two new nullable columns. Which TWO steps are required to achieve this schema evolution? (Choose 2)

Accepted Answer

Add the new columns using ALTER TABLE ADD COLUMN.. To achieve the schema evolution, you need to add the new nullable columns using ALTER TABLE ADD COLUMN (A) and change the existing REQUIRED column to NULLABLE using ALTER TABLE ALTER COLUMN DROP NOT NULL (the SET OPTIONS statement in option B approximates this). Exporting and recreating the table (D and E) are not required because BigQuery supports direct schema changes without data migration.

Answer

Update the IAM permissions on the table.

Answer

Use CREATE OR REPLACE TABLE with the new schema and import data.

Answer

Export the table to Cloud Storage in Avro format.

Question 3

A data engineer uses Cloud Composer to orchestrate a daily batch pipeline. A downstream task should only start after an upstream BigQuery load job finishes successfully and a specific file appears in Cloud Storage. Which combination of operators should the engineer use in the Airflow DAG?

Accepted Answer

BigQueryInsertJobOperator and GCSObjectExistenceSensor with upstream dependency. The BigQueryInsertJobOperator (or BigQueryOperator) handles the load job, and the GoogleCloudStorageObjectExistenceSensor (or GCSObjectExistenceSensor) waits for the file. Task dependencies link them.

Answer

BigQueryInsertJobOperator with wait_for_downstream=True

Answer

DataflowPythonOperator and GCSObjectExistenceSensor

Answer

BigQueryOperator and FileSensor with downstream dependency

Question 4

A company uses Dataflow streaming pipelines to process real-time events. They notice increasing system lag over time. Which two Cloud Monitoring metrics should be examined to diagnose the cause?

Accepted Answer

Dataflow job/system_lag and Dataflow job/data_freshness. System lag measures the time between event ingestion and processing. Data freshness shows the watermark. Worker CPU indicates compute resource issues.

Answer

Pub/Sub subscription/num_undelivered_messages and Dataflow job/watermark_lag

Answer

Dataproc cluster/yarn_allocated_memory_percentage and Dataflow job/worker_cpu

Answer

BigQuery query/execution_times and Dataflow job/elapsed_time

Question 5

A data team needs to share a BigQuery dataset with another business unit. They want to provide a point-in-time snapshot of the data without incurring additional storage costs for the copy. Which BigQuery feature should they use?

Accepted Answer

BigQuery table clones. Clones use the same underlying storage as the source table; snapshots also share storage but are immutable. Both are cost-effective. For regular updates, clones are more flexible.

Answer

BigQuery table snapshots

Answer

BigQuery authorized views

Answer

BigQuery export to Cloud Storage

Question 6

An engineer needs to create a reusable Dataflow pipeline that can be executed with different parameters without modifying code. Which Dataflow feature should they use?

Accepted Answer

Dataflow Flex Templates. Flex Templates allow packaging a pipeline into a Docker image with parameterization, enabling reuse across different environments.

Answer

Dataflow Shuffle

Answer

Dataflow SQL

Answer

Dataflow Classic Templates

Question 7

A company runs a Dataproc cluster for ETL jobs that process data nightly. They want to reduce costs while maintaining performance. Which strategy is MOST effective?

Accepted Answer

Use preemptible VMs for worker nodes only. Preemptible VMs are cheaper and suitable for fault-tolerant batch jobs. They can be used for worker nodes in Dataproc.

Answer

Use committed use discounts for all VMs

Answer

Enable Dataproc auto-scaling

Answer

Use preemptible VMs for all nodes including master

Question 8

A data engineer needs to alert when Pub/Sub subscription has messages older than 1 hour. Which Cloud Monitoring metric and filter should they use?

Accepted Answer

Metric: subscription/oldest_unacked_message_age; filter: subscription_id. The metric subscription/oldest_unacked_message_age gives the age of the oldest unacknowledged message. Filtering by subscription ID targets the specific subscription.

Answer

Metric: topic/send_message_operation_count; filter: topic_id

Answer

Metric: subscription/ack_message_count; filter: subscription_id

Answer

Metric: subscription/num_undelivered_messages; filter: subscription_id

Question 9

A team wants to enforce data quality rules on BigQuery tables using Dataplex. They need to run column-level checks for null values and row-level checks for value ranges on a schedule. Which Dataplex feature should they use?

Accepted Answer

Dataplex Data Quality Tasks. Dataplex Data Quality Tasks allow defining SQL-based rules (row and column) and scheduling scans.

Answer

Dataplex Data Profiling

Answer

BigQuery stored procedures with scheduled queries

Answer

Cloud DLP inspection jobs

Question 10

An organization uses BigQuery on-demand pricing. To control costs, they want to estimate the bytes processed by a query before running it. Which command or method should they use?

Accepted Answer

Use the bq query --dry_run command. BigQuery dry run estimates bytes processed without executing the query. It can be done via CLI with --dry_run flag or in the console.

Answer

Use bq ls to list table sizes

Answer

Use BigQuery reservations to get cost estimate

Answer

Use INFORMATION_SCHEMA.JOBS_BY_PROJECT to view past costs

Question 11

A company uses Cloud Composer for pipeline orchestration. They need to define task dependencies where Task B and Task C can run in parallel after Task A, and Task D must run after both B and C complete. How should they define the DAG?

Accepted Answer

A >> [B, C] >> D. Using bitshift operators: A >> [B, C] >> D sets B and C after A, and D after both B and C complete.

Answer

A >> B; B >> D; A >> C; C >> D

Answer

A >> B >> C >> D

Answer

A.set_downstream(B); B.set_upstream(C); C.set_downstream(D)

Question 12

A streaming Dataflow pipeline needs to be updated without draining the existing pipeline. Which update strategy should be used?

Accepted Answer

Replace the job with a new job using the same pipeline name. Dataflow Streaming Engine allows updates without draining by using the same pipeline name and enabling Streaming Engine.

Answer

Drain the pipeline first, then start a new one

Answer

Use a different pipeline name and cancel the old one

Answer

Stop the job, update the code, and restart

Question 13

A company wants to use Cloud DLP to inspect data in BigQuery for sensitive information and de-identify it by masking credit card numbers. They want to perform this on a schedule. Which approach should they take?

Accepted Answer

Use Cloud DLP inspect and de-identify jobs triggered by Cloud Scheduler. Cloud DLP can inspect BigQuery tables and de-identify using transforms like masking. Scheduling can be done via Cloud Scheduler.

Answer

Use Dataplex data quality rules with a custom SQL regex

Answer

Use Cloud Data Loss Prevention API with Cloud Composer

Answer

Use BigQuery column-level security with classification

Question 14

A data engineer notices that BigQuery queries are slower than expected. They want to identify the most expensive stages in the query execution. Which tool or command should they use?

Accepted Answer

Use bq query --format=prettyjson and look at statistics. To identify expensive stages in BigQuery query execution, you should run the query with the bq command-line tool using the --format=prettyjson flag. This returns a detailed job object that includes a query plan with stages, each having cost estimates like 'avg execution time' and 'peak memory'. Analyzing these stages reveals the most resource-intensive parts.

Answer

Use bq show to view job statistics

Answer

Use Cloud Monitoring to view query execution graphs

Answer

Use EXPLAIN statement in BigQuery

Question 15

A data engineer needs to migrate a schema from BigQuery where a column is currently REQUIRED and needs to become NULLABLE. Which TWO statements are correct? (Choose 2)

Accepted Answer

Drop the column and add it again as NULLABLE. BigQuery does not allow changing a column from REQUIRED to NULLABLE directly. One must either drop and recreate the column or use a query to create a new table.

Answer

Use ALTER TABLE RENAME COLUMN and then add new column

Answer

Use bq update --schema to change the mode

Answer

Use ALTER TABLE ALTER COLUMN SET DATA TYPE to change to NULLABLE