GCP-ADP Data Pipeline Orchestration Practice Test 2 — 15 Questions

Question 1

You are troubleshooting a stuck task in Cloud Composer. You want to see the stdout/stderr logs for that specific task attempt. Where can you find them?

Accepted Answer

In the 'logs/' folder in the environment's GCS bucket.. Airflow task logs are stored in the GCS bucket associated with the environment, organized by DAG and task name.

Answer

In the Airflow metadata database directly.

Answer

In the Cloud Composer environment console logs.

Answer

Via the 'kubectl' logs command on the worker pod.

Question 2

In Cloud Composer, what is the default behavior if a DAG execution takes longer than the 'dagrun_timeout' setting?

Accepted Answer

The DAG run is marked as failed.. If a DAG run exceeds the timeout, Airflow marks the DAG run as 'failed'.

Answer

The DAG enters a 'running' state indefinitely.

Answer

The DAG run is paused.

Answer

The scheduler automatically restarts the DAG.

Question 3

Which property in a Workflows YAML definition is required to define the entry point of the workflow?

Accepted Answer

main. The 'main' step is the required starting point for any workflow execution.

Answer

entry_point

Answer

init

Answer

start

Question 4

Which command-line tool do you primarily use to deploy and manage Workflows?

Accepted Answer

gcloud. The 'gcloud' CLI is the standard tool for managing Google Cloud services, including Workflows.

Answer

gsutil

Answer

kubectl

Answer

bq

Question 5

A task in your Airflow DAG involves a long-running process that might exceed the default Airflow worker pod timeout. How should you handle this?

Accepted Answer

Set the 'execution_timeout' parameter on the task instance.. Adjust the 'execution_timeout' parameter for the specific task in the Airflow DAG definition to allow longer processing times.

Answer

Move the task to a custom node pool in GKE.

Answer

Increase the 'default_dag_run_timeout'.

Answer

Decrease the number of parallel tasks.

Question 6

You need to pass a JSON object into a Workflows execution. How should you format the argument in the 'gcloud workflows executions' command?

Accepted Answer

Use the --data flag with a JSON string.. The '--data' flag accepts a JSON string, which is the standard format for passing parameters to workflows.

Answer

Pipe the JSON into the stdin of the command.

Answer

Use the --env-vars flag.

Answer

Upload the JSON to a GCS bucket and point to it.

Question 7

You are using Airflow with a Cloud Composer environment. You want to ensure that a specific task runs on a specific day of the week. How do you configure the schedule?

Accepted Answer

Set 'schedule_interval' using cron syntax.. The 'schedule_interval' parameter in the DAG definition supports cron syntax, which includes day-of-week specifications.

Answer

Create a separate DAG for each day of the week.

Answer

Enable the 'weekday_only' flag in the DAG configuration.

Answer

Use the 'timedelta' object with a 7-day offset.

Question 8

You want to dynamically generate tasks in an Airflow DAG based on the number of files in a GCS bucket. What is the standard way to do this?

Accepted Answer

Use a Python loop inside the DAG file to instantiate tasks.. Use a Python loop within the DAG file to iterate over the list of files (often using the GCS hook) and create Task instances dynamically.

Answer

Use the 'SubDagOperator'.

Answer

Configure an external metadata service to inject tasks.

Answer

Use the 'DynamicTaskGroup' operator in the Airflow library.

Question 9

A Cloud Composer environment is experiencing high latency in the Airflow UI. What is a common cause and mitigation for this?

Accepted Answer

Excessive DAG complexity; optimize/simplify DAG code.. High DAG count and complex dependency parsing can slow down the UI; using the 'DAG Serialization' feature (enabled by default in newer versions) helps, but optimizing DAG file size and complexity is the best mitigation.

Answer

GKE node power failure; restart cluster.

Answer

Too many users logged in; force logout.

Answer

Browser cache; clear your local cache.

Question 10

Which TWO of the following are valid ways to trigger a Cloud Composer DAG?

Accepted Answer

Airflow Web UI. DAGs can be triggered by a timer (schedule) or manually through the Airflow Web UI/CLI.

Answer

Sending an email to the Airflow scheduler.

Answer

Clicking 'Run' in the Cloud Composer console environment list.

Answer

Directly editing the DAG file in the bucket.

Question 11

Which THREE of the following are components of a Workflows definition?

Accepted Answer

main. Workflows consist of steps, main entry points, and can include variable assignments.

Answer

dag_schedule

Answer

worker_config

Question 12

What is the primary function of the Airflow 'Executor' in Cloud Composer?

Accepted Answer

Deciding how to distribute tasks to workers.. The executor is responsible for managing the execution of tasks, determining how to distribute work to the workers.

Answer

Serving the Airflow UI web pages.

Answer

Storing the DAG file metadata.

Answer

Automatically scaling the GKE cluster.

Question 13

Which TWO of the following are best practices for managing Cloud Composer DAG dependencies?

Accepted Answer

Use plugins for shared code. Best practices include keeping DAGs small and using modular code for shared tasks.

Answer

Store DAGs in a database

Answer

Hardcode all configurations

Answer

Run all tasks in a single DAG file

Question 14

Which THREE of the following are key benefits of using Cloud Composer for data orchestration?

Accepted Answer

Uses standard open-source Airflow. Composer provides managed infrastructure, native integration with GCP services, and the extensive Airflow ecosystem.

Answer

Automatic data transformation

Answer

Free tier for all users

Question 15

Which TWO of the following services can trigger Workflows executions?

Accepted Answer

Cloud Scheduler. Cloud Scheduler and Eventarc are common triggers for Workflows.

Answer

Cloud Functions

Answer

Cloud Logging

Answer

Cloud Build