Databricks-DE-Assoc Working with Lakeflow Jobs Practice Test 2 — 15 Questions

Question 1

Refer to the exhibit. If 'task1' fails due to a timeout, what happens to 'task2'?

Accepted Answer

It will be skipped by the job scheduler.. In Databricks Jobs, dependencies are strictly evaluated based on the success of preceding tasks. If a task fails—regardless of whether it was a timeout or a code error—the dependent tasks in the directed acyclic graph (DAG) will not be triggered. This behavior ensures that downstream data quality is protected from incomplete or stale upstream data processing, which is a fundamental requirement for maintaining reliable data pipelines.

Answer

It will run anyway because it is independent.

Answer

It will pause until task1 is restarted.

Answer

It will run with a 'warning' status.

Question 2

Which of the following is the primary benefit of using a 'Job Cluster' rather than an 'All-Purpose Cluster' for running scheduled data pipelines?

Accepted Answer

It provides lower pricing for compute resources.. Job clusters are ephemeral and specifically designed for automated tasks, offering significant cost savings compared to all-purpose clusters. By terminating automatically after the job completes, they eliminate idle compute time costs. Understanding this distinction is essential for data engineers to balance performance requirements with budgetary constraints, as choosing the correct compute type directly impacts the overall operational efficiency and cost-effectiveness of the data platform.

Answer

It allows interactive development in notebooks.

Answer

It supports faster library installation.

Answer

It caches data indefinitely for all users.

Question 3

A data engineer needs to pass the execution date to a job task dynamically. Which feature should they use?

Accepted Answer

Defining job parameters in the task configuration.. Using Job Parameters allows engineers to inject dynamic values into tasks at runtime. This capability is crucial for backfilling data or running incremental pipelines where the processing logic depends on specific dates or data partitions. By parameterizing tasks, engineers create reusable jobs that can handle varying datasets without requiring manual code changes, thereby improving the maintainability and scalability of the entire data pipeline architecture.

Answer

Hardcoding the date in the notebook.

Answer

Creating a new cluster for each date.

Answer

Using global workspace variables.

Question 4

Which THREE of the following are benefits of using Delta Live Tables (DLT) for managing your data pipelines?

Accepted Answer

Automated data quality monitoring via expectations.. Delta Live Tables simplifies the complexity of building reliable data pipelines by handling infrastructure orchestration, quality checks, and dependency management automatically. Understanding these benefits is essential for modern data engineering, as DLT reduces the burden of manual configuration, ensures data reliability through expectations, and provides declarative syntax that allows engineers to focus on business logic rather than low-level Spark optimization or cluster management tasks.

Answer

Manual management of cluster provisioning.

Answer

Ability to use non-Delta file formats.

Question 5

What is the primary function of the 'Retries' setting in a Databricks Job task?

Accepted Answer

To automatically recover from transient failures.. The 'Retries' setting is a robust mechanism for handling transient failures, such as network timeouts or temporary cloud provider issues, without manual intervention. By configuring retries, engineers increase the resilience of their pipelines. This is a critical best practice in production environments where external dependencies may be unstable, as it minimizes the need for on-call support and ensures that jobs eventually succeed despite minor, non-permanent infrastructure hiccups.

Answer

To increase the task execution speed.

Answer

To allow tasks to run in parallel.

Answer

To change the cluster node type.

Question 6

Refer to the exhibit. When is this job scheduled to run?

Accepted Answer

At 12:00 PM every day.. Understanding Quartz cron syntax is fundamental for scheduling jobs in Databricks. The expression '0 0 12 * * ?' maps to 12:00 PM (noon) daily. Being able to interpret and define schedules correctly is vital for data engineers to ensure that data pipelines run at the exact times required by business stakeholders, ensuring that data is fresh and available for reporting during peak business hours without unnecessary delays.

Answer

Every minute at 12 seconds.

Answer

Every 12 hours.

Answer

Every Monday at 12:00 AM.

Question 7

When a Data Engineer uses a 'Repair and Rerun' functionality on a failed Databricks Job, what happens?

Accepted Answer

It restarts from the failed task.. Repair and Rerun is a critical feature for managing complex DAG workflows. It allows engineers to restart a job from the specific failed task, inheriting the successful results of prior tasks. This saves significant time and compute costs by avoiding redundant processing of already completed data, which is essential for maintaining efficient pipelines and preventing unnecessary data re-computation in production environments.

Answer

It deletes all previous successful task logs.

Answer

It resets all task statuses to 'Pending'.

Answer

It requires a new cluster to be provisioned.

Question 8

A data engineer is setting up a Lakeflow Job that runs a notebook task. The engineer needs the task to always execute even if the upstream task in the workflow fails. Which configuration should be applied to the dependent task's condition?

Accepted Answer

Set the task's "Run if" condition to "All done".. In Lakeflow Jobs, task dependencies use "Run if" conditions to control execution based on upstream task states. "All done" ensures the task runs after all dependencies finish, regardless of success or failure. This is the only condition that satisfies the requirement to always execute, making it suitable for tasks that must run unconditionally, such as cleanup or alerting steps.

Answer

Set the task's "Run if" condition to "All succeeded".

Answer

Set the task's "Run if" condition to "At least one failed".

Answer

Set the task's "Run if" condition to "At least one succeeded".

Question 9

A data engineer has a Lakeflow Job with a notebook task that occasionally fails due to transient network errors when reading from an external REST API. The engineer wants the task to automatically retry up to three times, but only for this specific task, without affecting other tasks in the job. What should the engineer do?

Accepted Answer

Set the task's Retries to 3 in the task configuration.. The correct approach is to set the Retries field on the specific task to 3. This is a built-in feature of Lakeflow Jobs that applies per task, so other tasks remain unaffected. It is the simplest and most direct way to handle transient failures without modifying code or job-level settings.

Answer

Configure the task's Retries setting to 3 and add a retry condition using the task's timeout.

Answer

Use a notebook %run magic command to wrap the API call in a loop that retries three times.

Answer

Set the job-level maximum concurrent runs to 3 and enable retries.

Question 10

A data engineer is creating a Lakeflow Job that must run a Python script stored in DBFS. The engineer wants to ensure the script is executed with the correct dependencies and environment. Which task type should be used?

Accepted Answer

Python script task. Lakeflow Jobs offer a dedicated Python script task type that runs a Python file from DBFS or other storage locations. It supports specifying libraries and environment settings, ensuring the script runs with the correct dependencies. Notebook, JAR, and pipeline tasks serve different purposes and cannot directly execute a standalone Python script with its own environment.

Answer

JAR task

Answer

Notebook task

Answer

Pipeline task

Question 11

A data engineering team runs a nightly Lakeflow Job that ingests files from cloud storage, transforms them with a notebook, and then runs a SQL task. The team wants the SQL task to execute only after the notebook transform succeeds, but they do not want the SQL task to wait for a fixed delay. Which Lakeflow Jobs feature should they configure on the SQL task?

Accepted Answer

Set the SQL task's depends_on property to reference the notebook transform task.. The depends_on property is the native Lakeflow Jobs mechanism for expressing task-level dependencies. By setting depends_on on the SQL task to point at the notebook transform task, the SQL task is scheduled only after the transform completes successfully. This avoids fixed delays, avoids polling, and keeps the pipeline deterministic and observable within a single job run.

Answer

Configure a time-based trigger on the SQL task using a cron expression that starts two minutes after the job run begins.

Answer

Add a run_if condition to the SQL task that checks whether the transform task's output path exists.

Answer

Place the SQL task and the notebook transform task in separate jobs and chain them using a job-level trigger.

Question 12

A data engineer maintains a Lakeflow Job with a scheduled trigger set to run every day at 08:00. The job's source table is refreshed by an upstream process that sometimes finishes later than expected, causing the job to process stale data. The engineer wants the job to start only after the upstream refresh completes, regardless of the clock time, while still preserving the existing 08:00 schedule as a fallback. Which trigger configuration should the engineer implement?

Accepted Answer

Add a table update trigger on the upstream table while keeping the scheduled trigger, so the job runs when the table refreshes or at 08:00.. The requirement is a job that starts when upstream data is ready, yet still runs at 08:00 if that event has not occurred. A table update trigger detects upstream commits and starts the job promptly, while the retained scheduled trigger supplies the fallback. Together they deliver event-driven orchestration with a time-based safety net, which neither a pure schedule nor a polling file trigger provides.

Answer

Convert the scheduled trigger to a continuous trigger so the job restarts immediately after each run.

Answer

Add a file arrival trigger that monitors the upstream table's storage location and remove the scheduled trigger.

Answer

Add a run-duration timeout to the scheduled trigger so the job fails if the upstream has not finished by 08:00.

Question 13

A data engineer is creating a Lakeflow Job that must run a notebook every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a schedule trigger but the job runs at the wrong time. Which setting should the engineer verify first?

Accepted Answer

The job's time zone setting, which controls how the cron expression is interpreted.. The schedule time zone determines how the cron expression is interpreted. If the job is configured with a cron for 06:00 but the time zone is UTC, the run occurs at 06:00 UTC, which is a different local time. Setting the schedule time zone to America/New_York aligns the trigger with the team's expectation and avoids manual conversion mistakes.

Answer

The notebook's default language, because Python and SQL notebooks evaluate cron expressions differently.

Answer

The job's maximum concurrent runs setting, because concurrency delays the start time.

Answer

The cluster's Spark configuration, because executor time zones affect cron evaluation.

Question 14

A data engineer is building a Lakeflow Job with a task that runs a SQL notebook. The task must run only on weekdays and must be completed before 9 AM. The engineer wants to configure the schedule to meet these requirements. What should the engineer do?

Accepted Answer

Use a cron expression with a schedule that runs at 8 AM Monday through Friday.. The correct method is to use a cron schedule that runs at 8 AM Monday through Friday. Lakeflow Jobs natively support cron expressions, allowing precise control over run times and days. This satisfies the weekday-only and before-9-AM requirements without additional complexity.

Answer

Configure the job to run continuously with a 1-hour pause between runs and enable a weekday filter.

Answer

Set the job to trigger on a file arrival event and filter for weekdays.

Answer

Use a Databricks SQL alert to trigger the job when a condition is met.

Question 15

A data engineer has a Lakeflow Job with two tasks: Task1 and Task2. Task2 must run only if Task1 succeeds. The engineer also wants Task2 to be skipped if Task1 fails, but the overall job status should be marked as failed. Which configuration should the engineer use for the dependency between Task1 and Task2?

Accepted Answer

Set Task2 to depend on Task1 with the 'All succeeded' condition.. The 'All succeeded' condition ensures Task2 runs only when Task1 succeeds. If Task1 fails, Task2 is skipped, and the job is marked as failed because a task failed. This precisely meets the requirement of conditional execution and overall job failure.

Answer

Set Task2 to depend on Task1 with the 'All done' condition.

Answer

Set Task2 to depend on Task1 with the 'At least one succeeded' condition.

Answer

Set Task2 to depend on Task1 with the 'None failed' condition.