Databricks-DE-Assoc Working with Lakeflow Jobs — All Questions With Answers

Question 1

A data engineer wants to schedule a Databricks Job to run every Tuesday at 8:30 AM UTC. Which cron expression correctly represents this schedule?

Accepted Answer

30 8 ? * TUE *. Databricks Jobs support Quartz cron syntax for precise scheduling. A valid cron expression consists of fields for seconds, minutes, hours, day of month, month, and day of week. Understanding cron syntax ensures production pipelines execute automatically at expected maintenance windows and business hours.

Answer

30 8 * * 2

Answer

8 30 ? * 2 *

Answer

0 30 8 * * TUE

Question 2

When configuring a Databricks Job, which notification option ensures that a team is alerted specifically when a job fails?

Accepted Answer

Email notifications on failure.. In the Databricks Jobs UI, the specific setting is found under 'Notifications' where you can add an email address and select the 'On failure' trigger. The option 'Email notifications on failure' is a descriptive summary, but the UI specifically refers to 'Email notifications' with a trigger condition. Additionally, the explanation is generic and does not explain why the other options are incorrect in the context of Databricks functionality.

Answer

Automatic Slack integration without setup.

Answer

Displaying a popup in the UI.

Answer

Writing a log entry to the console.

Question 3

A data engineer is configuring a Lakeflow Job with multiple tasks. The engineer wants to ensure that the job can be repaired and that only failed tasks are re-run without re-executing successful tasks. Which TWO features or configurations enable this behavior? (Choose two.)

Accepted Answer

Use the "Repair run" option from the job run details page.. Repair runs in Lakeflow Jobs allow re-execution of only failed or skipped tasks, preserving the results of successful tasks. This is enabled by the "Repair run" feature, which can be initiated from the job run details page. Autoscaling, "Run if" conditions, and frequent schedules do not provide this selective re-execution capability, so they are incorrect for this scenario.

Answer

Configure tasks with "Run if" conditions set to "All succeeded".

Answer

Set the job to use a job cluster with autoscaling.

Answer

Set the job to trigger on a schedule with a short interval.

Question 4

A data engineer is configuring a Lakeflow Job with a task that runs a Python wheel. The engineer needs to ensure that the task can access a specific set of Python libraries that are not included in the Databricks Runtime. The engineer wants to avoid installing these libraries at runtime using %pip. Which TWO of the following approaches are valid for making these libraries available to the task? (Choose two.)

Accepted Answer

Attach the libraries to the job cluster as cluster-scoped libraries.. The two valid approaches are attaching libraries to the job cluster as cluster-scoped libraries and specifying them in the task's dependent libraries. Both methods ensure the libraries are installed before task execution, leveraging built-in job configuration rather than runtime installation.

Answer

Use an init script to pip install the libraries when the cluster starts.

Answer

Package the libraries into the Python wheel itself and upload it as the task.

Answer

Set the task's environment variable PYTHONPATH to point to a DBFS location containing the libraries.

Question 5

A data engineer is building a Lakeflow Job with three tasks: bronze_ingest, silver_transform, and gold_aggregate. The engineer wants gold_aggregate to run only when silver_transform succeeds, and wants the job to skip gold_aggregate without marking the whole job as failed when silver_transform is skipped. Which two configuration choices support this behavior? (Choose two.)

Accepted Answer

Enable the job-level setting so skipped tasks do not cause the run to be marked as failed.. To run a downstream task only on upstream success and to skip it without failing the job, the dependency condition must evaluate correctly when the upstream is skipped, and the job must treat skipped tasks as a non-failure. The at-least-one-succeeded condition ensures gold_aggregate runs only when silver_transform succeeds, and the job-level skip handling keeps the run from being marked failed. Other settings affect earlier edges or retries and do not satisfy the requirement.

Answer

Set the gold_aggregate task's dependency on silver_transform to use the default all-success requirement.

Answer

Add a retry policy to gold_aggregate so it retries when silver_transform is skipped.

Answer

Configure the silver_transform task to depend on bronze_ingest with an all-success requirement.

Question 6

A data engineer configures a Lakeflow Job with a single notebook task. The task sometimes runs longer than expected because of upstream source latency. The engineer wants the task to be automatically retried only when it fails, but they do not want to pay for idle compute between retries. Which configuration should they use?

Accepted Answer

Set the task's Retries to 3 and set the job cluster to terminate after a period of inactivity.. The task needs automatic reattempts on failure, which is exactly what the Retries setting provides. To avoid paying for idle compute between attempts, the job cluster should be configured to terminate after an inactivity period; that way the cluster does not sit idle indefinitely while waiting to retry. Combining task-level retries with cluster auto-termination satisfies both functional and cost requirements without introducing manual fallback tasks.

Answer

Set the task's Retries to 3 and configure the cluster with autoscaling.

Answer

Set the task's Retries to 3 and Max concurrent runs to 3.

Answer

Leave Retries at 0 and add a second identical task that runs only if the first fails.

Question 7

A data engineer needs to run a Lakeflow Job every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a scheduled trigger with a cron expression. Which cron expression correctly represents this schedule in Quartz format?

Accepted Answer

0 0 6 ? * MON-FRI *. Quartz cron expressions require six or seven fields in the order seconds, minutes, hours, day-of-month, month, day-of-week, and optional year. To run at 06:00 on weekdays, the hours field is 6, minutes and seconds are 0, day-of-week is MON-FRI, and day-of-month is a question mark to avoid conflict. The other expressions misplace fields or use an asterisk where a question mark is required.

Answer

0 6 * ? * MON-FRI *

Answer

0 0 6 ? * * MON-FRI

Answer

0 0 6 * * MON-FRI *

Question 8

A data engineer has a Lakeflow Job with a notebook task that reads from a streaming source and is expected to run indefinitely. The engineer wants the task to be restarted automatically if the cluster is terminated or the notebook fails due to an out-of-memory error, but does not want the entire job to be marked as failed. The engineer also wants to limit the number of restart attempts to 3. Which configuration should be used for this task?

Accepted Answer

Set the task's maximum retries to 3 and enable 'Retry on failure' for the task, and set the job to run continuously.. For a streaming notebook task that should run indefinitely and recover from failures with a limited number of retries, the correct approach is to enable retries on the task and set the job to continuous mode. This ensures the task restarts on failure or cluster termination, up to the specified maximum retries, without failing the entire job permanently.

Answer

Configure the task with a retry policy that includes 'on cluster termination' and set the maximum retries to 3.

Answer

Use a continuous job with the 'Restart on failure' option and set the maximum retries to 3.

Answer

Set the task's maximum retries to 3 and enable the 'Retry on failure' option.

Question 9

A data engineer is configuring a Lakeflow Job with a parameter named 'env' that has a default value of 'dev'. The engineer wants to override this value to 'prod' when triggering the job via the Databricks REST API. The engineer also wants to ensure that the parameter is available to all tasks in the job. Which approach should be used?

Accepted Answer

Pass the parameter as a job-level parameter in the API request using the 'job_parameters' field, and reference it in tasks using dynamic value references.. Job-level parameters allow a single value to be defined for the entire job and overridden at runtime via the API. They can be referenced in tasks using dynamic value references, making them available to all tasks. Task-level parameters or notebook-specific parameters do not provide the same job-wide scope and override capability.

Answer

Pass the parameter as a task-level parameter in the API request, and set it individually for each task.

Answer

Set the parameter in the job's cluster configuration as an environment variable, and access it using os.environ in the notebook.

Answer

Use the 'notebook_params' field in the API request to pass 'env=prod', and reference it in the notebook using dbutils.widgets.get('env').

Question 10

A data engineer needs to configure a Databricks Job containing multiple tasks where downstream tasks should only execute if all upstream parent tasks complete successfully. Which task dependency setting should be configured?

Accepted Answer

Add the upstream tasks as parents of the downstream task directly within the job configuration graph.. To ensure tasks execute conditionally based on the success of parent tasks, you configure task dependencies within the Databricks Jobs UI or JSON definition. By default, adding a parent task creates a strict dependency where downstream tasks trigger only upon successful completion. This mechanism orchestrates complex directed acyclic graphs for robust, reliable data pipelines.

Answer

Configure a retry policy with exponential backoff on the parent task to guarantee eventual success.

Answer

Set up a global workspace webhook to trigger downstream tasks manually after the parent completes.

Answer

Enable concurrent run suppression on the job level to prevent overlapping workflow executions.

Question 11

Which TWO of the following statements are true regarding the behavior and capabilities of Databricks Jobs parameters and values?

Accepted Answer

Task values can be used to pass small amounts of data, such as counts or status codes, from one task to a downstream task.. Databricks Jobs support parameterized runs, enabling dynamic workflows where values are passed at runtime or configured statically. Understanding how parameters propagate to tasks and how task values pass outputs between tasks is essential for building modular, reusable data engineering pipelines within the Lakeflow framework.

Answer

Job parameters are automatically persisted in a Unity Catalog volume after the workflow completes successfully.

Answer

Notebook tasks cannot accept parameter values passed from the parent job orchestration configuration.

Answer

Task values support passing large dataframes containing millions of rows between dependent tasks.

Question 12

A data engineer has configured a Databricks Job with multiple dependent tasks forming a linear pipeline. Task A extracts data, Task B transforms it, and Task C loads it into a gold table. The pipeline runs daily. The team notices that if Task B fails due to an intermittent schema validation issue, the entire job run fails, but they want Task C to execute conditionally only if Task B succeeds, while alerting the on-call engineer immediately upon any failure. How should the task dependencies and conditional execution be configured?

Accepted Answer

Ensure Task C has a dependency pointing exclusively to Task B, and configure a job-level email notification for failures.. Configuring explicit task dependencies via the UI or API ensures that downstream tasks only execute upon the successful completion of their parents. By linking Task C strictly to Task B's success, you prevent corrupting downstream layers with partial or failed upstream transformations. This workflow pattern is essential for maintaining reliable data engineering pipelines in production Lakeflow environments.

Answer

Set Task C to run on failure of Task B to capture the exception state before alerting.

Answer

Convert Task B and Task C into a single notebook task and handle exceptions internally with try-except blocks.

Answer

Disable the task dependency between Task B and Task C and run them in parallel with different cluster configurations.

Question 13

A data engineer needs to configure a Databricks Job containing three distinct tasks: ingest, transform, and report. The transform task must only execute if the ingest task completes successfully, but the report task should execute regardless of whether the transform task succeeds or fails. How should the task dependencies be configured?

Accepted Answer

Configure transform to depend on ingest, and do not include transform as a dependency for report.. Task dependencies in Databricks Jobs are managed by defining upstream parent tasks using the depends_on field. To make transform run after ingest, transform depends on ingest. To make report run regardless of transform's outcome, report must not depend on transform, or should be configured independently. Correctly structuring task dependencies ensures downstream consumers do not ingest corrupt data while maintaining resilience for reporting tasks.

Answer

Configure transform to depend on ingest, and configure report to depend on transform.

Answer

Configure ingest to depend on transform, and configure report to depend on ingest.

Answer

Configure both transform and report to depend on ingest simultaneously.

Question 14

A data engineer wants to ensure that a Databricks Job task only runs if the preceding task completes successfully, but needs to add a specific timeout threshold for this individual task. Where should this configuration be applied?

Accepted Answer

Within the individual task definition.. In Databricks Jobs, task-level configurations such as timeouts, retries, and dependencies are defined within the task definition itself. By specifying a timeout in the task settings, the engineer prevents runaway processes from consuming cluster resources indefinitely. Understanding this granularity is vital for cost management and workflow reliability, as it allows engineers to set different SLAs for ingestion versus transformation tasks within a single orchestrated pipeline.

Answer

At the job level settings.

Answer

Inside the cluster configuration JSON.

Answer

Using a global Workspace policy.

Question 15

A data engineer is designing a Databricks Job workflow. Which TWO of the following are valid ways to trigger a Databricks Job?

Accepted Answer

Using the 'Run now' button in the UI.. Databricks Jobs offer multiple entry points for execution. Understanding these triggers is essential for integrating pipelines into broader CI/CD cycles or event-driven architectures. By mastering these methods, engineers can effectively decouple their data processing logic from the scheduling mechanism, allowing for both manual verification during development and robust automated execution in production environments through APIs or service principals.

Answer

Directly editing the Python source code file.

Answer

Changing the cluster node type.

Answer

Modifying the job owner permissions.

Question 16

Refer to the exhibit. If 'task1' fails due to a timeout, what happens to 'task2'?

Accepted Answer

It will be skipped by the job scheduler.. In Databricks Jobs, dependencies are strictly evaluated based on the success of preceding tasks. If a task fails—regardless of whether it was a timeout or a code error—the dependent tasks in the directed acyclic graph (DAG) will not be triggered. This behavior ensures that downstream data quality is protected from incomplete or stale upstream data processing, which is a fundamental requirement for maintaining reliable data pipelines.

Answer

It will run anyway because it is independent.

Answer

It will pause until task1 is restarted.

Answer

It will run with a 'warning' status.

Question 17

Which of the following is the primary benefit of using a 'Job Cluster' rather than an 'All-Purpose Cluster' for running scheduled data pipelines?

Accepted Answer

It provides lower pricing for compute resources.. Job clusters are ephemeral and specifically designed for automated tasks, offering significant cost savings compared to all-purpose clusters. By terminating automatically after the job completes, they eliminate idle compute time costs. Understanding this distinction is essential for data engineers to balance performance requirements with budgetary constraints, as choosing the correct compute type directly impacts the overall operational efficiency and cost-effectiveness of the data platform.

Answer

It allows interactive development in notebooks.

Answer

It supports faster library installation.

Answer

It caches data indefinitely for all users.

Question 18

A data engineer needs to pass the execution date to a job task dynamically. Which feature should they use?

Accepted Answer

Defining job parameters in the task configuration.. Using Job Parameters allows engineers to inject dynamic values into tasks at runtime. This capability is crucial for backfilling data or running incremental pipelines where the processing logic depends on specific dates or data partitions. By parameterizing tasks, engineers create reusable jobs that can handle varying datasets without requiring manual code changes, thereby improving the maintainability and scalability of the entire data pipeline architecture.

Answer

Hardcoding the date in the notebook.

Answer

Creating a new cluster for each date.

Answer

Using global workspace variables.

Question 19

Which THREE of the following are benefits of using Delta Live Tables (DLT) for managing your data pipelines?

Accepted Answer

Automated data quality monitoring via expectations.. Delta Live Tables simplifies the complexity of building reliable data pipelines by handling infrastructure orchestration, quality checks, and dependency management automatically. Understanding these benefits is essential for modern data engineering, as DLT reduces the burden of manual configuration, ensures data reliability through expectations, and provides declarative syntax that allows engineers to focus on business logic rather than low-level Spark optimization or cluster management tasks.

Answer

Manual management of cluster provisioning.

Answer

Ability to use non-Delta file formats.

Question 20

What is the primary function of the 'Retries' setting in a Databricks Job task?

Accepted Answer

To automatically recover from transient failures.. The 'Retries' setting is a robust mechanism for handling transient failures, such as network timeouts or temporary cloud provider issues, without manual intervention. By configuring retries, engineers increase the resilience of their pipelines. This is a critical best practice in production environments where external dependencies may be unstable, as it minimizes the need for on-call support and ensures that jobs eventually succeed despite minor, non-permanent infrastructure hiccups.

Answer

To increase the task execution speed.

Answer

To allow tasks to run in parallel.

Answer

To change the cluster node type.

Databricks-DE-Assoc Working with Lakeflow Jobs — All Questions With Answers

A data engineer wants to schedule a Databricks Job to run every Tuesday at 8:30 AM UTC. Which cron expression correctly represents this schedule?

When configuring a Databricks Job, which notification option ensures that a team is alerted specifically when a job fails?

A data engineer is configuring a Lakeflow Job with multiple tasks. The engineer wants to ensure that the job can be repaired and that only failed tasks are re-run without re-executing successful tasks. Which TWO features or configurations enable this behavior? (Choose two.)

A data engineer needs to run a Lakeflow Job every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a scheduled trigger with a cron expression. Which cron expression correctly represents this schedule in Quartz format?

A data engineer needs to configure a Databricks Job containing multiple tasks where downstream tasks should only execute if all upstream parent tasks complete successfully. Which task dependency setting should be configured?

Which TWO of the following statements are true regarding the behavior and capabilities of Databricks Jobs parameters and values?

A data engineer wants to ensure that a Databricks Job task only runs if the preceding task completes successfully, but needs to add a specific timeout threshold for this individual task. Where should this configuration be applied?

A data engineer is designing a Databricks Job workflow. Which TWO of the following are valid ways to trigger a Databricks Job?

Refer to the exhibit. If 'task1' fails due to a timeout, what happens to 'task2'?

Exhibit

Which of the following is the primary benefit of using a 'Job Cluster' rather than an 'All-Purpose Cluster' for running scheduled data pipelines?

A data engineer needs to pass the execution date to a job task dynamically. Which feature should they use?

Which THREE of the following are benefits of using Delta Live Tables (DLT) for managing your data pipelines?

What is the primary function of the 'Retries' setting in a Databricks Job task?

Refer to the exhibit. When is this job scheduled to run?

Exhibit

When a Data Engineer uses a 'Repair and Rerun' functionality on a failed Databricks Job, what happens?

A data engineer is setting up a Lakeflow Job that runs a notebook task. The engineer needs the task to always execute even if the upstream task in the workflow fails. Which configuration should be applied to the dependent task's condition?

A data engineer is creating a Lakeflow Job that must run a Python script stored in DBFS. The engineer wants to ensure the script is executed with the correct dependencies and environment. Which task type should be used?

A data engineer is creating a Lakeflow Job that must run a notebook every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a schedule trigger but the job runs at the wrong time. Which setting should the engineer verify first?

A data engineer is building a Lakeflow Job with a task that runs a SQL notebook. The task must run only on weekdays and must be completed before 9 AM. The engineer wants to configure the schedule to meet these requirements. What should the engineer do?

A data engineer is configuring a Lakeflow Job that processes sensitive customer data. The job must notify the on-call team when a run fails and must also capture the run's output for auditing. Which TWO actions should the engineer take in the Lakeflow Jobs configuration? (Choose two.)

A data engineer is building a Lakeflow Job that must process a parameterized date range. The engineer wants to pass start_date and end_date values into a notebook task at runtime and have those values available as widget-like parameters inside the notebook. Which approach should the engineer use?

A data engineer has a Lakeflow Job that runs daily. They want to receive an email only when the job fails, not on every run. Which notification configuration should they set?

A data engineer wants to pass a file path from Task A to Task B in a Lakeflow Job. Task A is a notebook that computes the path, and Task B is a notebook that reads from that path. Which mechanism should the engineer use to share the value between tasks?

Databricks-DE-Assoc Working with Lakeflow Jobs — All Questions With Answers

A data engineer wants to schedule a Databricks Job to run every Tuesday at 8:30 AM UTC. Which cron expression correctly represents this schedule?

When configuring a Databricks Job, which notification option ensures that a team is alerted specifically when a job fails?

A data engineer is configuring a Lakeflow Job with multiple tasks. The engineer wants to ensure that the job can be repaired and that only failed tasks are re-run without re-executing successful tasks. Which TWO features or configurations enable this behavior? (Choose two.)

A data engineer needs to run a Lakeflow Job every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a scheduled trigger with a cron expression. Which cron expression correctly represents this schedule in Quartz format?

A data engineer needs to configure a Databricks Job containing multiple tasks where downstream tasks should only execute if all upstream parent tasks complete successfully. Which task dependency setting should be configured?

Which TWO of the following statements are true regarding the behavior and capabilities of Databricks Jobs parameters and values?

A data engineer wants to ensure that a Databricks Job task only runs if the preceding task completes successfully, but needs to add a specific timeout threshold for this individual task. Where should this configuration be applied?

A data engineer is designing a Databricks Job workflow. Which TWO of the following are valid ways to trigger a Databricks Job?

Refer to the exhibit. If 'task1' fails due to a timeout, what happens to 'task2'?

Exhibit

Which of the following is the primary benefit of using a 'Job Cluster' rather than an 'All-Purpose Cluster' for running scheduled data pipelines?

A data engineer needs to pass the execution date to a job task dynamically. Which feature should they use?

Which THREE of the following are benefits of using Delta Live Tables (DLT) for managing your data pipelines?

What is the primary function of the 'Retries' setting in a Databricks Job task?

Refer to the exhibit. When is this job scheduled to run?

Exhibit

When a Data Engineer uses a 'Repair and Rerun' functionality on a failed Databricks Job, what happens?

A data engineer is setting up a Lakeflow Job that runs a notebook task. The engineer needs the task to always execute even if the upstream task in the workflow fails. Which configuration should be applied to the dependent task's condition?

A data engineer is creating a Lakeflow Job that must run a Python script stored in DBFS. The engineer wants to ensure the script is executed with the correct dependencies and environment. Which task type should be used?

A data engineer is creating a Lakeflow Job that must run a notebook every weekday at 06:00 in the company's local time zone, which is America/New_York. The engineer configures a schedule trigger but the job runs at the wrong time. Which setting should the engineer verify first?

A data engineer is building a Lakeflow Job with a task that runs a SQL notebook. The task must run only on weekdays and must be completed before 9 AM. The engineer wants to configure the schedule to meet these requirements. What should the engineer do?

A data engineer is configuring a Lakeflow Job that processes sensitive customer data. The job must notify the on-call team when a run fails and must also capture the run's output for auditing. Which TWO actions should the engineer take in the Lakeflow Jobs configuration? (Choose two.)

A data engineer is building a Lakeflow Job that must process a parameterized date range. The engineer wants to pass start_date and end_date values into a notebook task at runtime and have those values available as widget-like parameters inside the notebook. Which approach should the engineer use?

A data engineer has a Lakeflow Job that runs daily. They want to receive an email only when the job fails, not on every run. Which notification configuration should they set?

A data engineer wants to pass a file path from Task A to Task B in a Lakeflow Job. Task A is a notebook that computes the path, and Task B is a notebook that reads from that path. Which mechanism should the engineer use to share the value between tasks?