Databricks-DE-Pro Debugging and Deploying — Timed 20 Questions

Question 1

A data engineer is debugging a Databricks Job that frequently fails due to transient network issues while reading from an external S3 bucket. Which configuration should the engineer implement to improve job resilience?

20 minute time limit — choose an answer to begin.

Accepted Answer

Set the 'max_retries' attribute in the task configuration to a value greater than zero.. Databricks Jobs use the 'max_retries' field within the 'retry_policy' object of the Job API, but it is not a top-level attribute as implied by the option. Furthermore, the question asks for a configuration to improve resilience against transient network issues during reads; while retries are correct, the specific configuration for Databricks Jobs is 'max_retries' inside a 'retry_policy' block, and the explanation incorrectly describes it as a simple task attribute.

Answer

Increase the driver node instance type to provide more memory for buffering S3 connections.

Answer

Implement a try-catch block within the Python script to manually restart the cluster.

Answer

Use a standard cluster instead of a job cluster to ensure persistent connectivity to S3.

Question 2

A data engineer is using Databricks Asset Bundles (DABs) to deploy a project. The deployment fails because the local configuration differs from the remote environment. What is the best way to synchronize the environment?

Accepted Answer

Update the bundle target configuration and run 'databricks bundle deploy'.. Databricks Asset Bundles (DABs) are declarative. When local configuration differs from the remote environment, the 'databricks bundle deploy' command calculates the difference and applies the necessary changes to the workspace to match the local configuration. Option B is the correct approach as it ensures the target configuration is accurate before triggering the reconciliation process.

Answer

Manually delete all resources in the workspace and redeploy the bundle.

Answer

Force the deployment by setting the 'ignore-errors' flag to true.

Answer

Use the 'databricks bundle import' command to pull the remote state locally.

Question 3

A developer is writing code that reads a large amount of data from a table and applies a complex transformation. The job succeeds locally but fails in the production job cluster. What is the most likely reason?

Accepted Answer

The local notebook has implicit access to cached data not available in production.. Interactive notebook environments (often referred to as 'local' development in this context) maintain state across cell executions. A developer might have cached intermediate results or created temporary views in previous steps that are not explicitly defined in the production script. When the job runs in a production cluster, it starts with a fresh SparkSession. If the complex transformation relied on those implicit caches to stay within memory limits or to bypass expensive re-computations, the production job will fail. This is a common 'gotcha' when moving from interactive development to automated jobs.

Answer

The production cluster is using a different version of the Python interpreter.

Answer

The production cluster is configured to use a single node, which is faster.

Answer

The Spark UI is disabled in the production job cluster for performance.

Question 4

A data engineer is debugging a Databricks Job that fails intermittently due to cluster startup delays. Which strategy best minimizes this impact on production reliability?

Accepted Answer

Attach the job to an existing instance pool.. Utilizing instance pools pre-allocates a set of idle, ready-to-use instances, effectively reducing cluster start-up times to near-zero. This approach is critical for mission-critical jobs where strict SLAs must be met. By decoupling cluster provisioning from job execution, engineers ensure that transient cloud provider capacity issues do not trigger cascading failures in automated ETL pipelines, thereby improving overall system stability and reducing time-to-completion metrics for downstream dependencies.

Answer

Increase the maximum number of workers in the cluster configuration.

Answer

Enable cluster termination after 10 minutes of inactivity.

Answer

Switch to a single-node cluster to eliminate node communication overhead.

Question 5

A data engineer has a Databricks Asset Bundle (DAB) project with a job defined in `resources/jobs.yml`. After running `databricks bundle validate`, they see the error: `unknown field: job_clusters`. They are using Databricks CLI version 0.205.0. What is the most likely cause of this error?

Accepted Answer

The Databricks CLI version is too old and does not support the `job_clusters` field in the bundle schema.. The error occurs because the installed Databricks CLI version does not include `job_clusters` in its bundle schema. Asset Bundles are version-dependent, and fields are added over time. Upgrading the CLI ensures compatibility with newer bundle features. The other options misdiagnose the cause as a missing field, indentation, or missing metadata, which would produce different errors.

Answer

The `job_clusters` field is not supported in the bundle schema for the CLI version; they should use `new_cluster` instead.

Answer

The bundle configuration is missing a `bundle` section that defines the CLI version compatibility.

Answer

The `job_clusters` field is defined at the wrong indentation level, causing the YAML parser to misinterpret it.

Question 6

A data engineer has a Databricks job that reads from a Delta table and writes to another Delta table. The job sometimes fails with a 'FileNotFoundException'. They suspect that the source table is being overwritten by another process while the job is running. Which Delta Lake feature should they use to ensure the job reads a consistent snapshot of the source table?

Accepted Answer

Set the Spark configuration 'spark.databricks.delta.snapshotIsolation' to 'true' to enable snapshot isolation.. Delta Lake provides snapshot isolation for reads by default, meaning a query sees a consistent version of the table. However, if another process overwrites the table and removes old files, a long-running read may fail with FileNotFoundException. To prevent this, the job should either use time travel to pin to a specific version or ensure that vacuum does not delete files needed for the read. Setting 'spark.databricks.delta.snapshotIsolation' to 'true' reinforces this behavior, but it is already the default. The key is to manage file retention and possibly use time travel.

Answer

Use the 'mergeSchema' option when reading the table to handle schema changes.

Answer

Configure the job to read the table with the 'ignoreFileNotFound' option set to 'true'.

Answer

Enable Delta Lake time travel by setting the Spark configuration 'spark.databricks.delta.timeTravel.enabled' to 'true'.

Question 7

A data engineer is deploying a new production Databricks job using Databricks Asset Bundles (DABs). The bundle includes a job with a schedule and a task that runs a Python wheel. After running `databricks bundle deploy -t prod`, the job appears in the workspace but the schedule is not active. What is the most likely cause?

Accepted Answer

The schedule is defined in the bundle but the job's `schedule` property is set to `PAUSED` in the target override.. The schedule not being active after deployment often results from a target-specific override in the bundle configuration that sets the schedule status to PAUSED. Databricks Asset Bundles allow per-target customizations, and a common practice is to pause schedules in production until explicitly enabled. The other options either describe issues that would prevent deployment or are unrelated to schedule activation.

Answer

The job was deployed with `--fail-on-active-runs` flag, which pauses schedules by default.

Answer

The bundle target `prod` does not have a `workspace.host` configured in `databricks.yml`.

Answer

The job cluster does not have autoscaling enabled, so the schedule cannot be activated.

Question 8

A data engineer is troubleshooting a Databricks Workflow job that fails intermittently with 'Cluster terminated: Cloud provider launch failure'. The job uses a job cluster with a fixed number of workers. Which change is most likely to improve reliability?

Accepted Answer

Configure the job cluster to use spot instances with fallback to on-demand. Cloud provider launch failures are caused by the inability to provision requested instances, often due to spot instance unavailability. Configuring spot instances with on-demand fallback lets Databricks use alternative capacity when spot is scarce, directly improving the reliability of job cluster launches for intermittent failures.

Answer

Add a cluster init script to pre-install dependencies

Answer

Increase the driver node size

Answer

Enable autoscaling on the job cluster

Question 9

A data engineer is using Databricks Repos to manage a project. They need to ensure that the production job always uses the latest committed version of the code from the main branch. The job is configured to run from a Repo path. Which Git reference should be specified in the job configuration to achieve this?

Accepted Answer

The remote branch reference `refs/heads/main`. To ensure a job always uses the latest committed version from the main branch, the job should reference the branch (e.g., `main` or `refs/heads/main`). Databricks Repos will check out the latest commit on that branch when the job runs. Using a specific commit SHA pins to a fixed version, and tags are static. The full ref `refs/heads/main` is explicit and correct.

Answer

The tag `latest`

Answer

A specific commit SHA

Answer

The branch name `main`

Question 10

A data engineer maintains a Databricks Workflow that runs a notebook task referencing an external JAR library stored in DBFS at dbfs:/libs/etl-utils-1.2.jar. The notebook runs successfully in development on an all-purpose cluster because the engineer installed the library manually. In production, the Workflow task fails immediately with ClassNotFoundException. The cluster policy for the job cluster does not permit installing libraries through the UI. What should the engineer do to resolve the failure?

Accepted Answer

Add the JAR as a task library on the notebook task in the Workflow definition, specifying the dbfs:/libs/etl-utils-1.2.jar path.. Task-level libraries in a Workflow definition ensure that required JARs are installed on the job cluster before execution, independent of cluster policy restrictions. This is the supported, automated method for making libraries available to notebook tasks. Manually installing or using init scripts introduces fragility and does not guarantee classpath inclusion.

Answer

Modify the cluster policy to allow library installation, then install the JAR manually on the job cluster before each run.

Answer

Configure the job cluster to use an init script that downloads the JAR from DBFS and places it in the Spark classpath.

Answer

Use %pip install in the notebook to install the JAR from DBFS at the start of each task execution.

Question 11

A data engineer is deploying a Databricks job that uses a Python wheel task. The job needs to access a secret stored in Azure Key Vault to connect to an external database. The engineer wants to avoid hardcoding the secret and ensure the secret is not exposed in logs or notebook outputs. Which TWO configurations should be used to securely retrieve the secret? (Choose two.)

Accepted Answer

Use `dbutils.secrets.get(scope="keyvault", key="db-password")` to retrieve the secret.. The secure methods to retrieve secrets in Databricks are using `dbutils.secrets.get` to fetch the secret at runtime with redaction, and referencing secrets in task configurations via the `{{secrets/scope/key}}` syntax. Both avoid hardcoding and protect the secret from exposure. The other options either expose the secret or add unnecessary complexity and risk.

Answer

Use the Azure Key Vault SDK directly in the Python wheel to fetch the secret at runtime.

Answer

Embed the secret in the Python wheel as a constant and obfuscate it with base64 encoding.

Answer

Set the secret as an environment variable in the cluster's Spark configuration using `spark.executorEnv.DB_PASSWORD`.

Question 12

A data engineer is debugging a Databricks job that fails with the error 'org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 10.0 failed 4 times, most recent failure: Lost task 3.4 in stage 10.0 (TID 1234) (10.0.0.5 executor 2): java.lang.OutOfMemoryError: Java heap space'. The job processes a large dataset and performs a groupBy on a skewed key. The engineer has already increased the executor memory, but the error persists. Which action should the engineer take next to resolve the failure?

Accepted Answer

Enable adaptive query execution (AQE) and set spark.sql.adaptive.skewJoin.enabled to true to handle skewed partitions automatically.. Skewed groupBy keys cause a single task to process a disproportionately large partition, leading to OOM even with increased memory. Adaptive query execution with skew join optimization dynamically splits and replicates skewed partitions, balancing the load. This is the recommended approach for skew-related OOM in Spark SQL and DataFrame operations.

Answer

Increase the number of shuffle partitions by setting spark.sql.shuffle.partitions to a higher value, such as 4000.

Answer

Repartition the DataFrame on a different column before the groupBy to distribute data evenly across partitions.

Answer

Cache the DataFrame before the groupBy to avoid recomputation and reduce memory pressure.

Question 13

A data engineer is using Databricks Repos to manage a project that includes a Databricks Asset Bundle (DAB) for deployment. The engineer runs `databricks bundle validate` and receives an error: 'Error: cannot find bundle configuration file'. The engineer confirms that databricks.yml exists in the current working directory. Which action should the engineer take to resolve the error?

Accepted Answer

Ensure that the Databricks CLI is authenticated and the current working directory is the bundle root, then run the command again.. The Databricks CLI searches for databricks.yml in the current working directory and its parents. If the CLI is not authenticated, it may not perform the search correctly, or if the user is in a subdirectory without a bundle root, it fails. Ensuring authentication and correct directory resolves the 'cannot find bundle configuration file' error.

Answer

Add a bundle section to the databricks.yml file with a name and include the path to the configuration.

Answer

Run the command with the --config-file flag pointing to the absolute path of databricks.yml.

Answer

Rename the configuration file to bundle.yml, as the CLI expects that filename by default.

Question 14

A data engineer is troubleshooting a Databricks Workflow where a downstream task relies on an upstream task's output. Which TWO actions ensure the data dependency is correctly handled during a failure scenario?

Accepted Answer

Configure the downstream task to use the 'depends_on' attribute to reference the upstream task ID.. Proper dependency management ensures that downstream tasks do not attempt to process incomplete or missing data from upstream failures. By using task dependencies and repair-and-rerun capabilities, engineers can isolate failures and maintain data integrity. These features are critical in production pipelines to prevent downstream corruption and ensure that data lineage remains consistent throughout the workflow execution cycle regardless of individual component failures.

Answer

Set the task timeout to zero to prevent the workflow from ever stopping during a failure.

Answer

Hardcode the file path of the upstream output into the downstream task configuration.

Answer

Disable all retries to ensure that the error log is captured immediately upon failure.

Question 15

Refer to the exhibit. A data engineer is deploying a production pipeline that references a table in the default schema. The job fails with the provided error. What is the root cause?

Accepted Answer

The table exists only in the temporary session catalog of a different notebook.. The error indicates that the Spark session cannot resolve the table 'default.sales_data'. In Databricks, jobs often run in a different environment or context than an interactive notebook. If the table was created in an interactive session, it might not exist in the environment where the job runs, or the database context is missing. This highlights the importance of using absolute paths or proper schema initialization in production code.

Answer

The cluster is running an outdated version of the Spark runtime.

Answer

The job is running under a service principal that lacks permissions to the Hive Metastore.

Answer

The cluster has insufficient memory to load the table metadata.

Question 16

A data engineer is automating the deployment of Databricks assets using CI/CD. The pipeline fails because the 'databricks-cli' command cannot find the workspace. What is the most likely cause?

Accepted Answer

The runner does not have the DATABRICKS_HOST and DATABRICKS_TOKEN environment variables set.. Deployment failures in CI/CD often stem from authentication or environment variable misconfiguration. The Databricks CLI requires a properly configured profile or environment variables containing the host URL and token. When these are missing, the CLI fails to authenticate or locate the workspace, preventing the deployment of code. Ensuring that the runner has access to these secure variables is foundational for a reliable DevOps lifecycle for Databricks infrastructure.

Answer

The Databricks CLI version on the build agent is incompatible with the server.

Answer

The workspace API is currently disabled for security reasons.

Answer

The CI/CD runner is missing the required library dependencies like PySpark.

Question 17

You are monitoring a long-running Databricks job. You notice that the memory usage on the driver node is steadily increasing until it crashes. Which debugging action is most appropriate?

Accepted Answer

Use the Spark UI to identify tasks using 'collect()' or 'toPandas()' on large datasets.. Driver memory crashes are typically caused by collecting large datasets from workers to the driver. By identifying the 'collect' or 'toPandas' operations, you can optimize the code to process data in a distributed manner. This is critical for scaling data pipelines; understanding why the driver is overloaded ensures that jobs remain performant as data volumes grow and prevents common failures in production batch processing environments.

Answer

Increase the number of worker nodes in the cluster.

Answer

Update the cluster's Spark configuration to disable the driver's log monitoring.

Answer

Lower the 'spark.driver.maxResultSize' setting in the cluster configuration.

Question 18

Which THREE strategies should a data engineer use to optimize the debugging of failed production Databricks Jobs?

Accepted Answer

Implement structured logging within the application code to track variable states.. Efficient debugging relies on observability, modular design, and access control. By leveraging logs, modularizing code, and ensuring proper access, engineers can quickly isolate root causes. These strategies are essential for minimizing Mean Time to Recovery (MTTR) in production. Proactive monitoring and well-structured codebases allow for faster identification of failures, ensuring that business-critical pipelines remain operational and that the team can respond to incidents with precision and speed.

Answer

Grant all developers full cluster permissions to access logs directly on the nodes.

Answer

Run all production jobs as the root user to avoid permission-related errors.

Question 19

Where can a data engineer find the standard output and error logs for a specific task within a Databricks Workflow?

Accepted Answer

By clicking the 'Logs' tab within the specific task run details in the Jobs UI.. Databricks provides a comprehensive UI that aggregates logs for every task execution. By navigating to the job run and selecting the specific task, the engineer can view stdout and stderr logs directly. This is the primary method for diagnosing runtime exceptions, syntax errors, or logic failures in automated data pipelines, facilitating rapid troubleshooting without needing external tools or direct node access.

Answer

In the Databricks Filesystem (DBFS) root directory under /logs.

Answer

By querying the 'sys.logs' table in the Unity Catalog.

Answer

In the cluster configuration's 'Advanced Options' tab.

Question 20

A data engineer is debugging a slow-running query. They notice that the data is skewed, causing one task to take significantly longer than others. Which approach effectively addresses this skew?

Accepted Answer

Add a salt column to the skewed key to distribute the data across more partitions.. Data skew occurs when one partition contains disproportionately more data than others, causing a single executor to bottleneck the entire stage. By using techniques like salt, broadcast joins, or repartitioning, the engineer can distribute the load more evenly across the cluster. This is essential for optimizing performance and preventing timeouts, ensuring that production jobs adhere to defined SLAs and utilize cluster resources efficiently.

Answer

Increase the cluster's disk size to allow for more local shuffle storage.

Answer

Change the file format from Parquet to CSV to reduce overhead.

Answer

Enable 'Auto-scaling' to automatically add more nodes during the skewed stage.