Databricks-DE-Assoc Implementing CI/CD — 25 Questions

Question 1

Refer to the exhibit. The JSON represents a Databricks Job configuration. To implement this as part of a CI/CD pipeline, which approach ensures the job is created with the exact specifications provided?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the Databricks CLI 'databricks jobs create --json-file job.json' command.. The command 'databricks jobs create --json-file job.json' is deprecated in the current Databricks CLI (v0.200+). The modern syntax is 'databricks jobs create --json @job.json'.

Answer

Manually import the JSON into the Databricks Job UI for every release.

Answer

Upload the JSON to a DBFS location and trigger it using a Python script.

Answer

Hardcode the job configuration inside a notebook and run it via the job.

Question 2

A data engineering team uses Databricks Repos linked to a remote Git repository. They want to run automated unit tests against the notebook code in a CI pipeline without deploying to a production workspace. Which approach should they use?

Accepted Answer

Use the Databricks CLI to export the notebooks as .py files and run pytest locally in the CI runner.. Unit tests should run in the CI environment without requiring a Databricks workspace or production resources. Because Databricks Repos syncs notebooks as source files, the CI runner can clone the repository and execute tests locally. This provides fast, isolated feedback and prevents defective code from being merged or deployed.

Answer

Use the Databricks REST API to execute the notebooks in a temporary workspace and collect the exit status.

Answer

Use Databricks Jobs to run the notebooks on a production cluster and inspect the job run output for test results.

Answer

Use Delta Live Tables expectations to validate the notebook code before merging.

Question 3

A team is implementing a CI/CD pipeline for Databricks using Databricks Asset Bundles (DABs). They want to ensure that the deployment is idempotent and can be rolled back if needed. Which TWO practices should they follow? (Choose two.)

Accepted Answer

Store the bundle configuration in a Git repository and use version tags for releases.. Storing the bundle configuration in Git with version tags ensures that each deployment is tied to a specific, reproducible version, enabling rollback. Running databricks bundle validate before deployment catches errors early, preventing failed deployments and ensuring that the deployment is applied correctly. Together, these practices support idempotent and rollback-capable CI/CD.

Answer

Use the databricks bundle deploy command with the --force flag to overwrite existing resources.

Answer

Define the bundle's resources with unique names that include the environment (e.g., dev, prod).

Answer

Manually update the Databricks workspace after deployment to fix any issues.

Question 4

A team is implementing CI/CD for a Databricks job that uses a Python wheel task. They want to ensure that the wheel is built and deployed correctly. Which TWO practices should they follow? (Choose two.)

Accepted Answer

Use the Databricks CLI to upload the wheel to DBFS and update the job definition to point to the new wheel version.. For a Python wheel task, the wheel must be built in CI and made available to the Databricks job. The standard practice is to build the wheel, upload it to a persistent location like DBFS or cloud storage using the Databricks CLI, and then update the job definition to reference the new wheel. This ensures the job uses the correct version and supports repeatable deployments.

Answer

Embed the wheel file inside the notebook and use %pip install to install it at runtime.

Answer

Install the wheel directly on the cluster using an init script that downloads it from the CI artifact repository.

Answer

Use Databricks Repos to sync the wheel source code and rely on automatic building on the cluster.

Question 5

A data engineer runs `databricks bundle validate` locally and it succeeds, but the CI pipeline fails at the `databricks bundle deploy -t prod` step with an error that the target 'prod' is not defined. The engineer is certain the target exists in the bundle. What is the most likely cause?

Accepted Answer

The CI pipeline is running from a different working directory, so the CLI cannot locate the bundle root containing databricks.yml.. The bundle CLI resolves databricks.yml relative to the current working directory (or an explicit path). When CI checks out the repo and runs from the repository root but the bundle lives in a subdirectory, the CLI reads no bundle file and therefore cannot find any target. Running the deploy step from the bundle root, or using the CLI's bundle path option, resolves the failure.

Answer

The 'prod' target is defined under `targets:` but the CI pipeline passes `--target prod` with a trailing whitespace that the CLI treats as a different name.

Answer

The Databricks CLI version in CI is older than the one used locally, and older versions do not support named targets.

Answer

The 'prod' target is missing the `workspace.host` field, so the CLI silently drops it during parsing.

Question 6

A team uses Databricks Asset Bundles with a CI/CD pipeline that runs `databricks bundle deploy -t prod`. They want the pipeline to fail fast if the deployed job's cluster configuration diverges from what is defined in the bundle, before any scheduled run executes. Which approach best enforces this?

Accepted Answer

Add a pipeline step that runs `databricks bundle summary -t prod` and diffs the output against the bundle's expected job definition.. Bundle summary reports the resolved resources and their workspace state, making it the natural way to detect drift before a scheduled run. Running it in CI and comparing against the expected bundle configuration catches manual edits that would otherwise silently diverge from source control. Validation alone only checks the bundle definition, not the deployed state.

Answer

Enable job-level change auditing in the workspace and have the pipeline read the audit log for configuration changes.

Answer

Configure the job's cluster to use a fixed cluster ID instead of a cluster policy, so the bundle cannot override it.

Answer

Add a pipeline step that runs `databricks bundle validate -t prod` and fails on non-zero exit codes.

Question 7

A data engineering team wants to implement Git integration for their Databricks notebooks. Which workflow is considered the best practice for CI/CD in Databricks Repos?

Accepted Answer

Develop code in feature branches, merge via pull requests, and use Databricks Repos to sync production.. Integrating Databricks Repos with a Git provider like GitHub enables version control at the notebook level. This workflow allows teams to use feature branches for development, submit pull requests for code review, and merge into a main branch that triggers automated deployments. Adopting this standardizes the development lifecycle, ensures code traceability, and prevents manual, error-prone deployments in production environments, which is essential for maintaining production-grade data pipelines.

Answer

Export notebooks manually to local machines and commit them via Git CLI.

Answer

Use the Databricks REST API to push code updates directly to production notebooks.

Answer

Develop code in the production workspace and use Git only for final snapshots.

Question 8

A team is designing a CI/CD pipeline using Databricks Asset Bundles (DABs). Which TWO of the following are primary benefits of using DABs for managing Databricks projects?

Accepted Answer

DABs enable consistent deployment of jobs and pipelines across different environments.. Databricks Asset Bundles represent the modern standard for orchestrating Databricks projects as code. By codifying infrastructure and application configuration into YAML, teams achieve environment parity across development, staging, and production. This approach reduces manual configuration errors, simplifies deployment automation via CLI-based workflows, and provides a version-controlled foundation for managing complex data assets, which is critical for large-scale enterprise data engineering environments.

Answer

DABs automatically provision cloud-native IAM roles for all users.

Answer

DABs provide built-in visual drag-and-drop tools for building ETL pipelines.

Answer

DABs bypass the need for any authentication tokens during deployment.

Question 9

Which of the following is a fundamental principle of implementing effective CI/CD for Databricks workflows?

Accepted Answer

Using environment variables to manage service endpoints and authentication secrets.. Effective CI/CD requires separating code logic from environment-specific configurations. By parameterizing job settings and using environment-specific variables, teams ensure that the same code can be deployed safely across development, testing, and production. This practice minimizes errors during promotion and allows for automated testing cycles, which are essential for maintaining a robust, stable, and scalable data platform in any enterprise Databricks environment.

Answer

Deploying code from a local laptop directly to the production cluster.

Answer

Merging all development code into the main branch every few months.

Answer

Ignoring unit testing until the code is fully deployed in production.

Question 10

Why is it important to use Service Principals instead of Personal Access Tokens (PATs) for CI/CD automation in Databricks?

Accepted Answer

Service Principals are not tied to an individual's lifecycle.. Service Principals provide a secure and stable identity for automated systems, independent of individual user accounts. Because PATs are tied to a specific user, they become invalid when that user leaves the company or their access is revoked. Service Principals ensure continuous operation of CI/CD pipelines, simplify security auditing, and follow the principle of least privilege, which is crucial for enterprise-grade automation and governance.

Answer

Service Principals are faster to execute than personal tokens.

Answer

Service Principals do not require any permission configuration.

Answer

Service Principals allow anyone in the organization to run pipelines.

Question 11

A team is implementing a CI/CD process for their Delta Live Tables (DLT) pipelines. Which THREE of the following practices are recommended to ensure reliable deployment?

Accepted Answer

Use Git branches to manage feature development and production code.. Reliable DLT deployments depend on version control, automated testing, and environment abstraction. By defining DLT pipelines as code via DABs or Terraform, teams ensure consistent environments. Testing code before deployment prevents runtime errors, and using Git-based workflows allows for code reviews, which are essential for maintaining high-quality pipelines. These practices collectively ensure that DLT pipeline changes are predictable, verifiable, and safe to deploy into production environments without disrupting existing operations.

Answer

Keep all DLT configuration values hardcoded within the pipeline notebook.

Answer

Manually update the pipeline source code in the production workspace.

Question 12

When promoting code from a development workspace to a production workspace, what is the primary risk of using manual notebook exports?

Accepted Answer

The process creates an inconsistent state between development and production.. Manual notebook exports lack version integrity and metadata synchronization, leading to 'code drift' where the production code doesn't match the source of truth in Git. This manual process is prone to human error, such as importing the wrong version or missing library dependencies. Standardizing on Git-integrated workflows or Databricks Asset Bundles mitigates this risk by automating the deployment and ensuring that exactly what was tested in development reaches production.

Answer

The notebooks will run significantly slower in the production environment.

Answer

The Databricks workspace will automatically delete the old notebooks.

Answer

Manual exports are prohibited by Databricks security policies.

Question 13

Which component of Databricks CI/CD is responsible for executing automated tests on code before it is merged into the main branch?

Accepted Answer

The CI/CD runner (e.g., GitHub Actions, Jenkins).. CI/CD runners or build servers (like GitHub Actions, GitLab CI, or Jenkins) are the components responsible for triggering automated testing. By running tests in an isolated environment during the pull request process, these tools ensure that only validated code is merged. This 'fail-fast' approach is critical for maintaining code quality, reducing regression risks, and providing developers with immediate feedback on their changes before they ever impact the production pipeline.

Answer

The Databricks Delta Lake storage layer.

Answer

The Unity Catalog governance framework.

Answer

The Databricks SQL Warehouse.

Question 14

Refer to the exhibit. A CI/CD pipeline fails with the provided error. What is the most likely cause?

Accepted Answer

The Service Principal lacks sufficient permissions for the workspace path.. The 403 Forbidden error indicates that the identity running the command lacks the necessary permissions to perform the action. In a CI/CD context, this usually happens because the Service Principal lacks 'Can Manage' or 'Can Edit' permissions on the target Repos folder. Ensuring the Service Principal has the correct workspace-level permissions is a standard requirement for successful automated deployments, preventing access issues during the sync process.

Answer

The branch 'main' does not exist in the remote repository.

Answer

The Databricks CLI is not installed on the runner.

Answer

The Git repository URL is invalid.

Question 15

Why should developers avoid using 'notebook' references in production pipelines that point to the 'Shared' folder for shared development work?

Accepted Answer

It lacks the version control and access isolation required for production.. The 'Shared' folder is meant for collaborative development, not for hosting production-ready code. Pipelines referencing this folder are susceptible to unpredictable changes, as any developer with access can modify the code. In CI/CD, production code should reside in a controlled, versioned, and immutable location. Using the 'Shared' folder for production pipelines violates the separation of concerns and increases the risk of unauthorized or accidental changes breaking critical data workflows.

Answer

Databricks jobs cannot execute notebooks from the 'Shared' directory.

Answer

The 'Shared' directory is deleted automatically every 24 hours.

Answer

Notebooks in the 'Shared' directory are limited to a smaller file size.

Question 16

What is the primary goal of implementing 'environment parity' in a Databricks CI/CD pipeline?

Accepted Answer

To minimize failures caused by differences between testing and production.. Environment parity ensures that the development, staging, and production environments are identical in configuration, libraries, and settings. This eliminates 'environment drift,' where code fails in production due to subtle discrepancies. Achieving parity through Infrastructure-as-Code (IaC) is the cornerstone of reliable CI/CD, as it guarantees that testing in staging provides a true representation of how the code will behave in production, significantly reducing the frequency of deployment-related failures.

Answer

To ensure every developer has full admin access to production.

Answer

To reduce the cost of compute resources in development workspaces.

Answer

To allow for manual code deployments in all environments.

Question 17

Which of the following is a recommended strategy for managing library dependencies in a CI/CD pipeline for Databricks?

Accepted Answer

Include a requirements.txt file and install libraries as part of the job definition.. Managing libraries through a centralized repository or a requirements.txt file ensures that the exact same versions are installed consistently across all environments. This avoids the 'dependency hell' where code fails because of incompatible library versions. By integrating dependency management into the CI/CD build process, teams can audit used versions, ensure security compliance, and guarantee that the production pipeline runs with the exact dependencies that were tested during the development phase.

Answer

Install libraries manually on the cluster via the Databricks UI.

Answer

Use the latest version of all libraries to ensure best performance.

Answer

Store library binary files directly in the Git repository.

Question 18

What is the primary purpose of a 'feature branch' in a Git-based workflow for Databricks?

Accepted Answer

To isolate changes so they can be tested before merging to production.. Feature branches allow developers to build and test new functionality in isolation, without affecting the main production code. This workflow is essential for CI/CD because it facilitates peer reviews, automated integration testing, and safe integration. By keeping the main branch stable and only merging validated features, teams maintain a high standard of quality and avoid breaking the production environment with incomplete or untested code changes, which is a core benefit of modern DevOps practices.

Answer

To allow multiple users to edit the same file simultaneously without sync.

Answer

To bypass the authentication requirements for the workspace.

Answer

To automatically deploy changes to the production environment.

Question 19

A data engineering manager wants to ensure that all production code in Databricks is fully audited and versioned. Which TWO of the following steps are required?

Accepted Answer

Enforce a policy where only the CI/CD Service Principal can update production assets.. Ensuring auditability and versioning requires locking down the production environment and forcing all changes through a Git-based CI/CD pipeline. By disabling manual changes in production and requiring all updates to be merged through a pull request process, teams ensure that every modification is reviewed, logged, and linked to a specific version. These controls are essential for compliance, stability, and maintaining a clear history of system changes in professional data environments.

Answer

Grant developers 'Can Manage' permissions on production workspace folders.

Answer

Store all notebook development directly in the production workspace.

Answer

Allow administrators to use personal tokens for manual emergency hotfixes.

Question 20

When utilizing Databricks Asset Bundles, how should secrets (e.g., API keys, database credentials) be handled to ensure security during CI/CD?

Accepted Answer

Use the bundle's secret reference syntax to map variables to a secret scope.. Hardcoding secrets in configuration files is a critical security vulnerability. Databricks Asset Bundles support referencing secrets through the 'secrets' scope, which securely retrieves credentials at runtime. By decoupling sensitive data from the version-controlled YAML files, teams maintain security, avoid leaking credentials in Git, and simplify secret rotation. This practice is essential for enterprise security and ensures that sensitive infrastructure data is managed in alignment with the principle of least privilege.

Answer

Hardcode the secrets in the bundle's YAML configuration files.

Answer

Store secrets as environment variables in the Git repository.

Answer

Ask all developers to manually enter secrets in the production UI.