Databricks-DE-Assoc Databricks Intelligence Platform Practice Question
A data engineer needs to configure a Databricks Job to orchestrate a data pipeline that includes a Python task, a SQL task, and a notebook task. The pipeline requires passing a dynamic run identifier from the Python task to the subsequent SQL and notebook tasks. Which mechanism should the data engineer use to achieve this task-to-task dependency parameter passing?
⚠ Common exam trap
Candidates often suggest writing task variables to external storage or global variables, forgetting Databricks provides a native task values API for inter-task communication.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use the dbutils.jobs.taskValues.set method in the Python task and reference it downstream using the task value syntax.
Databricks workflows natively support passing values between tasks using task values. A Python task can set a task value using dbutils.jobs.taskValues.set, which downstream tasks can then reference using standard task value interpolation syntax. This eliminates external storage dependencies, ensuring robust, serverless orchestration directly managed by the Databricks control plane.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Write the dynamic identifier to a shared Unity Catalog volume and read it inside the downstream tasks.
Why it's wrong here
Unity Catalog volumes provide secure file storage, but relying on disk writes introduces unnecessary I/O overhead, potential race conditions, and file cleanup complexities. Databricks workflows provide native task values specifically designed for memory-based variable sharing across pipeline tasks without external file management.
- ✓
Use the dbutils.jobs.taskValues.set method in the Python task and reference it downstream using the task value syntax.
Why this is correct
The dbutils.jobs.taskValues API allows tasks within a Databricks workflow to securely pass small payloads like run identifiers to downstream tasks. Downstream tasks retrieve these values via task value interpolation, enabling seamless dynamic parameter passing and robust pipeline orchestration.
- ✗
Store the identifier as an environment variable in the cluster configuration shared by all three tasks.
Why it's wrong here
Cluster-level environment variables are static and defined at cluster startup. They cannot be dynamically mutated by a running task to pass runtime state to other tasks executing subsequently within the same job run or across different tasks.
- ✗
Modify the global job parameters dynamically via the Databricks REST API from within the running Python script.
Why it's wrong here
Job parameters are resolved when the run starts, so REST API edits during execution do not propagate to downstream tasks already queued with their own parameter values. It tempts as a dynamic workaround, but the supported mechanism is task values, set via dbutils.jobs.taskValues.set and read by dependent tasks.
Quick reference
Cloud Service Model Comparison
| Model | You Manage | Provider Manages | Examples |
|---|---|---|---|
| IaaS | OS, runtime, apps, data | Hardware, hypervisor, networking | EC2, Azure VMs, GCP Compute Engine |
| PaaS | Apps and data | OS, runtime, middleware, hardware | Elastic Beanstalk, Azure App Service |
| SaaS | Data and settings only | Everything else | Microsoft 365, Salesforce, Workday |
| FaaS / Serverless | Function code only | Infra, scaling, runtime | Lambda, Azure Functions, Cloud Run |
| CaaS | Containers and apps | Kubernetes, OS, hardware | EKS, AKS, GKE |
About these practice questions
Courseiva writes every Databricks-DE-Assoc question from scratch — 276 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Assoc exam.