Databricks-DE-Assoc Practice Test 17 — 15 Questions

Question 1

A CI/CD pipeline uses the Databricks CLI to deploy a job to production. The pipeline must ensure that the job configuration is identical across environments except for the cluster size, which differs between staging and production. Which approach best supports this requirement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Define the job in a JSON file with placeholders, and use environment variables in the CI/CD pipeline to substitute cluster size values before deployment.. Templating the job JSON with environment variables allows the pipeline to substitute environment-specific values like cluster size while keeping the rest of the configuration identical. This ensures consistency and reduces drift, as the same template is used for all environments, and differences are explicitly managed through variables.

Answer

Maintain separate JSON files for staging and production, and manually update both when the job logic changes.

Answer

Store the job definition in a Databricks notebook and use widgets to parameterize the cluster size at runtime.

Answer

Use the Databricks REST API to create the job in production, then manually edit the cluster size in the Databricks UI after deployment.

Question 2

Which THREE of the following are core components of the Databricks Intelligence Platform?

Accepted Answer

Delta Lake. The Databricks Intelligence Platform is built on a foundation that includes high-performance compute, unified governance, and open storage. These components work together to provide a seamless experience for data engineering, science, and analytics. Recognizing these elements helps engineers understand the platform's capabilities to scale horizontally, secure data assets centrally, and maintain interoperability through open standards, which are essential for modern data architecture requirements.

Answer

Proprietary File System Format

Answer

Hardware-level Storage Controllers

Question 3

A data engineer notices that a production Delta Lake table is experiencing slow read performance during concurrent write operations. The table contains millions of small files. Which action should the engineer take to resolve this performance degradation?

Accepted Answer

Execute the OPTIMIZE command on the table.. Compacting small files into larger files using the OPTIMIZE command reduces metadata overhead and improves I/O efficiency. This is a critical maintenance task in Databricks because small files cause excessive object storage requests and metadata listing latency. By consolidating these files, the engine can scan data much more effectively during read operations, even while concurrent writes are occurring, as Delta Lake ensures ACID transactions remain consistent and performant.

Answer

Run the VACUUM command on the table.

Answer

Increase the number of worker nodes in the cluster.

Answer

Enable Z-Ordering on all columns in the table.

Question 4

A data engineer is optimizing a Databricks job that performs a join between a large fact table and a small dimension table. The job is slow, and the engineer suspects that the join strategy is not optimal. Which TWO actions should the engineer take to improve performance? (Choose two.)

Accepted Answer

Enable Adaptive Query Execution (AQE) by setting spark.sql.adaptive.enabled to true. Broadcasting the small dimension table avoids shuffling the large fact table, which is a major performance win. Enabling Adaptive Query Execution allows Spark to automatically choose a broadcast join at runtime if the dimension table is small enough, and to optimize other aspects of the query. Together, these actions address the inefficient join strategy and improve performance.

Answer

Cache the large fact table in memory

Answer

Increase spark.sql.shuffle.partitions to a very high value

Answer

Repartition the large fact table by the join key before the join

Question 5

What is the primary benefit of using Data Live Tables (DLT) for managing dependencies between tables in a pipeline?

Accepted Answer

It automatically manages the dependency graph and ensures execution order.. DLT automatically manages the Directed Acyclic Graph (DAG) of the entire data pipeline. By defining table relationships through code, DLT handles the execution order and ensures dependencies are met. This simplifies pipeline management and maintenance, reducing the need for manual orchestration and ensuring that data is always processed in the correct order across the entire Medallion architecture.

Answer

It allows for manual control of the Spark cluster settings for every single step.

Answer

It eliminates the need for any data transformation logic to be written in SQL.

Answer

It forces all data to be stored in the Gold layer for final reporting.

Question 6

When considering the Databricks Intelligence Platform, what is the primary role of the 'Lakehouse' architecture?

Accepted Answer

To unify the best features of data warehouses and data lakes.. The Lakehouse architecture combines the data management, schema support, and ACID transactions of a data warehouse with the scale, flexibility, and low-cost storage of a data lake. This is important for engineers because it eliminates the need to maintain separate systems for BI and AI/ML, allowing teams to use a single platform for all data processing tasks, thereby reducing data duplication and architectural complexity.

Answer

To replace cloud object storage with a proprietary proprietary binary format.

Answer

To isolate streaming and batch data processing into distinct storage layers.

Answer

To mandate the use of SQL as the only supported programming language.

Question 7

A data engineer is investigating a slow-running SQL query. Which TWO metrics from the Spark UI should the engineer examine to identify data skew?

Accepted Answer

Task duration. Data skew occurs when one partition is significantly larger than others, causing one task to take much longer than the rest. By examining task duration and task input size, an engineer can spot outliers where a single executor handles a disproportionate amount of data. Identifying these metrics allows the engineer to apply remediation techniques like salting or repartitioning to balance the workload across the cluster effectively.

Answer

Executor CPU usage

Answer

Shuffle read size

Answer

Driver memory usage

Question 8

A platform team is setting up a CI/CD pipeline that deploys Databricks jobs and notebooks from a Git repository. They must ensure deployments are secure and auditable. (Choose two.)

Accepted Answer

Grant the deployment identity only the permissions required to manage the specific jobs and notebooks it deploys.. Secure, auditable deployments rely on a dedicated non-human identity with narrowly scoped permissions. A service principal whose secret lives in the CI/CD secret store provides traceable, rotatable credentials, and least-privilege grants limit what a compromised pipeline could do while keeping audit records meaningful.

Answer

Disable workspace audit logs during deployments to reduce log volume and improve pipeline speed.

Answer

Use a different engineer's personal account for each environment so responsibility is clear.

Answer

Commit the workspace personal access token to the repository so all pipeline runs use the same credential.

Question 9

A data engineer is configuring a Databricks job that must run on a schedule and send an email notification if the job fails. They want to minimize manual intervention. Which feature should they use to define the schedule and failure notification?

Accepted Answer

Databricks Jobs with a cron schedule and email notifications on failure. Databricks Jobs provide built-in scheduling via cron and support email notifications on failure. This directly meets the need to automate job runs and alert the engineer when something goes wrong, reducing manual intervention. Other features like SQL alerts or cluster policies serve different purposes and cannot replace job scheduling and failure notifications.

Answer

Cluster policies with auto-termination settings

Answer

Databricks SQL alerts with a custom schedule

Answer

Delta Live Tables with expectations for failure handling

Question 10

Which TWO of the following scenarios are valid use cases for utilizing Delta Lake's Change Data Feed (CDF)?

Accepted Answer

Auditing data changes for compliance reporting.. Change Data Feed allows users to track row-level changes (inserts, updates, deletes) in a Delta table. It is essential for incremental processing, auditing, and syncing data between systems. By enabling CDF, engineers can decouple the processing logic from the source table history, simplifying complex pipelines that need to respond to specific data mutations rather than re-scanning entire datasets periodically.

Answer

Performing full table overwrites in a batch job.

Answer

Compressing large files into smaller partitions.

Answer

Querying the table history using the VERSION AS OF syntax.

Question 11

A data engineer is building a Databricks job that processes millions of small JSON files landed in cloud storage each hour. The job currently spends most of its runtime on file listing and task scheduling overhead. The engineer wants to improve throughput without changing the downstream table schema. Which change should be made to the ingestion step?

Accepted Answer

Use Auto Loader with cloudFiles and enable file notification mode to ingest new files incrementally.. The bottleneck is file discovery and task scheduling for many small files, not the target table layout or columnar format. Auto Loader with cloudFiles incrementally discovers new files, and file notification mode avoids full directory listings by using cloud storage event notifications, which scales to high file counts. This preserves the existing schema while improving ingestion throughput.

Answer

Convert the source JSON files to Parquet with a separate Spark batch job before each hourly run.

Answer

Enable Auto Compaction on the target Delta table so small files are merged after each write.

Answer

Repartition the DataFrame to one partition per input file before writing to the target table.

Question 12

Refer to the exhibit. A data engineer is running these commands before performing heavy write operations into a Delta table. What is the primary benefit of enabling these configurations?

Accepted Answer

It mitigates the small file problem and improves read performance.. Enabling 'optimizeWrite' and 'autoCompact' is a common strategy to mitigate the 'small file problem' in Delta Lake. 'OptimizeWrite' rearranges data before writing to ensure consistent, larger file sizes, while 'autoCompact' merges small files post-write. Together, they ensure that the table remains highly performant for read operations by maintaining an optimal file layout, which is critical for reducing metadata overhead and maximizing query scan efficiency in large datasets.

Answer

It increases the number of concurrent write operations allowed on the table.

Answer

It forces the data to be written in Parquet format only.

Answer

It disables the need for periodic VACUUM maintenance.

Question 13

A data engineer is using Structured Streaming to ingest data from a Kafka topic. They want to ensure that if the pipeline fails, it can resume exactly where it left off, without processing duplicate data. Which component enables this functionality?

Accepted Answer

A persistent checkpoint location. Checkpointing is the fundamental mechanism in Spark Structured Streaming that persists state information, including the offset of the processed records, to reliable cloud storage. When a job restarts after a failure, it reads the checkpoint location to determine which records were already successfully processed. This ensures fault tolerance and enables exactly-once processing semantics, which is a requirement for reliable and accurate data engineering pipelines in production environments.

Answer

The Delta transaction log

Answer

An external database to store row IDs

Answer

The cluster's memory state

Question 14

Which TWO of the following statements are true regarding the behavior and capabilities of Databricks Jobs parameters and values?

Accepted Answer

Task values can be used to pass small amounts of data, such as counts or status codes, from one task to a downstream task.. Databricks Jobs support parameterized runs, enabling dynamic workflows where values are passed at runtime or configured statically. Understanding how parameters propagate to tasks and how task values pass outputs between tasks is essential for building modular, reusable data engineering pipelines within the Lakeflow framework.

Answer

Job parameters are automatically persisted in a Unity Catalog volume after the workflow completes successfully.

Answer

Notebook tasks cannot accept parameter values passed from the parent job orchestration configuration.

Answer

Task values support passing large dataframes containing millions of rows between dependent tasks.

Question 15

A data engineer is migrating legacy tables to Unity Catalog. Which TWO of the following statements regarding the transition to three-level namespace (catalog.schema.table) are true?

Accepted Answer

Three-level namespace applies to all Unity Catalog objects.. Unity Catalog introduces a hierarchical namespace that forces a clear separation between data assets. Understanding this structure is critical for governance, as permissions are inherited down from the catalog and schema levels. By adopting this structure, organizations can enforce consistent policies across environments, simplify cross-workspace data sharing, and ensure that all metadata is discoverable through a centralized catalog that replaces the legacy Hive Metastore structure.

Answer

Hive Metastore objects are automatically migrated to Unity Catalog.

Answer

External locations are not required for three-level namespace tables.

Answer

Three-level namespace is optional in Unity Catalog.