Databricks-DE-Assoc Practice Test 8 — 10 Questions

Question 1

Refer to the exhibit. A data engineer encounters this error while trying to list tables in a schema. Based on the error, what must the engineer do to resolve it?

Accepted Answer

Grant 'USE CATALOG' privilege on the catalog to the user.. The error indicates that the user lacks the 'USE CATALOG' privilege, which is the foundational permission required to even browse objects within a specific catalog in Unity Catalog. This is a critical concept because Unity Catalog enforces a hierarchical security model where access must be granted from the root down to the table level, ensuring that users cannot bypass organizational security boundaries by guessing table names or paths.

Answer

Grant 'SELECT' privilege on the table to the user.

Answer

Increase the cluster's worker node count.

Answer

Upgrade the workspace to the Premium tier.

Question 2

When designing an ingestion strategy for a Delta Lake architecture, which TWO advantages does Delta Lake provide over traditional Parquet tables for incoming data?

Accepted Answer

ACID transactions ensure partial writes do not occur during failures.. Delta Lake enhances the standard Parquet format by adding a transaction log that enables ACID compliance and efficient metadata handling. These features are vital for ingestion as they prevent data corruption and allow for advanced operations like schema evolution and time travel, which are not possible with standard Parquet files on cloud storage.

Answer

Native support for sub-second streaming latency from any source.

Answer

Automatic indexing of all string columns for faster search queries.

Answer

Support for data encryption at rest using proprietary algorithms.

Question 3

A data engineer is evaluating whether to use Auto Loader or the COPY INTO command for a new ingestion pipeline. Which TWO features are unique to Auto Loader compared to COPY INTO?

Accepted Answer

Native support for schema evolution through schemaEvolutionMode.. Auto Loader and COPY INTO both offer idempotent loading, but Auto Loader is built on Structured Streaming. This allows it to provide more advanced features like automatic schema evolution and a notification mode that uses cloud services to detect new files. Understanding these differences is crucial for selecting the right tool based on volume and schema complexity.

Answer

The ability to perform incremental loads by tracking processed files.

Answer

Support for ingesting data from cloud object storage like S3 or ADLS.

Answer

Requirement for a Delta Lake table as the final destination.

Question 4

A data engineer is designing a pipeline on Databricks to process streaming data. Which architectural component acts as the unified storage layer, allowing both batch and streaming workloads to access the same underlying data files in a data lake?

Accepted Answer

Delta Lake. The Databricks Intelligence Platform leverages the Lakehouse architecture, which combines the best elements of data lakes and data warehouses. Delta Lake is the critical component that sits on top of object storage, providing ACID transactions, schema enforcement, and time travel. By using Delta Lake, the platform ensures that data consistency is maintained across concurrent streaming and batch processes, which is fundamental for reliable data engineering at scale.

Answer

Unity Catalog

Answer

Databricks SQL

Answer

Photon Engine

Question 5

When using Auto Loader to ingest data, why is it recommended to provide a 'schemaLocation' rather than manually defining the entire schema in the code?

Accepted Answer

To allow Auto Loader to persist and evolve the schema over time.. Auto Loader uses the schemaLocation to store inferred schemas and track changes over time. This allows for schema inference and evolution, which are critical for handling unpredictable data sources. Without this location, the stream cannot persist the state of the schema, making it impossible to use the automatic evolution features provided by Databricks.

Answer

To improve the performance of the initial file listing process.

Answer

To bypass the need for a checkpoint location for the stream.

Answer

To encrypt the data schema for security and compliance reasons.

Question 6

A data engineer needs to optimize the layout of a massive Delta Lake table that suffers from poor query performance due to a large number of small files and unsorted data records. Which TWO operations should the engineer execute to resolve these performance bottlenecks?

Accepted Answer

Execute the OPTIMIZE command to compact small files into larger files.. Optimizing a Delta table requires addressing both small file consolidation and multidimensional physical organization. Running OPTIMIZE compacts small files into larger, uniform file sizes, while combining it with Liquid Clustering or Z-Ordering organizes data records physically to maximize data skipping during filter queries. Together, these maintenance operations significantly reduce read input/output overhead across cloud object storage.

Answer

Execute the VACUUM command with a retention period of zero hours.

Answer

Run REPARTITION on the Delta table using the Spark DataFrame API directly.

Answer

Enable predictive optimization on the Databricks Unity Catalog managed table.

Question 7

A data engineer is working on a Delta Lake table that has accumulated millions of small files due to frequent streaming updates. This fragmentation has significantly degraded query performance. Which operation should the engineer execute to optimize file layout without altering table data?

Accepted Answer

Invoke the OPTIMIZE command to compact small parquet files into larger files using bin-packing.. Running the OPTIMIZE command on a Delta table compacts many small files into larger, optimized files using the bin-packing algorithm. This directly addresses fragmentation issues caused by frequent streaming micro-batches, improving read efficiency and scan speeds. Data engineers frequently schedule optimization jobs as part of bronze-to-silver and silver-to-gold maintenance pipelines in production environments to maintain optimal query performance and reduce cloud storage request latency.

Answer

Execute REFRESH TABLE to update the file metadata cache across all cluster worker nodes.

Answer

Run the VACUUM command with a retention threshold of zero hours to purge all historical data files.

Answer

Execute ALTER TABLE SET TBLPROPERTIES to enable automatic file compaction on every write transaction.

Question 8

An engineer needs to identify the root cause of a job failure. Which THREE of the following are valid locations or methods to investigate the logs?

Accepted Answer

Cluster event logs. Troubleshooting in Databricks requires access to various log levels. The cluster event log captures infrastructure-level changes, driver logs provide application execution details, and executor logs offer task-specific debugging info. Using these three sources provides a holistic view, covering everything from cluster startup issues to specific code-level exceptions. This multi-layered approach is essential for isolating whether a problem is infrastructure-related, configuration-based, or rooted in the logic of the transformation code itself.

Answer

Notebook workspace browser history

Answer

Databricks account profile settings

Question 9

Refer to the exhibit. A data engineer is attempting to run a VACUUM command on a table, but the command fails with an error indicating that the retention period is too short. Given the configuration, what is the most appropriate action the engineer should take to safely remove files older than 7 days?

Accepted Answer

Execute VACUUM table_name RETAIN 168 HOURS.. The configuration restricts the vacuuming of files to ensure Time Travel remains available. Databricks prevents setting a retention period shorter than the log retention to avoid data loss. To safely vacuum, the engineer must temporarily adjust the configuration or ensure the retention period respects the safety threshold. Understanding these safety constraints is critical for data lifecycle management and cost control in cloud storage environments.

Answer

Set 'spark.databricks.delta.retentionDurationCheck.enabled' to false.

Answer

Delete the underlying parquet files directly from cloud storage.

Answer

Run the OPTIMIZE command followed by REFRESH TABLE.

Question 10

A data engineer needs to share a table in Unity Catalog with external partners who do not have access to the Databricks workspace. Which feature should the data engineer configure?

Accepted Answer

Create a Delta Sharing recipient and publish tables to a sharing share object.. Delta Sharing enables secure data sharing with external organizations without requiring recipients to have access to your Databricks workspace or even be on the same cloud platform. It leverages cloud object storage credentials and open protocols to ensure governed data exchange.

Answer

Configure cross-workspace table linking using standard Spark mount points.

Answer

Create a shared Unity Catalog metastore and provision external IAM database users.

Answer

Export the Delta table files to CSV format and upload them to a public cloud bucket.