Databricks-DE-Pro Practice Test 15 — 15 Questions

Question 1

A data engineer at a healthcare company needs to share a Delta table containing patient records with an external research partner. The partner uses a non-Databricks platform and requires read-only access to the latest data, with updates reflected in near real-time. The engineer creates a Delta Share and adds the table. Which additional step is required to allow the partner to access the shared data?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Create a recipient object and provide the recipient with the activation link or credential file.. For a non-Databricks recipient to access a Delta Share, the provider must create a recipient object in Unity Catalog. This generates an activation link or credential file that the recipient uses to authenticate. The share must contain the table, and the recipient is granted access to the share. This process ensures secure, read-only access without requiring the recipient to have a Databricks account.

Answer

Configure a Databricks SQL warehouse and share its connection string with the recipient.

Answer

Grant the recipient's user account SELECT privileges on the shared table.

Answer

Generate a bearer token for the recipient and provide it along with the share name.

Question 2

A data engineer is preparing to deploy a production Databricks Workflow that must be maintainable and auditable. The engineer wants to ensure that changes to the workflow are tracked and that failures can be diagnosed quickly. Which TWO practices should the engineer implement? (Choose two.)

Accepted Answer

Store the workflow definition in a Git repository and deploy using Databricks Asset Bundles. Version-controlling workflow definitions in Git and deploying with Databricks Asset Bundles ensures changes are tracked and reproducible. Delivering job logs to cloud storage retains diagnostic data for troubleshooting and auditing. Together, these practices provide the maintainability and auditability required for production workflows.

Answer

Manually document each deployment in a shared spreadsheet

Answer

Use an all-purpose cluster instead of a job cluster for the workflow

Answer

Configure email notifications for job success and failure

Question 3

A data engineer maintains a Delta Lake table that stores 5 years of order data. Analysts frequently query the most recent 90 days, but compliance requires that older data remain queryable. The table is currently partitioned by order_date and has 200,000 small files because data arrives continuously via Structured Streaming. Queries on the last 90 days are slow and expensive. Which combination of actions will most effectively reduce query cost and improve performance for the recent-data queries?

Accepted Answer

Run OPTIMIZE on the partitions covering the last 90 days to compact small files, and configure the streaming writer with a longer trigger interval and optimized writes to produce larger files.. The dominant problem is the 200,000 small files created by continuous streaming ingestion. Queries on the last 90 days must open many tiny files, which drives up overhead and cost. Compacting only the hot partitions with OPTIMIZE reduces file count where it matters most, and adjusting the streaming writer to produce larger files prevents the problem from recurring. Z-ORDER on a partition key that is already the partition column adds little value, so focusing on compaction and ingestion file sizing is the correct optimization.

Answer

Run OPTIMIZE with Z-ORDER BY order_date on the entire table and set delta.autoOptimize.optimizeWrite to true.

Answer

Enable Delta Lake column mapping and rewrite the table with generated columns for order month.

Answer

Run OPTIMIZE on partitions covering the last 90 days, then use Z-ORDER BY order_date on those partitions, and enable optimized writes for the streaming ingestion.

Question 4

A retail company is designing a Gold-layer dimension table in Delta Lake for its product catalog. The catalog changes slowly: a product's category is occasionally reclassified, but historical sales fact rows must continue to reflect the category that was valid at the time of each sale. The team wants to avoid duplicating the entire product row for every change. Which Delta Lake modeling technique should the engineer implement?

Accepted Answer

Type 2 slowly changing dimension with effective-date and current-flag columns populated via MERGE INTO. Preserving point-in-time accuracy for a slowly changing attribute requires versioned dimension rows with validity ranges and a current flag, which is the defining behavior of a Type 2 slowly changing dimension. A MERGE INTO statement can close the existing row and insert a new one atomically, letting fact rows resolve the correct historical category through an effective-date join or surrogate key.

Answer

A junk dimension that collapses category combinations into a separate table keyed by a surrogate

Answer

Type 1 slowly changing dimension implemented with MERGE INTO that overwrites the category column in place

Answer

A Type 3 dimension that adds a previous_category column to the existing product row

Question 5

A Data Engineer is using Lakehouse Federation to query a Snowflake database from Databricks. The engineer has created a connection and a foreign catalog. Which statement correctly describes how data is accessed when a user queries a table in the foreign catalog?

Accepted Answer

The query is executed on Snowflake, and only the results are returned to Databricks.. Lakehouse Federation enables querying external databases without moving data. When a user queries a foreign catalog table, Databricks pushes the query down to the external database, such as Snowflake. The external database executes the query and returns only the results. This approach minimizes data transfer and leverages the external system's compute. The foreign catalog provides a seamless experience, but the data remains in the source system.

Answer

The data is cached in the Databricks workspace after the first query.

Answer

The data is copied into Delta Lake before the query runs.

Answer

The query is executed on Databricks, and the data is streamed from Snowflake.

Question 6

A financial institution uses a Databricks Lakehouse with a Silver table transactions that is partitioned by transaction_date. The table is frequently queried with filters on transaction_date and account_id. The data engineering team notices that queries filtering on account_id are slow because they scan all partitions. They want to optimize the table to accelerate these queries without repartitioning. Which Delta Lake feature should they use?

Accepted Answer

Z-ORDER BY account_id. Z-ORDER BY account_id clusters data within each partition by account_id, allowing Delta Lake to skip files that do not contain the filtered account_id values. This accelerates queries filtering on account_id without altering the existing partitioning by transaction_date. Other options either do not improve performance or introduce negative side effects.

Answer

Partition the table by account_id as well.

Answer

Convert the table to a Parquet table and use predicate pushdown.

Answer

Enable change data feed on the table.

Question 7

A data engineer is building a Silver layer table that combines data from multiple Bronze tables. The engineer wants to ensure that the Silver table only contains the most recent version of each record based on a 'last_updated' timestamp. Which Delta Lake operation should be used to achieve this?

Accepted Answer

MERGE INTO with a condition that updates when the source timestamp is greater. MERGE INTO is the correct operation because it can conditionally update existing rows with newer timestamps and insert new rows, ensuring the Silver table contains only the latest version of each record. INSERT OVERWRITE, DELETE+INSERT, and time travel do not provide the same atomic upsert capability and would either be inefficient or incorrect for this scenario.

Answer

DELETE then INSERT the new records

Answer

Use Delta Lake time travel to revert to a previous version

Answer

INSERT OVERWRITE with the entire dataset

Question 8

A data engineering team is running a nightly batch job on a Databricks job cluster that processes a 10 TB Delta table. The job reads the entire table, performs transformations, and writes results to another Delta table. The team notices that the job takes 4 hours and consumes significant DBUs. They want to reduce runtime and cost without changing the business logic. The table is partitioned by ingestion date, but queries often filter on a high-cardinality column 'customer_id'. Which optimization technique is most appropriate to improve performance and reduce cost?

Accepted Answer

Optimize file layout using Z-ORDER BY on the 'customer_id' column.. Z-ORDER BY on 'customer_id' colocates related data, enabling Delta Lake's data skipping to prune files during reads. This reduces I/O and compute, directly lowering runtime and DBU cost. Other options either increase cost, are impractical, or degrade performance. The optimization is persistent and does not require changes to business logic, making it ideal for this scenario.

Answer

Enable Delta Lake caching by running CACHE TABLE on the source table before the job.

Answer

Increase the cluster size by adding more worker nodes to the job cluster.

Answer

Convert the table to a Parquet table and use partition pruning on 'customer_id'.

Question 9

A data engineer is configuring a Databricks SQL warehouse to handle a workload that consists of many concurrent short queries during business hours and almost no queries at night. The engineer wants to minimize cost while ensuring low latency during peak hours. Which configuration should the engineer use?

Accepted Answer

Use a serverless SQL warehouse with auto-stop set to 10 minutes and auto-scale enabled.. A serverless SQL warehouse with auto-stop and auto-scaling provides the best balance: it scales up for concurrency during peak hours, scales down and stops when idle, and charges only for usage. The short auto-stop minimizes idle cost, while auto-scaling ensures low latency. Other options either incur idle cost or cannot handle variable concurrency efficiently.

Answer

Use a serverless SQL warehouse with auto-stop disabled to avoid cold starts.

Answer

Use a classic SQL warehouse with auto-scaling enabled and auto-stop set to 5 minutes.

Answer

Use a classic SQL warehouse with a fixed cluster size and auto-stop set to 60 minutes.

Question 10

A data engineer is managing a Delta Share that includes a table with customer transactions. The share is used by multiple recipients. The engineer needs to update the shared data daily with new transactions and also remove data for customers who have requested deletion (right to be forgotten). The engineer wants to ensure recipients see the updated data without having to recreate the share. What is the best approach?

Accepted Answer

Update the underlying Delta table with new data and use DELETE to remove records; recipients will see changes on their next query.. Delta Sharing shares the current state of the Delta table. When the provider performs updates or deletes on the table, recipients see the changes on their next query. There is no need to recreate the share or use materialized views. This approach ensures recipients always access the latest data, including deletions for compliance.

Answer

Use a materialized view to capture changes and share the view instead of the table.

Answer

Use Delta Sharing's built-in history sharing to automatically propagate updates and deletions.

Answer

Recreate the share with the updated table and notify recipients to update their credentials.

Question 11

A data engineer is using Lakehouse Federation to query data from an external PostgreSQL database. The engineer has created a foreign catalog and foreign schema. Which statement accurately describes how the data is accessed during a query?

Accepted Answer

The query is translated into PostgreSQL SQL, executed on the external database, and the results are streamed back to Databricks for further processing.. Lakehouse Federation enables querying external databases without copying data. It uses query pushdown to translate and execute parts of the query on the external system, then streams results back to Databricks for any remaining processing. This approach minimizes data movement and leverages the external database's compute. The other options incorrectly describe data copying or full remote execution.

Answer

The external data is cached in the Databricks workspace's local storage for the duration of the query.

Answer

The query is executed entirely within the external PostgreSQL database, and only the results are returned to Databricks.

Answer

The data is copied into Delta Lake tables in Unity Catalog before the query runs.

Question 12

A data engineer is managing a Delta Share that includes a table with frequent updates. The engineer wants to ensure that recipients always see the latest version of the data without manual intervention. Which statement is correct regarding how Delta Sharing handles updates to shared tables?

Accepted Answer

Updates are automatically propagated to recipients, and they see the latest data on their next query.. Delta Sharing provides live access to the shared table's current state. When the provider updates the table, recipients automatically see the changes on their next query without any manual refresh or CDC application. This is because the Delta Sharing server reads the latest Delta transaction log and serves the current snapshot. The other options incorrectly suggest manual steps or default to stale data.

Answer

Recipients must manually refresh their local copy of the shared table to see updates.

Answer

The shared table is versioned, and recipients must specify a version to query; otherwise, they see the initial snapshot.

Answer

Recipients receive change data capture (CDC) streams and must apply them to a local table.

Question 13

A data engineer wants to share a Delta table with an external partner who does not have a Databricks account. The partner needs to access the data using Python. Which method should the engineer recommend to the partner for accessing the shared data?

Accepted Answer

Install the delta-sharing Python library and use the provided credential file to read the shared table.. For recipients without a Databricks account, the delta-sharing Python library is the standard way to access shared data. It uses the credential file to authenticate and allows reading the shared table into Python. This method is secure, supports live data, and does not require Databricks infrastructure on the recipient side. Other methods either require Databricks access or are not part of the Delta Sharing protocol.

Answer

Mount the shared table as an external table in their local Spark cluster using the Delta Sharing connector.

Answer

Use the Databricks SQL Connector for Python with the partner's Databricks personal access token.

Answer

Download the shared table as a Parquet file from the Databricks workspace and load it into their Python environment.

Question 14

A data engineer is using Delta Sharing to share a table with an external partner. The partner needs to access the shared data using their own Databricks workspace. Which protocol does Delta Sharing use to enable this cross-platform sharing?

Accepted Answer

REST API with pre-signed URLs. Delta Sharing is an open protocol that uses a REST API to facilitate data sharing. The provider's Delta Sharing server authenticates requests and returns pre-signed URLs that allow the recipient to download data directly from cloud storage. This design enables secure, cross-platform sharing without exposing the provider's storage credentials or requiring the recipient to use Databricks.

Answer

SFTP transfer of Parquet files

Answer

JDBC/ODBC connection to the provider's SQL warehouse

Answer

Direct access to the provider's cloud storage with IAM roles

Question 15

A data engineer is using Lakehouse Federation to query an external PostgreSQL database. The engineer creates a connection with the PostgreSQL JDBC URL and credentials, and then creates a foreign catalog. Users report that queries against foreign tables are slow and sometimes fail with connection timeouts. The engineer checks the connection and confirms the credentials are correct. What is the most likely cause of the performance and timeout issues?

Accepted Answer

The Databricks cluster lacks the necessary network connectivity to the PostgreSQL database, causing timeouts.. Lakehouse Federation queries run on Databricks compute, which must connect to the external database over the network. If the cluster cannot reach the database due to network restrictions, queries will be slow or time out. Ensuring proper network connectivity, such as VPC peering or firewall rules, is essential for reliable federation.

Answer

The foreign catalog is not configured with a read-only mode, causing write attempts that time out.

Answer

The foreign catalog must be refreshed to update statistics, causing slow queries.

Answer

The PostgreSQL database is not configured with the correct Unity Catalog connection parameters.