Databricks-DE-Pro Practice Test 14 — 15 Questions

Question 1

A data engineer is using Databricks Asset Bundles to deploy a job that runs a Python wheel task. The bundle is deployed to a production workspace using a service principal. The job fails with the error: `Library installation failed for library due to user error: Could not find wheel file`. The engineer confirms the wheel file exists in the bundle's `dist` folder. What is the most likely cause of this failure?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The wheel file is not included in the bundle's artifact definition, so it is not uploaded to the workspace.. For a Python wheel task in a Databricks Asset Bundle, the wheel must be declared as an artifact in the bundle configuration. This ensures it is built and uploaded to the workspace during deployment. Without this, the job cannot locate the wheel, leading to the error. The other options incorrectly attribute the failure to permissions, path resolution, or compatibility.

Answer

The wheel file path in the task definition is relative to the workspace root instead of the bundle root.

Answer

The wheel file is not compatible with the Databricks Runtime version used by the cluster.

Answer

The service principal lacks permission to read from the `dist` folder in the workspace.

Question 2

A data engineer is troubleshooting a Databricks SQL query that occasionally fails with 'Query exceeded the maximum allowed execution time' on a shared SQL warehouse. The query is a complex aggregation over a large Delta table. The engineer needs to identify the root cause and ensure the query can complete successfully. Which action should the engineer take first?

Accepted Answer

Examine the query profile in the Databricks SQL query history to identify stages with high data skew or spill.. The query profile in Databricks SQL provides detailed execution metrics that can reveal the root cause of long-running queries, such as data skew or spill. By analyzing the profile first, the engineer can make targeted optimizations, such as repartitioning or rewriting the query, which may resolve the timeout without unnecessarily increasing compute resources.

Answer

Increase the SQL warehouse size to provide more compute resources for the query.

Answer

Set the Spark configuration 'spark.sql.adaptive.enabled' to false to disable adaptive query execution.

Answer

Change the query to use a larger cluster by switching from a SQL warehouse to an all-purpose cluster.

Question 3

A data engineer is configuring a Unity Catalog external location to allow a service principal to write to an ADLS Gen2 container. The storage credential uses a managed identity. The engineer grants the service principal `WRITE FILES` on the external location. However, when the service principal attempts to write, it fails with a permissions error. The engineer verifies that the managed identity has the Storage Blob Data Contributor role on the container. What is the most likely cause of the failure?

Accepted Answer

The service principal does not have the `USE EXTERNAL LOCATION` privilege on the external location.. Unity Catalog requires both `USE EXTERNAL LOCATION` and the appropriate file-level privileges (`READ FILES`, `WRITE FILES`) to access data in an external location. The `USE EXTERNAL LOCATION` privilege grants the ability to reference the external location in queries and commands, while `WRITE FILES` allows writing data. Without `USE EXTERNAL LOCATION`, the service principal cannot utilize the external location, resulting in a permissions error despite having `WRITE FILES` and cloud storage permissions.

Answer

The service principal lacks the `CREATE EXTERNAL TABLE` privilege on the external location.

Answer

The external location is missing the `READ FILES` privilege for the service principal.

Answer

The managed identity lacks the Storage Blob Data Reader role on the container.

Question 4

A data engineer is debugging a Databricks job that fails with a `SparkException: Job aborted due to stage failure` in production. They need to identify the root cause. Which two actions should they take to gather relevant diagnostic information? (Choose two.)

Accepted Answer

Review the Spark driver logs for the failed job run in the Databricks Jobs UI.. To debug a Spark stage failure, the driver logs provide the exception details, and the Spark UI offers performance metrics that reveal issues like skew or spills. Together, they help identify whether the failure is due to data, code, or resource problems. The other actions are either for different error types or less direct for this specific failure.

Answer

Run the job locally with a small sample of data to reproduce the error.

Answer

Examine the event log for the job cluster to see if nodes were terminated unexpectedly.

Answer

Check the cluster's init script logs to see if a library installation failed.

Question 5

A data engineer needs to grant a group of users the ability to run a specific Databricks job but not modify its configuration. The job is managed by a service principal. Which permission level should be assigned to the group on the job?

Accepted Answer

Can Run. Databricks job permissions include Can View, Can Run, Can Manage, and sometimes Is Owner. Can Run permits users to execute the job and view run results but prevents them from editing the job's configuration. This aligns with the requirement to allow running without modification. Can View is too restrictive, while Can Manage grants excessive privileges. Can Attach To is unrelated to job permissions.

Answer

Can View

Answer

Can Manage

Answer

Can Attach To

Question 6

A data engineer is responsible for monitoring a production Databricks job that runs critical ETL tasks. The job occasionally fails due to transient issues such as cloud storage throttling or network timeouts. The engineer wants to set up automated alerts that notify the team only when the job fails after all retries are exhausted. Which TWO actions should the engineer take to achieve this? (Choose two.)

Accepted Answer

Enable notifications on the job to send an email or webhook when the job fails after retries.. To alert only after all retries are exhausted, the engineer should configure a retry policy on the job and enable job notifications for failure. The retry policy handles transient issues automatically, and the notification fires only on final failure. Other methods either alert on every failure or require custom code to determine final failure status.

Answer

Set up a Databricks SQL alert that queries the job run history and triggers when a run fails with a specific error code.

Answer

Use the Databricks REST API to monitor job runs and trigger an alert via an external system when a run fails.

Answer

Create a scheduled notebook that checks the job status every minute and sends an alert if the status is FAILED.

Question 7

A Data Engineer is implementing column-level security on a Unity Catalog table `sales.customers` that contains `email`, `ssn`, and `region` columns. The requirement is that analysts in the `analyst` group see only the last four digits of `ssn` and a hashed `email`, while members of the `compliance` group see full values. The engineer plans to use column masks. Which TWO actions are required to meet the requirement? (Choose two.)

Accepted Answer

Create a masking function that inspects the invoking user's group membership and returns either the full value or a masked value.. Column masks require two things: a function that decides the returned value based on the invoking user's groups, and the ALTER TABLE statement that binds that function to each sensitive column. Together they let analysts see truncated SSN and hashed email while compliance sees full values. Revoking column SELECT breaks queries, row filters hide rows instead of transforming values, and USE SCHEMA is only a prerequisite.

Answer

Grant the `analyst` group the `USE SCHEMA` privilege on the schema containing `sales.customers`.

Answer

Revoke SELECT on the `ssn` and `email` columns from the `analyst` group before applying the mask.

Answer

Create a row filter function that returns TRUE only for rows where the user belongs to the `compliance` group.

Question 8

A data engineer is setting up a Delta Share to provide a partner with access to a specific table. The partner uses Databricks and wants to query the shared data using their own Databricks workspace. What is the correct sequence of actions for the data engineer to enable this Databricks-to-Databricks sharing?

Accepted Answer

Create a share, add the table, create a recipient of type Databricks, and grant the recipient access to the share.. Databricks-to-Databricks sharing uses Unity Catalog to create a share, add tables, and define a recipient using the partner's Databricks sharing identifier. The recipient is then granted access to the share. The partner mounts the share in their workspace, enabling them to query the data with their own compute. This method avoids credential files and ensures secure, real-time access.

Answer

Export the table to a cloud storage location and provide the partner with the storage credentials.

Answer

Create a share, add the table, generate a credential file, and send it to the partner.

Answer

Grant the partner's Databricks workspace SELECT privileges on the table, and they can query it directly.

Question 9

A data engineer is debugging a Databricks job that reads from a Delta table and writes to another Delta table. The job occasionally fails with 'ConcurrentAppendException'. The engineer wants to minimize failures while maintaining data correctness. Which approach should the engineer take?

Accepted Answer

Use optimistic concurrency control with retry logic and partition the target table to reduce conflicts. ConcurrentAppendException arises when multiple writers append to the same Delta table partition. Delta's optimistic concurrency control detects the conflict. Adding retry logic allows transient conflicts to resolve, and partitioning the target table by a key that separates writers reduces overlapping appends, preserving correctness while minimizing failures.

Answer

Disable Delta Lake transaction logging on the target table

Answer

Increase the job cluster's autoscaling maximum worker count

Answer

Switch the target table to a Parquet table to avoid transaction conflicts

Question 10

A data engineer is configuring a Unity Catalog external location to allow access to an S3 bucket. The security team requires that all access to the bucket be authenticated using a specific IAM role, and that the credentials not be stored in Databricks. Which Unity Catalog object should the engineer create to meet this requirement?

Accepted Answer

A storage credential that assumes the IAM role using AWS STS.. Unity Catalog storage credentials are the correct object for authenticating to cloud storage. They reference an IAM role that Databricks assumes via AWS STS, providing temporary credentials without storing secrets. External locations then reference the storage credential and define the bucket path. This design meets the requirement of using a specific IAM role and avoiding stored credentials.

Answer

A storage credential that references the IAM role via an instance profile.

Answer

A service principal with an attached IAM role in AWS.

Answer

An external location that directly embeds the IAM role's access key and secret key.

Question 11

A data engineer is implementing column-level masking in Unity Catalog. They need to mask the 'email' column in the table 'prod.customers' such that only members of the 'hr_group' see the full email, while all other users see a masked version. The engineer creates a masking function and applies it using ALTER TABLE. Which statement correctly applies the mask?

Accepted Answer

ALTER TABLE prod.customers ALTER COLUMN email SET MASK prod.security.mask_email;. The correct syntax to apply a column mask in Unity Catalog is ALTER TABLE table_name ALTER COLUMN column_name SET MASK function_name, where function_name is a fully qualified function that returns the masked value. This ensures that the mask is applied consistently and only authorized users see the original data.

Answer

ALTER TABLE prod.customers ALTER COLUMN email SET MASK FUNCTION prod.security.mask_email;

Answer

ALTER TABLE prod.customers ALTER COLUMN email SET MASK mask_email;

Answer

ALTER TABLE prod.customers MODIFY COLUMN email SET MASK prod.security.mask_email;

Question 12

A Data Engineer is using Lakehouse Federation to query an external PostgreSQL database from Databricks. The engineer creates a foreign catalog named 'pg_catalog' using a connection that specifies the host, port, and credentials. Users report that queries against the foreign catalog fail with a permission error, even though the connection works when tested. The engineer confirms that the connection has the correct credentials and that the PostgreSQL user has SELECT privileges on the required tables. What is the most likely cause?

Accepted Answer

The users have not been granted USE on the foreign catalog and SELECT on the tables.. In Lakehouse Federation, the connection stores credentials to access the external database, but end users must still have Unity Catalog privileges to query the foreign catalog. Specifically, they need USE on the foreign catalog and SELECT on the tables. Without these, queries fail with permission errors. The connection test succeeds because it uses the stored credentials directly, bypassing user-level Unity Catalog checks. The correct answer addresses the missing Unity Catalog grants.

Answer

The PostgreSQL database is not in the same region as the Databricks workspace.

Answer

The connection uses a service principal instead of a user account.

Answer

The foreign catalog was created without specifying the 'postgresql' database type.

Question 13

A data engineer is setting up Lakehouse Federation to query an external MySQL database from Databricks. The engineer creates a connection using the MySQL connector and a foreign catalog. Users in the 'analysts' group report that they can see the foreign catalog but cannot query any tables. The engineer has granted USAGE on the connection to the 'analysts' group. Which TWO additional permissions must be granted to the 'analysts' group to allow them to query tables in the foreign catalog? (Choose two.)

Accepted Answer

USE CATALOG on the foreign catalog. To query foreign tables in a foreign catalog, users need USE CATALOG on the foreign catalog and USE SCHEMA on the specific schemas. USAGE on the connection only allows the catalog to use the connection; it does not grant access to the catalog's objects. These two privileges are the minimum required for read access.

Answer

BROWSE on the foreign catalog

Answer

CREATE on the foreign catalog

Answer

SELECT on the foreign catalog

Question 14

A data engineer is configuring a Unity Catalog metastore to use a customer-managed key (CMK) for encryption at rest. The engineer has created the necessary Key Vault and key in Azure. Which additional configuration is required to enable CMK for the metastore?

Accepted Answer

Assign the Key Vault Crypto Service Encryption User role to the Databricks managed identity used for the metastore.. Unity Catalog CMK for Azure requires that the Databricks managed identity for the metastore has the Key Vault Crypto Service Encryption User role on the Key Vault. This role allows Databricks to perform wrap and unwrap operations with the key. Without this role assignment, the metastore cannot use the CMK for encryption.

Answer

Create a private endpoint for the Key Vault.

Answer

Grant the Databricks access connector the Key Vault Crypto Service Encryption User role.

Answer

Enable soft delete on the Key Vault.

Question 15

A data engineer needs to grant a service principal permission to read data from a Unity Catalog table named sales.orders. The service principal is used by an automated job and should have only the minimum necessary privileges. Which Unity Catalog privilege should be granted on the table to allow the service principal to read data?

Accepted Answer

SELECT. In Unity Catalog, SELECT is the privilege that grants read access to a table. For a service principal that only needs to read data, granting SELECT on the specific table is the least-privilege approach. Other privileges like MODIFY or ALL PRIVILEGES would provide additional capabilities that are not required and could increase risk.

Answer

ALL PRIVILEGES

Answer

USAGE

Answer

MODIFY