Databricks-DE-Pro Practice Test 9 — 10 Questions

Question 1

Which THREE techniques are recommended for improving the performance of Spark SQL joins on large Databricks tables?

Accepted Answer

Broadcast the smaller table in a join to avoid shuffling large datasets.. Optimizing joins is critical to performance as they are often the most resource-intensive operations in Spark. Techniques like broadcasting small tables, using bucketing to avoid shuffles, and ensuring data is properly partitioned help the Spark engine execute joins efficiently. By minimizing the amount of data moved across the network (shuffling) and maximizing local processing, jobs complete faster and consume fewer compute resources, leading to a more performant and cost-effective overall data architecture.

Answer

Increase the number of partitions to the maximum possible value to ensure maximum parallelism.

Answer

Use Cross Join for every join operation to ensure no data rows are missed.

Question 2

A Databricks SQL warehouse is experiencing high costs due to idle resources. Which TWO configurations should be implemented to effectively manage and reduce warehouse costs?

Accepted Answer

Set the auto-stop duration to a very low value, such as 1 minute, for serverless SQL warehouses.. Auto-stop and serverless scaling are the primary levers for cost control in Databricks SQL. Auto-stop terminates warehouses when no queries are running, preventing billing for idle time. Serverless SQL warehouses provide faster startup times and more granular scaling, allowing users to configure aggressive auto-stop durations without compromising user experience. Together, these configurations ensure that compute resources are only consumed when active query processing is required, directly minimizing unnecessary cloud spending.

Answer

Enable multi-cluster load balancing to ensure all queries are executed on the largest instance type.

Answer

Configure SQL warehouse scaling to use the maximum cluster size at all times to avoid resizing overhead.

Answer

Disable the query cache to ensure all results are freshly computed for accurate billing metrics.

Question 3

An enterprise data team runs a large nightly batch job using a standard all-purpose cluster. The job frequently fails due to cloud provider spot instance pre-emptions and takes over four hours to complete. How should the engineer refactor this architecture for maximum cost efficiency and reliability?

Accepted Answer

Convert the workload to use a Databricks Job cluster configured with spot instances and automatic fallback to on-demand.. Migrating the workload from an all-purpose interactive cluster to a Databricks Job cluster running on spot instances with an automatic fallback mechanism ensures cost-effective batch execution. Job clusters consume lower DBU rates than all-purpose clusters, and spot instances drastically reduce infrastructure costs while fallback guarantees completion despite cloud provider interruptions.

Answer

Provision a larger all-purpose cluster with double the worker nodes to brute-force execution speed.

Answer

Upgrade the cloud provider virtual machine family to the latest generation without changing cluster types.

Answer

Increase the Apache Spark executor memory fraction and decrease shuffle partition counts.

Question 4

A data engineer needs to ensure that PII data in a Delta table is accessible only to users in the 'HR_Manager' group. Which approach provides the most granular and scalable security implementation?

Accepted Answer

Apply a row filter function to the table using Unity Catalog.. Using Unity Catalog's Row-Level Security (RLS) via SQL functions is the standard best practice. It dynamically filters rows at query time based on the execution context, such as the current user's group membership. This approach avoids duplicating data into siloed tables, minimizes maintenance overhead, and ensures consistent enforcement across different BI tools and compute resources connecting to the same catalog.

Answer

Create separate views for each group and grant access to those views.

Answer

Implement column-level masking on the PII column.

Answer

Use data partitioning to store HR data in separate folders.

Question 5

Which of the following is the most secure method for a Data Engineer to provide access to a specific Delta table for a temporary project?

Accepted Answer

Adding the user to a temporary Unity Catalog group.. Granting temporary access should be done using time-bound permissions or dedicated temporary groups. In Unity Catalog, the best approach is to manage access through a group and remove the user from that group when the project concludes. This ensures a clean, auditable, and repeatable process for managing temporary access requests, which is essential for maintaining a secure environment and avoiding the 'permission creep' that happens when users retain access indefinitely.

Answer

Granting ownership of the table to the user.

Answer

Sharing the credentials of a Service Principal.

Answer

Creating a copy of the table for the user.

Question 6

A Data Engineer needs to ensure that PII data in a Delta table is accessible only to members of the 'hr_admin' group, while allowing all other users to view the non-PII columns. Which Unity Catalog feature is the most efficient way to implement this requirement?

Accepted Answer

Apply a column mask using a SQL function in Unity Catalog.. Unity Catalog's row-level security and column-level masking allow for fine-grained access control directly at the table level. By defining a masking function or a column filter, the Data Engineer ensures that the security policy is enforced consistently across all SQL warehouses and Databricks Runtime versions. This approach centralizes governance, reduces administrative overhead compared to view-based security, and ensures that data privacy compliance is maintained without duplicating data or creating multiple table versions.

Answer

Create separate physical tables for HR and general users.

Answer

Use standard SQL views for every user to filter columns.

Answer

Assign the 'SELECT' permission on individual columns in the UI.

Question 7

A production Databricks workflow involves a task that runs a notebook. The notebook takes 15 minutes to finish, but the workflow is set to timeout after 10 minutes. What happens?

Accepted Answer

The task is terminated and marked as 'Failed' by the scheduler.. When a task exceeds its configured 'timeout' value, the Databricks scheduler forcibly terminates the task execution. This is a deliberate safety measure to prevent runaway processes from consuming cluster resources indefinitely. In production, this highlights the necessity of monitoring execution times and setting appropriate timeouts that account for normal data volume fluctuations while catching truly stuck jobs that could impact cost and resource availability.

Answer

The workflow continues to run, but logs a warning about the duration.

Answer

The workflow pauses and waits for the engineer to manually approve continuation.

Answer

The task continues to run, but is moved to a background queue.

Question 8

A data engineer is troubleshooting a Databricks Workflow where a downstream task relies on an upstream task's output. Which TWO actions ensure the data dependency is correctly handled during a failure scenario?

Accepted Answer

Configure the downstream task to use the 'depends_on' attribute to reference the upstream task ID.. Proper dependency management ensures that downstream tasks do not attempt to process incomplete or missing data from upstream failures. By using task dependencies and repair-and-rerun capabilities, engineers can isolate failures and maintain data integrity. These features are critical in production pipelines to prevent downstream corruption and ensure that data lineage remains consistent throughout the workflow execution cycle regardless of individual component failures.

Answer

Set the task timeout to zero to prevent the workflow from ever stopping during a failure.

Answer

Hardcode the file path of the upstream output into the downstream task configuration.

Answer

Disable all retries to ensure that the error log is captured immediately upon failure.

Question 9

An organization is migrating to Unity Catalog and needs to secure sensitive data. Which TWO of the following statements regarding Unity Catalog security best practices are correct?

Accepted Answer

Use groups instead of individual users for access grants.. Effective security in Unity Catalog requires a deep understanding of object ownership and the principle of least privilege. By ensuring that objects are owned by a group rather than an individual, organizations prevent access gaps when staff turnover occurs. Additionally, using service principals for automated pipelines ensures that data access is tied to the workload rather than a user, maintaining consistent security postures across environments.

Answer

Assign table ownership to individual users for better tracking.

Answer

Ensure that the metastore admin has access to all data.

Answer

Public access should be granted to the root catalog.

Question 10

An organization needs to share a dataset with a client who does not use Databricks. What is the most efficient and secure way to share this data using Unity Catalog?

Accepted Answer

Use Delta Sharing with an open sharing provider.. Delta Sharing is an open-standard protocol that allows sharing data with any recipient, regardless of their platform or environment. By using Unity Catalog to share data, the organization maintains centralized governance and audit trails while enabling the recipient to consume the data using familiar tools. This approach is highly efficient because it avoids the need for data duplication and ensures the recipient always accesses the most current version.

Answer

Export the data to CSV and upload it to a public FTP server.

Answer

Grant the client read-only access to a dedicated Databricks workspace.

Answer

Create a public API endpoint that queries the table for the client.