Databricks-DE-Assoc Databricks Intelligence Platform Practice Test 3 — 15 Questions

Question 1

Which feature of the Databricks Intelligence Platform allows users to manage fine-grained access control across workspaces for tables, files, and machine learning models?

Accepted Answer

Unity Catalog. Unity Catalog is the centralized governance layer within the Databricks Intelligence Platform. It provides a single interface to manage permissions across the entire data estate. This is vital for security and compliance because it eliminates the need for siloed permission management in individual workspaces, ensuring that security policies are consistently applied and auditable across all users, data assets, and compute resources in the platform.

Answer

Cluster Policies

Answer

Workspace Admin Settings

Answer

Secret Scopes

Question 2

Refer to the exhibit. A data engineer is deploying an automated job. Based on the provided configuration, what is the primary benefit of using the 'autoscale' attribute in this cluster definition?

Accepted Answer

It enables the cluster to adjust resources dynamically based on current job load.. The 'autoscale' configuration allows the Databricks Intelligence Platform to dynamically adjust the number of worker nodes based on the workload demands. By setting a minimum and maximum range, the cluster automatically adds nodes during intensive processing and removes them during idle periods. This is critical for optimizing costs and performance, ensuring that compute resources are consumed only when necessary without requiring manual intervention by the engineer.

Answer

It forces the cluster to always use 8 nodes to ensure maximum performance.

Answer

It ensures that the cluster uses specific instance types defined in the policy.

Answer

It mandates the use of Spot Instances to reduce total cost of ownership.

Question 3

A team is transitioning to the Databricks Intelligence Platform. Which TWO actions are required to successfully register a table in Unity Catalog using the three-level namespace?

Accepted Answer

Grant the 'USE CATALOG' privilege on the target catalog.. The three-level namespace (catalog.schema.table) is a fundamental feature of Unity Catalog. To register a table, an engineer must have sufficient privileges in the specific catalog and schema, and the table must be created using the correct fully qualified path. Understanding this structure is crucial because it enables unified data discovery and governance across different environments, preventing namespace collisions and allowing for standardized cross-database operations in the platform.

Answer

Configure a local Hive Metastore on the cluster.

Answer

Manually update the cluster's Spark configuration to point to the metastore URL.

Answer

Set the default database to 'default' before executing the CREATE statement.

Question 4

A data engineer is designing a secure architecture using the Databricks Intelligence Platform. Which TWO of the following statements accurately describe the role and capabilities of Unity Catalog within this platform? (Choose TWO)

Accepted Answer

Unity Catalog provides centralized access control for tables, views, volumes, and AI models across multiple workspaces.. Unity Catalog is the fine-grained governance solution for data and AI across the Databricks platform, centralizing access control for tables, volumes, and models. It operates across multiple workspaces in a metastore, ensuring consistent security policies and audit logging regardless of which workspace or compute resource executes the user queries.

Answer

Unity Catalog replaces the underlying cloud provider IAM roles for all compute resources executing queries.

Answer

Unity Catalog automatically handles the physical partitioning of Delta tables for optimal query performance.

Answer

Unity Catalog requires all data to be migrated into proprietary binary formats before governance policies can be applied.

Question 5

An organization is adopting the Databricks Intelligence Platform and wants to leverage Mosaic AI for building custom machine learning models. Which feature allows data engineers to track machine learning experiments, log parameters, and manage model artifacts reliably?

Accepted Answer

MLflow. MLflow is the open-source platform integrated natively into Databricks Mosaic AI for managing the end-to-end machine learning lifecycle. It provides comprehensive tracking of experiments, parameters, metrics, and registered model versions, ensuring reproducibility and smooth transition from development to production environments within the data lakehouse architecture.

Answer

Databricks SQL dashboards

Answer

Unity Catalog volumes

Answer

Delta Live Tables

Question 6

A data engineering team needs to ingest streaming data from Kafka into a Delta table while maintaining exactly-once processing guarantees and low latency. Which Databricks Intelligence Platform feature should they utilize to build this streaming pipeline declaratively?

Accepted Answer

Delta Live Tables. Delta Live Tables (DLT) is a declarative framework that simplifies ETL pipeline development. By defining data sources and target tables using SQL or Python, DLT manages infrastructure deployment, task orchestration, and error handling automatically, while leveraging structured streaming under the hood for low-latency, exactly-once ingestion.

Answer

Databricks Job clusters with manual spark-submit scripts

Answer

Databricks SQL query history

Answer

Unity Catalog lineage graphs

Question 7

A data engineer is designing a Delta Lake table that will be used for both batch and streaming reads. The table must support upserts from a streaming source and maintain ACID transactions. The engineer wants to ensure that the table can be efficiently queried by downstream consumers using SQL while minimizing storage costs. Which TWO actions should the engineer take to meet these requirements? (Choose two.)

Accepted Answer

Use the MERGE INTO command to perform upserts from the streaming source into the Delta table.. To support streaming upserts with ACID transactions and efficient SQL queries while minimizing storage costs, the engineer should use MERGE INTO for upserts and regularly run OPTIMIZE with Z-ORDER. MERGE INTO ensures atomic upserts, and OPTIMIZE with Z-ORDER compacts files and improves data skipping, reducing storage and query costs. Other options either add overhead, degrade performance, or remove essential Delta Lake features.

Answer

Enable change data feed on the Delta table to allow downstream consumers to read change events.

Answer

Partition the Delta table by a high-cardinality column to improve query performance.

Answer

Convert the Delta table to a Parquet table to reduce storage costs.

Question 8

A data engineer needs to run a Databricks notebook on a schedule every day at 8:00 AM UTC. The notebook performs data transformations and writes results to a Delta table. The engineer wants to ensure the job runs reliably and can be monitored. Which Databricks feature should the engineer use to schedule and monitor the notebook?

Accepted Answer

Databricks Jobs with a cron schedule.. Databricks Jobs is the correct feature for scheduling and monitoring notebooks. It supports cron-based schedules, provides a UI for monitoring runs, and can send alerts on failure. Other options like SQL Alerts, Repos, or Delta Live Tables serve different purposes and do not provide the required scheduling and monitoring for a notebook. Therefore, Databricks Jobs is the appropriate choice.

Answer

Databricks Delta Live Tables with a continuous pipeline.

Answer

Databricks Repos with a CI/CD pipeline.

Answer

Databricks SQL Alerts.

Question 9

A data engineer is working in a Databricks workspace where Unity Catalog is enabled. They need to run a SQL query that reads from the table sales in the catalog prod and schema marketing. Which fully qualified name should they use?

Accepted Answer

prod.marketing.sales. Unity Catalog uses a three-level namespace: catalog.schema.table. For a table named sales in the marketing schema of the prod catalog, the fully qualified name is prod.marketing.sales. This naming convention allows objects to be referenced unambiguously across the workspace and is fundamental to Unity Catalog governance.

Answer

prod.sales.marketing

Answer

marketing.prod.sales

Answer

sales.marketing.prod

Question 10

A data engineer needs to create a Delta table in Unity Catalog that will be used by multiple teams. The engineer wants to ensure that the table is governed by Unity Catalog and that access can be controlled using SQL GRANT statements. What is the correct way to create the table?

Accepted Answer

CREATE TABLE my_catalog.my_schema.my_table (id INT, name STRING) USING DELTA;. To create a managed Delta table in Unity Catalog, use the CREATE TABLE statement with a three-part name (catalog.schema.table) and specify USING DELTA. This ensures the table is governed by Unity Catalog and can be secured with GRANT statements.

Answer

CREATE EXTERNAL TABLE my_catalog.my_schema.my_table (id INT, name STRING) USING DELTA LOCATION '/mnt/data/my_table';

Answer

CREATE TABLE my_catalog.my_schema.my_table AS SELECT * FROM parquet.`/mnt/data/my_table`;

Answer

CREATE TABLE my_table (id INT, name STRING) USING DELTA LOCATION '/mnt/data/my_table';

Question 11

A data engineer is configuring a Databricks job that must run on a schedule and send an email notification if the job fails. They want to minimize manual intervention. Which feature should they use to define the schedule and failure notification?

Accepted Answer

Databricks Jobs with a cron schedule and email notifications on failure. Databricks Jobs provide built-in scheduling via cron and support email notifications on failure. This directly meets the need to automate job runs and alert the engineer when something goes wrong, reducing manual intervention. Other features like SQL alerts or cluster policies serve different purposes and cannot replace job scheduling and failure notifications.

Answer

Cluster policies with auto-termination settings

Answer

Databricks SQL alerts with a custom schedule

Answer

Delta Live Tables with expectations for failure handling

Question 12

A data engineer needs to run a Databricks notebook that processes data stored in an external ADLS Gen2 location. The notebook must access the data securely without embedding credentials in the notebook code. The engineer has already configured a Unity Catalog external location with a storage credential. Which method should the engineer use to read the data?

Accepted Answer

Reference the external location path directly in the notebook using the Unity Catalog external location URL.. Unity Catalog external locations are designed to provide secure access to cloud storage using storage credentials managed by Unity Catalog. Once an external location is configured, data engineers can reference the path directly in their notebooks, and Unity Catalog handles authentication and authorization. This approach avoids embedding credentials in code, aligns with security best practices, and ensures that access is governed by Unity Catalog permissions.

Answer

Use a personal access token (PAT) stored in Databricks secrets to authenticate to ADLS Gen2.

Answer

Set Spark configuration spark.hadoop.fs.azure.account.oauth2.client.id and related properties in the notebook to authenticate.

Answer

Mount the ADLS Gen2 container to DBFS using a service principal and then read the data from the mount point.

Question 13

A data engineer working in a Databricks workspace needs to create a new notebook that will be shared with teammates in the same workspace. They want the notebook to be organized in a folder named 'team_project' and be visible to all workspace users. Which action should the data engineer take to accomplish this in the Databricks workspace?

Accepted Answer

Navigate to the Workspace folder, create a new folder named 'team_project', then create the notebook inside it and set its permissions to 'Can View' for all workspace users.. In Databricks, the Workspace folder is the shared area where notebooks and folders can be organized and accessed by workspace users. Creating a folder named 'team_project' there and placing the notebook inside, then setting permissions to 'Can View' for all workspace users, directly meets the requirement. This method uses native workspace features without introducing unnecessary complexity.

Answer

Create the notebook in the Workspace folder, then use the Databricks CLI to set the notebook's permission to 'Can Manage' for the 'users' group.

Answer

Create the notebook in the Workspace folder under their user directory, then move it to a shared folder and grant 'Can Edit' permission to all workspace users.

Answer

Use the Databricks Repos feature to create a new repository, add a notebook, and set the repository permissions to 'Can Read' for all workspace users.

Question 14

A data engineer is using Databricks Asset Bundles to deploy a data pipeline that includes a job and a notebook. The engineer wants to ensure that the deployment is idempotent and can be rolled back if needed. Which TWO of the following statements accurately describe the benefits of using Databricks Asset Bundles for this scenario? (Choose two.)

Accepted Answer

Databricks Asset Bundles allow you to define the entire pipeline as code, enabling version control and reproducible deployments.. Databricks Asset Bundles treat infrastructure as code, enabling version control and reproducible deployments. They allow you to define jobs, notebooks, and other resources in YAML, which can be committed to Git. This makes deployments idempotent and supports rollback by redeploying a previous version. The other options describe features not provided by Asset Bundles, such as automatic cluster creation, data synchronization, or mandatory Unity Catalog usage.

Answer

Databricks Asset Bundles require the use of Unity Catalog for all deployments.

Answer

Databricks Asset Bundles automatically synchronize data between different environments during deployment.

Answer

Databricks Asset Bundles automatically create a new cluster for each deployment to ensure isolation.

Question 15

A data engineer is configuring a Databricks cluster to run a Spark job that processes large datasets. The job requires high memory and will run for several hours. The engineer wants to minimize costs while ensuring the job completes successfully. Which cluster configuration should the engineer choose?

Accepted Answer

Use a job cluster with autoscaling enabled, high-memory instance types, and a minimum of 2 workers and a maximum of 10 workers.. For long-running batch jobs, job clusters are more cost-effective than all-purpose clusters. Autoscaling allows the cluster to scale up or down based on workload, ensuring resources are used efficiently. High-memory instances meet the memory requirement. Setting a minimum and maximum worker count provides boundaries for autoscaling, preventing over-provisioning. This combination minimizes cost while ensuring the job has sufficient resources.

Answer

Use an all-purpose cluster with a fixed number of high-memory workers and enable autoscaling to reduce costs during peak hours.

Answer

Use a job cluster with a fixed number of high-memory workers and enable auto-termination after 30 minutes of inactivity.

Answer

Use an all-purpose cluster with autoscaling enabled and a minimum of 2 workers and a maximum of 10 workers, with high-memory instance types.