CCNA Data Governance Questions

Audit logs in Unity Catalog record all access and query activities, including the user, timestamp, and the tables accessed. They are essential for security auditing and compliance. Data lineage, information schema, and Delta transaction logs serve different purposes and do not provide a record of user access.

Dynamic views or Row-level security (RLS) and Column-level security (CLS) are essential for compliance. By utilizing SQL functions like current_user() within a defined mask, you ensure that PII is obscured based on user identity. This approach centralizes security within the data platform, preventing the need to duplicate datasets for different access levels, which significantly reduces the administrative burden and minimizes the risk of unauthorized data exposure in production environments.

Unity Catalog requires both `USE EXTERNAL LOCATION` and the appropriate file-level privileges (`READ FILES`, `WRITE FILES`) to access data in an external location. The `USE EXTERNAL LOCATION` privilege grants the ability to reference the external location in queries and commands, while `WRITE FILES` allows writing data. Without `USE EXTERNAL LOCATION`, the service principal cannot utilize the external location, resulting in a permissions error despite having `WRITE FILES` and cloud storage permissions.

Customer-managed keys (CMK) for managed storage are the standard governance tool for ensuring that data at rest is encrypted according to organizational security policies. By configuring CMK, the organization maintains control over the encryption lifecycle, satisfying compliance requirements. This feature is critical for highly regulated industries where the entity, rather than the cloud provider, must maintain the ultimate authority over data access and encryption standards in the cloud.

A column mask in Unity Catalog applies a user-defined function at query time to transform or redact column values based on the invoking user's identity or group memberships. This allows the same table to return different values for different users without duplicating data or creating separate views. It is the native mechanism for dynamic column-level security and satisfies the requirement to hide email addresses from marketing analysts while still permitting aggregate queries.

In Unity Catalog, querying a table requires a chain of privileges: `USE CATALOG` on the catalog, `USE SCHEMA` on the schema, and `SELECT` on the table. Granting these three privileges provides read-only access without allowing any modifications. This follows the principle of least privilege and ensures the analyst can only read data as required.

Unity Catalog automatically captures lineage at the table and column level for queries executed on the platform. This metadata is crucial for impact analysis and compliance auditing. By providing a unified view of how data moves from source to destination, organizations can maintain transparency, satisfy regulatory requirements for data tracking, and easily debug complex ETL pipelines that span across multiple workspaces and various cloud storage locations.

Adopting the principle of least privilege during migration is essential. By reviewing and re-granting permissions in the new Unity Catalog environment, you can eliminate legacy security debt and ensure that only necessary access is provided. This is the perfect opportunity to implement a robust, role-based access control (RBAC) model, ensuring the new environment is more secure than the legacy Hive Metastore it replaces, and facilitating long-term compliance.

The correct syntax to apply a column mask in Unity Catalog is ALTER TABLE table_name ALTER COLUMN column_name SET MASK function_name, where function_name is a fully qualified function that returns the masked value. This ensures that the mask is applied consistently and only authorized users see the original data.

Delta Sharing is an open-standard protocol that allows sharing data with any recipient, regardless of their platform or environment. By using Unity Catalog to share data, the organization maintains centralized governance and audit trails while enabling the recipient to consume the data using familiar tools. This approach is highly efficient because it avoids the need for data duplication and ensures the recipient always accesses the most current version.

Unity Catalog can manage access to external cloud storage locations by creating 'External Locations'. This allows administrators to grant specific permissions to users to read from or write to these storage buckets without providing them with direct cloud provider credentials. This abstraction is vital for security, as it centralizes control and auditing of data access within the platform while keeping the underlying storage infrastructure shielded from the users.

Enabling audit logging captures all access and permission changes, providing the necessary audit trail. Applying tags to sensitive columns and using ABAC policies enforces fine-grained access control based on those tags, protecting sensitive data consistently. Together, these actions create a governance model that is both auditable and secure, leveraging Unity Catalog's native capabilities for centralized policy management.

Delta Sharing is an open protocol that uses a REST API to facilitate data sharing. The provider's Delta Sharing server authenticates requests and returns pre-signed URLs that allow the recipient to download data directly from cloud storage. This design enables secure, cross-platform sharing without exposing the provider's storage credentials or requiring the recipient to use Databricks.

Unity Catalog CMK for Azure requires that the Databricks managed identity for the metastore has the Key Vault Crypto Service Encryption User role on the Key Vault. This role allows Databricks to perform wrap and unwrap operations with the key. Without this role assignment, the metastore cannot use the CMK for encryption.