CCNA Secure, monitor, and optimize data storage and data processing Questions

POSIX-style ACLs on Data Lake Storage Gen2 apply at directory and file level, enforcing least-privilege access for individual users and groups independently of broad role assignments. This granular permission model directly prevents unauthorised reads by limiting each principal to only the paths they require.

Lifecycle management rules evaluate blob age and apply actions automatically, so a rule scoped to the container with a delete action after 90 days enforces the retention policy without custom code. This matches Azure Data Lake Storage Gen2's native tiering and expiry capability.

This option uses customer-managed keys stored in Azure Key Vault, which allows you to control rotation and revoke access by disabling or deleting the key. Configuring a rotation policy in Key Vault automates rotation. Granting the storage account a managed identity with access to the key vault enables secure key usage without storing secrets.

Customer-managed keys in Azure Key Vault replace Microsoft-managed keys for storage encryption, and enabling Key Vault logging records every key access. Together these satisfy both the CMK encryption requirement and the key-access auditing constraint for the financial data.

Blob soft delete retains deleted blobs for a configurable retention period, letting you recover data removed accidentally. Enabling it at the storage account level protects all containers, satisfying the requirement to prevent accidental deletion in the Data Lake Storage Gen2 account.

ACLs in Azure Data Lake Storage Gen2 allow you to grant permissions to specific Microsoft Entra ID security groups at the directory or file level. By setting an ACL entry for the group and a mask, you can ensure that only members of that group have read access, even if others have broader RBAC roles. ACLs are evaluated together with RBAC to determine effective permissions.

Parquet is a columnar format that enables column pruning and predicate pushdown, reducing I/O and cost. Partitioning by the timestamp column further limits the data scanned when queries filter on that column. This directly addresses slow reads and high egress by minimizing the amount of data transferred and processed.

A cluster policy sets auto-termination at 30 minutes by default while permitting overrides, so idle clusters terminate automatically yet long-running workloads can extend the timeout. This satisfies both the cost-control and override requirements without manual intervention.

Configuring the workspace for Microsoft Entra ID authentication enforces identity-based access, while diagnostic settings stream audit logs to a Log Analytics workspace or storage account. Together these satisfy both constraints: Entra ID authentication for all access and auditable API calls.

Enabling 'Secure output' prevents activity output from being written to Data Factory monitoring logs, directly satisfying the requirement that sensitive data must not appear in pipeline logs. Unlike 'Secure input', which masks only input values, this setting suppresses the logged output itself, ensuring credentials or personal data captured during execution remain hidden.

A scan rule set containing built-in classification rules tells Purview which sensitive data types to detect during scans of Data Lake Storage Gen2. Classification then applies automatically, satisfying the requirement for automatic detection and classification of sensitive data.

Hash distribution on a column with many nulls causes skew because all nulls are placed in the same distribution. Choosing a column with high cardinality and even distribution (e.g., a surrogate key) ensures data is evenly spread across distributions, improving query parallelism and reducing data movement for joins on that column.

Data Lake Storage Gen2 supports POSIX-style ACLs at the directory and file level. Granting the analyst's Azure AD identity read and execute on the specific folder, without granting permissions on the parent or sibling folders, enforces least privilege. Execute permission on the parent directories is needed for traversal, but it does not grant read access to their contents.

Auto-optimize automatically compacts small files during writes, removing the need for manual OPTIMIZE jobs while preserving Delta Lake ACID guarantees. This directly satisfies the storage-cost constraint by reducing many small files into larger ones without extra orchestration.

The queued queries metric counts requests waiting for a concurrency slot in the dedicated SQL pool. Rising values indicate that active queries have exhausted available slots, directly identifying concurrency-limit queuing rather than resource pressure such as DWU saturation or tempdb usage.

Dedicated SQL pool emits query execution metrics to Azure Monitor, and Log Analytics stores and queries them via KQL. This satisfies the monitoring requirement by correlating DMV-level query performance data across the pool, which neither storage nor networking services provide.

POSIX-like ACLs grant per-file and per-directory permissions to individual security principals, exceeding what coarse role assignments allow. This satisfies the least-privilege requirement by scoping read, write and execute rights at the folder or file level within Azure Data Lake Storage Gen2.

Parquet's columnar, compressed layout lets analytical engines read only the referenced columns and skip irrelevant row groups, sharply cutting I/O against the stem's optimal-performance requirement. Row-based formats force full-row deserialisation regardless of projection, so Parquet directly satisfies the constraint that analytics workloads over Data Lake Storage Gen2 must minimise scanned data.

To enforce file and folder level permissions in Azure Data Lake Storage Gen2, you must use both Azure RBAC and ACLs. RBAC grants the user or service principal access to the storage account or container, while ACLs provide fine-grained permissions on individual files and folders. This combination ensures that users can only access the data for which they have explicit permissions, meeting the requirement.

Streaming Units govern the compute and parallelism available to the job; when throughput exceeds current capacity, partitions queue and latency climbs. Scaling SUs raises processing bandwidth, directly addressing the backlog constraint without altering the query or Event Hub configuration.

Configuring Azure Key Vault integration lets Storage Service Encryption use a customer-managed key stored in Key Vault rather than a Microsoft-managed key, giving the organisation control over the key lifecycle. This satisfies the requirement for encryption at rest with a customer-managed key.

Blob soft delete retains deleted blobs for a configurable retention period, satisfying the 30-day recoverability constraint. Setting the retention period to 30 days ensures accidental deletions remain restorable within that window, directly meeting the stated requirement.

Storage firewalls and virtual network rules restrict network access to the storage account, blocking public internet traffic and permitting only approved subnets or IP ranges. This protects data in transit by enforcing network-level perimeter controls on Azure Data Lake Storage Gen2.

tempdb usage in a dedicated SQL pool indicates spill-to-disk from insufficient memory grants during large sorts, joins and aggregations. Monitoring it reveals queries exceeding granted memory, which degrade performance and signal the need for statistics updates or query tuning.

ACLs provide POSIX-style permissions at file and directory level, so individual folders within a container can grant read access independently. This satisfies the requirement for folder-scoped authorisation, which container-level RBAC roles cannot achieve because they apply to the whole container.

Partition elimination allows the query optimizer to skip partitions that do not contain relevant data. If queries filter on the partitioning column (e.g., date), and the predicate is sargable, the engine can scan only the necessary partitions. This directly reduces I/O and improves performance for date-range queries.

Azure Monitor collects Stream Analytics job metrics including SU % utilisation and supports metric alerts on thresholds. It satisfies the stem's requirement to track resource utilisation in real time and notify when the job approaches capacity, unlike log-only or billing-focused services.

Log Analytics stores Azure Data Factory diagnostic logs, and alert rules defined against that workspace trigger notifications on pipeline run failures or duration thresholds. This satisfies the monitoring and alerting requirement for Data Factory pipeline runs.

sys.dm_exec_requests shows currently executing requests in the serverless SQL pool, including state, command, and session ID.

Partitioning folders by date lets the serverless SQL pool prune irrelevant directories, so a query filtered on date reads only the matching partition folders instead of the whole container. Combining that with an OPENROWSET wildcard path scoped to those folders directly reduces bytes scanned and cost, and it requires no rewrite of the underlying files.

Storage account diagnostic settings stream control-plane and data-plane events, including authenticated read, write and delete operations, to Log Analytics, storage or Event Hubs. This provides the audit trail of data access and changes required by the stem.

Azure Log Analytics is part of Azure Monitor and is used to query pipeline logs and create alerts for failures.

Customer-managed keys in Azure Key Vault satisfy the at-rest encryption requirement by letting you supply your own RSA key that wraps the account's data encryption key, rather than relying on Microsoft-managed keys. This gives you control over rotation and revocation, meeting the stem's demand for a Blob Storage encryption feature.

The activity policy's retry property sets the maximum retry count, and retryIntervalInSeconds defines the pause between attempts. Setting retry to 3 and retryIntervalInSeconds to 300 yields three retries at five-minute intervals, exactly matching the stated requirement.

Microsoft Defender for Storage continuously analyses data-plane telemetry and raises security alerts for anomalous access, such as unusual locations or suspicious enumeration. This directly satisfies the requirement to detect and alert on access patterns indicating a possible breach in the hierarchical-namespace account.

POSIX ACLs on the target directory grant read, write and execute to the scientists, scoping access to that path only. This satisfies the constraint of limiting folder creation and uploads to a specific directory, which RBAC roles cannot do at path level.

Configuring diagnostic settings for Azure Data Lake Storage Gen2 allows you to stream resource logs, such as StorageRead and StorageWrite, to a Log Analytics workspace. These logs capture details about each access, including the identity, operation, and timestamp. You can then query and retain the logs in Log Analytics for 90 days (or more) to meet auditing requirements.

Configuring customer-managed keys in the storage account's encryption blade satisfies the requirement for encryption at rest using a key held in Azure Key Vault. Azure Storage encryption applies AES-256 to all data at rest, and selecting customer-managed keys delegates key control to Key Vault rather than Microsoft-managed keys.

Diagnostic settings route Azure Data Factory activity logs, including pipeline run records with a Duration property, to Log Analytics. A log search alert rule can run a scheduled query such as ADFPipelineRun | where Duration > 60m and trigger an action group when results are found. This is the standard, low-effort method to alert on pipeline duration without custom code or external monitoring.

TDE with a customer-managed key allows you to bring your own key stored in Azure Key Vault. This gives you control over the encryption key, including rotation and revocation. It satisfies the requirement to encrypt data at rest using a customer-managed key in Azure Key Vault.

Azure RBAC grants coarse access at account, container or folder scope, while POSIX-like ACLs refine permissions down to individual directories and files for specific principals. Combining both enforces least privilege without touching the storage account firewall.

Customer-managed keys allow you to use your own keys in Azure Key Vault to encrypt the storage account. This gives you control over key rotation and access, and you can audit key usage through Key Vault logging. It meets the requirements for encryption at rest, key control, and auditing.

Workload groups with classifiers assign queries to resource buckets based on rules, giving each workload dedicated memory and concurrency. This reduces resource class conflicts and queuing by preventing competing queries from contending for the same resources.

Transient connection refusals during peak load are best absorbed by retry logic. A retry policy with backoff interval reattempts the copy activity after progressively longer waits, resolving intermittent FTP failures at negligible cost and no infrastructure changes.

Partitioning Parquet files by date lets the serverless SQL pool read only relevant folders, since partition pruning eliminates scanning of unrelated files. This directly reduces data scanned and I/O, which is the dominant performance constraint for serverless queries over Data Lake Storage Gen2.

Dedicated SQL pool auditing is configured directly on the pool and supports Azure Storage, Log Analytics, and Event Hubs as destinations. This captures all queries and unauthorized IP access attempts, meeting the compliance requirement. The setting is found in the Azure portal under the dedicated SQL pool's Security section, and it can also be set via PowerShell or Azure CLI.

Azure Data Lake Storage Gen2 supports POSIX-like ACLs when hierarchical namespace is enabled. Applying ACLs at the directory level allows granular control per user or group without affecting other containers. This meets the requirement for directory-level access and least privilege. The groups must exist in Azure AD and be assigned as principals on the ACL entries.

Custom sensitivity labels in Microsoft Purview Information Protection extend classification beyond built-in types, letting you tag organisation-specific sensitive data and apply those labels across registered data sources. This satisfies the stem's requirement that sensitive data be properly classified, while label usage feeds Purview's monitoring of access to labelled assets.

Queue depth measures pending requests awaiting a free self-hosted integration runtime node, so a rising value directly signals the IR cannot keep pace with copy activity demand. This exposes the bottleneck constraint, whereas CPU or memory alone may not reflect queued work.

Column-level security in Azure Synapse Analytics is enforced through T-SQL GRANT statements, letting you deny SELECT on specific columns while permitting access to the rest of the table. This directly satisfies the requirement to restrict sensitive columns, unlike object-level permissions or dynamic data masking, which obscures values rather than blocking access.

To encrypt data in transit between the self-hosted integration runtime and the on-premises SQL Server, you must enable SSL/TLS on the SQL Server connection by setting Encrypt=True in the connection string. For Windows authentication, the linked service must use a domain account with the necessary permissions. This combination ensures both secure transmission and proper authentication, meeting the requirements.

Dynamic data masking applies a masking rule to the column so that non-privileged users see masked data, while users with the UNMASK permission see the actual values. This directly meets the requirement with minimal administrative effort because the masking rule and UNMASK grant are set once and apply to all queries.

The SAS token supplies delegated, time-limited credentials appended to the external data source location, letting the serverless SQL pool authenticate to the storage container. Without it, queries against the external data source would fail authorisation, so it grants the read access needed for querying.

Backlogged input events counts events queued but not yet processed because the job lacks sufficient streaming units. Rising values indicate the job cannot keep pace with incoming IoT data, signalling that events are delayed or dropped due to inadequate processing capacity.

Input Events Backlogged counts events awaiting processing, exposing whether the job's streaming units or query logic cannot keep pace with incoming throughput. Rising backlog directly signals a performance bottleneck in the Azure Stream Analytics pipeline.

Dynamic Data Masking applies masking rules to designated columns so unauthorised users see obfuscated values at query time, while privileged users retain full data. This satisfies the requirement to mask PII without altering stored data.

To use customer-managed keys for Azure Storage encryption, you create a key in Azure Key Vault (or Managed HSM) and then configure the storage account to use that key for encryption at rest. This allows you to control key lifecycle, rotation, and access. The storage account must have a managed identity with permissions to access the key vault. This is the standard method to meet the requirement of customer-managed keys for data at rest.

Watermark delay measures the difference between the latest event processed and the newest event received, expressed in seconds. It directly quantifies end-to-end processing lag, so comparing it against the 10-second SLA threshold tells you whether the job meets the required latency.

Scan rule sets bundle classification rules, including built-in system rules for credit card and other sensitive types, which Purview applies during scans to detect and label matching data automatically. This satisfies the automatic detection requirement in the stem.

Default ACLs apply only to new child items created within the directory, not to the directory itself or to existing objects, so they cannot grant the service principal read and write access to the target directory directly. This option is tempting because default ACLs are designed to propagate permissions to future files and subdirectories, making them correct when the requirement is to control access for newly created content rather than the existing directory.

The source SQL Server could be the bottleneck due to resource contention. Monitoring its performance counters during the pipeline run can reveal if the server is under heavy load, causing slower data reads. This helps identify whether the degradation is due to source-side issues, which is a common cause in on-premises to cloud copy scenarios.

Retry logic in the Data Factory activity handles transient deadlock failures without altering the stored procedure's transaction semantics or isolation level. Since the procedure updates multiple tables in one transaction and is critical for reporting, retrying the activity preserves Read Committed Snapshot behaviour and avoids performance penalties, satisfying the minimal-impact constraint.

Dynamic data masking with the partial function allows you to mask a portion of the data while revealing the last four characters. The MASKED WITH clause is applied at the column level without changing storage size. Granting UNMASK permission to elevated users allows them to see the full data. This meets all requirements: masking for regular users, full access for privileged users, and no storage impact.

Lifecycle management policies automatically transition blobs between hot, cool and archive tiers based on age or last-access rules, directly reducing storage cost for ageing data. This satisfies the requirement to optimise Data Lake Storage Gen2 costs without manual intervention.

Azure Monitor ingests Azure Databricks diagnostic logs and Spark metrics, enabling alert rules on job failures and performance thresholds. It is the native monitoring plane for the workspace, satisfying the requirement to track Spark job performance and trigger failure alerts.

Transactions counts the total number of successful and failed requests against the storage account, directly satisfying the health-monitoring requirement. It is the only metric that surfaces both outcomes, letting you alert on failure spikes without enabling diagnostic logging.

Partitioning data by date into folders, such as year=2024/month=10/day=01, enables query engines to skip irrelevant folders when a query includes a filter on the partition column. This reduces the amount of data scanned, lowering cost and improving performance. Because the data format remains JSON and the files stay in the same account, no format conversion or data movement is required.

Storage Service Encryption is enabled by default on every Azure Data Lake Storage Gen2 account, encrypting data at rest with Microsoft-managed keys automatically. No configuration is required, satisfying the requirement for Microsoft-managed key encryption without customer key setup.

Transparent Data Encryption operates at the storage layer, encrypting data and log files at rest, and supports a customer-managed key held in Azure Key Vault — satisfying the stem's requirement that the dedicated SQL pool's data at rest be encrypted under organisational key control.

Hierarchical namespace enables file and folder level ACLs, allowing granular permissions similar to a file system. Azure Storage logging (or Azure Monitor integration) can capture access details for auditing. This combination provides fine-grained access control and auditing with minimal administrative effort, as ACLs can be inherited and managed via tools like Azure Storage Explorer or scripts.

Workload groups let you assign importance and a resource percentage per group, so high-priority queries pre-empt batch work while the batch group retains a guaranteed minimum, preventing starvation under the dedicated SQL pool's concurrency limits.

This is the correct approach for implementing row-level security in Azure Synapse Analytics dedicated SQL pool. You create an inline table-valued function that returns 1 when a user's region matches the row's region, then bind it to a security policy on the target table. Users are assigned to database roles, and the predicate is applied automatically to all queries, regardless of the client tool, ensuring consistent enforcement.

Column-level security uses GRANT and DENY statements on individual columns, restricting which roles can read specified fields within the table. This directly enforces the requirement that only users holding a certain role view particular columns.

Replicating the small dimension table places a full copy on each compute node. When the fact table is hash-distributed on the join key, the join can occur locally on each node without shuffling the dimension data. This eliminates data movement for the dimension side, which is a standard optimization for star-schema joins in dedicated SQL pools.

Azure Blob Storage lifecycle management policies can automatically transition blobs between access tiers based on age. Moving data to Cool after 30 days reduces storage costs for infrequently accessed data, and moving to Archive after 90 days further reduces costs for rarely accessed data. This aligns with the access pattern and minimizes costs while keeping data available (though Archive requires rehydration for access).