Courseiva

CCNA Develop data processing Questions

36 of 261 questions · Page 4/4 · Develop data processing · Answers revealed

226
Multi-Selectmedium

Which TWO components are required to set up a streaming data pipeline using Azure Synapse Analytics? (Select two.)

Select 2 answers
A.Azure Data Factory
B.Azure Event Hubs
C.Azure Analysis Services
D.Azure Blob Storage
E.Azure Synapse Pipelines (or Spark)
AnswersB, E

Event Hubs ingests streaming events.

Why this answer

To set up a streaming data pipeline in Azure Synapse Analytics, you need a streaming ingestion source and a processing engine. Azure Event Hubs (Option B) is the correct ingestion service for real-time streaming data. Azure Synapse Pipelines (or Spark) (Option E) provides the processing engine to transform and analyze the streaming data.

Azure Data Factory (Option A) is primarily for batch data integration, not streaming. Azure Analysis Services (Option C) is for OLAP modeling, not streaming. Azure Blob Storage (Option D) is a storage destination, not a required component for streaming ingestion or processing.

227
Multi-Selecthard

You are optimizing an Azure Synapse Analytics pipeline that uses mapping data flows. The pipeline runs slowly when processing 100 GB of data. Which THREE settings should you adjust to improve performance?

Select 3 answers
A.Set 'partition option' to 'Round robin' with a higher number of partitions.
B.Set a tumbling window trigger to run the pipeline every 5 minutes.
C.Increase the 'Compute type' to 'Memory Optimized' and increase the number of cores.
D.Enable 'Data flow debug' to monitor execution details.
E.Use 'Optimize shuffle' in the data flow settings.
AnswersA, C, E

Round robin partitioning distributes data evenly across partitions, improving parallelism.

Why this answer

Options A, C, and E are correct. Setting a higher number of partitions with Round robin partitioning improves parallelism across nodes. Increasing Compute type to Memory Optimized and increasing cores provides more resources for data processing.

Using Optimize shuffle reduces data movement during joins and aggregations, improving performance. Option B (tumbling window trigger) is a scheduling mechanism, not a performance tuning setting. Option D (Data flow debug) is for development and debugging, not for production performance optimization.

228
MCQmedium

Your organization has an Azure Data Factory pipeline that executes a series of activities to transform data. One of the activities is an Azure Databricks notebook that should run only if the previous activity succeeds. You need to configure the pipeline to handle failures gracefully and send an email alert if the Databricks activity fails. What should you do?

A.Add a failure output path from the Databricks activity to a Web activity that calls an email API.
B.Configure a retry policy and a timeout for the Databricks activity.
C.Use a Schedule trigger to run the pipeline and check for failures using Azure Monitor.
D.Set a dependency condition on the Databricks activity to 'Succeeded' and add a Send Email activity on the success path.
AnswerA

You can route failure output to a Web activity to send an email via Logic Apps or Azure Functions.

Why this answer

You can configure a failure output path from the Databricks activity and attach a Web activity that calls an email API to send an alert when the Databricks activity fails. Option B is incorrect because retry and timeout settings do not send email alerts. Option C is incorrect because a Schedule trigger does not handle activity-level failures directly.

Option D is incorrect because setting a dependency to 'Succeeded' only runs subsequent activities on success, not on failure.

229
Multi-Selecteasy

Which TWO features of Azure Databricks help manage data governance and security for sensitive data?

Select 2 answers
A.Structured Streaming
B.Secret Scopes
C.Auto Loader
D.Delta Live Tables
E.Unity Catalog
AnswersB, E

Secret Scopes securely store and manage access tokens and keys.

Why this answer

Secret Scopes (B) allow secure storage and referencing of sensitive credentials (e.g., API keys, database passwords) in Azure Databricks, preventing hardcoding in notebooks. Unity Catalog (E) provides fine-grained access control, data lineage, and centralized metadata management across workspaces, enabling governance of sensitive data through policies and auditing.

Exam trap

The trap here is that candidates confuse data processing features (Structured Streaming, Auto Loader, Delta Live Tables) with governance/security tools, because all are part of the Databricks ecosystem but serve fundamentally different purposes.

230
MCQhard

Refer to the exhibit. You are an Azure data engineer responsible for ensuring that all storage accounts used in data pipelines enforce HTTPS traffic. You apply the Azure Policy definition shown above. Later, a data engineer creates a new storage account with 'Enable secure transfer' set to Disabled. What will happen when the policy is evaluated?

A.The storage account will be created, but the policy will be evaluated later during a compliance scan.
B.The storage account will be created with HTTPS enabled automatically.
C.The storage account creation will be denied and the request will fail.
D.The storage account will be created, but an audit event will be logged.
AnswerC

The policy denies the creation if the condition is met.

Why this answer

The Azure Policy definition shown in the exhibit uses the 'Deny' effect for the 'Secure transfer to storage accounts should be enabled' policy. When a data engineer attempts to create a storage account with 'Enable secure transfer' set to Disabled, the policy evaluation occurs at resource creation time (before the resource is provisioned). Because the request violates the policy's condition (HTTPS traffic not enforced), Azure Resource Manager denies the creation request and returns a failure, preventing the non-compliant storage account from being created.

Exam trap

The trap here is that candidates often confuse the 'Deny' effect with 'Audit' or 'Disabled' effects, mistakenly believing that policy evaluation only happens during periodic compliance scans rather than at resource creation time, leading them to choose Option A or D.

How to eliminate wrong answers

Option A is wrong because Azure Policy with a 'Deny' effect is evaluated synchronously during resource creation, not deferred to a later compliance scan; a 'Deny' policy blocks the request immediately. Option B is wrong because the policy does not automatically modify the storage account configuration to enable HTTPS; it only denies or audits, and with 'Deny' effect it rejects the request outright. Option D is wrong because an 'audit' event would only occur if the policy effect were 'Audit' or 'AuditIfNotExists', but the exhibit specifies a 'Deny' effect, which blocks creation rather than logging an audit event.

231
MCQhard

You have a production pipeline in Azure Data Factory that copies data from an on-premises SQL Server to Azure Blob Storage using a self-hosted integration runtime. The pipeline fails intermittently with a 'Connection closed' error. The data volume is 50 GB per run. What should you first troubleshoot to resolve this issue?

A.Increase the memory and CPU resources on the self-hosted integration runtime machine and check network stability.
B.Increase the 'connection timeout' setting in the linked service to 30 minutes.
C.Change the copy activity to use staged copy with Azure Blob Storage as an intermediate store.
D.Disable fault tolerance in the copy activity to improve performance.
AnswerA

The self-hosted IR needs sufficient resources for large data transfers; 'Connection closed' often indicates resource exhaustion or network interruptions.

Why this answer

A 'Connection closed' error during large data transfers (50 GB) often occurs due to resource exhaustion or network instability on the self-hosted integration runtime (IR) machine. Insufficient memory or CPU on the IR node or unreliable network connectivity can cause the connection to drop. This is the first thing to troubleshoot.

Option B (increasing connection timeout) might help if the error is due to timeouts, but 'Connection closed' typically indicates an abrupt termination, not a timeout. Option C (staged copy) can improve reliability by using Azure Blob as an intermediate store, but it adds complexity and is not the first step. Option D (disabling fault tolerance) would make the pipeline less resilient and is not recommended; fault tolerance helps handle transient errors.

232
Multi-Selecthard

Which THREE factors should you consider when choosing between Azure Data Factory Mapping Data Flows and Azure Synapse Spark pools for data transformation?

Select 3 answers
A.Scheduling: Only Data Flows can be scheduled via triggers.
B.Ease of use: Mapping Data Flows provide a visual designer, while Spark requires code.
C.Data volume: Data Flows are limited to 100 GB, while Spark can handle petabytes.
D.Integration with other services: Data Flows can use integration runtimes, while Spark is limited to Synapse.
E.Debugging: Data Flows have a debug session limit of 8 hours, while Spark pools have no debug limit.
AnswersB, D, E

Data Flows are no-code, Spark requires coding.

Why this answer

Azure Data Factory Mapping Data Flows offer a visual, no-code designer for building data transformations, which lowers the barrier for users who are not proficient in programming. In contrast, Azure Synapse Spark pools require writing code in languages like PySpark, Scala, or SQL, making them more suitable for developers comfortable with coding. This distinction directly addresses ease of use as a key factor in choosing between the two services.

Exam trap

The trap here is that candidates assume Mapping Data Flows have a hard data volume limit (like 100 GB) or that Spark pools cannot be scheduled, when in fact both services are highly scalable and can be orchestrated via triggers, and the key differentiator is the coding versus visual interface.

233
MCQeasy

You are designing a data processing solution using Azure Databricks. The data is stored in Delta Lake format. You need to ensure that when you read the latest version of the table, you only see committed data and not uncommitted transactions. Which isolation level should you use?

A.WriteSerializable
B.Serializable
C.ReadUncommitted
D.SnapshotIsolation
AnswerD

Delta Lake uses Snapshot isolation to read the latest committed version.

Why this answer

Snapshot isolation is the correct choice because it provides a consistent view of the table by reading only the latest committed data, ignoring any uncommitted transactions. In Delta Lake, snapshot isolation ensures that readers see a snapshot of the table at a specific version, which includes only committed changes, making it ideal for read consistency without blocking concurrent writes.

Exam trap

The trap here is that candidates confuse write isolation levels (like WriteSerializable) with read isolation levels, or assume that Serializable is always the safest choice for consistency, when in fact Snapshot isolation is the specific Delta Lake mechanism for reading only committed data without blocking.

How to eliminate wrong answers

Option A (WriteSerializable) is wrong because it is a write isolation level that ensures serializable isolation for write operations, but it does not control read behavior to exclude uncommitted data. Option B (Serializable) is wrong because it is a general isolation level that prevents dirty reads, non-repeatable reads, and phantom reads, but it is not specifically designed to guarantee that only committed data is visible in Delta Lake; it also imposes more overhead than needed. Option C (ReadUncommitted) is wrong because it allows reading uncommitted data (dirty reads), which directly violates the requirement to see only committed data.

234
MCQmedium

You are troubleshooting a failed Azure Synapse Pipeline execution. The pipeline uses a Copy activity to load data from an on-premises SQL Server to Azure Data Lake Storage Gen2. The error indicates a 'Connection timeout' to the on-premises source. The Integration Runtime is Self-Hosted and has been running successfully for months. What is the most likely cause?

A.The SQL Server authentication credentials have expired.
B.The Self-Hosted Integration Runtime is not installed.
C.The on-premises network configuration has changed.
D.The Azure Storage account firewall is blocking access.
AnswerC

Network changes can block connectivity to the SQL Server.

Why this answer

A 'Connection timeout' error when using a Self-Hosted Integration Runtime (SHIR) that has been running successfully for months indicates a network-level issue rather than an authentication or configuration problem. The most likely cause is a change in the on-premises network configuration (e.g., firewall rules, proxy settings, or DNS changes) that prevents the SHIR from reaching the SQL Server on the specified port (typically TCP 1433). Since the SHIR is already installed and was working, the timeout points to a connectivity break, not a credential or installation issue.

Exam trap

The trap here is that candidates confuse authentication errors (which produce specific error messages like 'Login failed') with connectivity errors (which produce timeouts), leading them to incorrectly select credential-related options when the symptom is clearly a network timeout.

How to eliminate wrong answers

Option A is wrong because expired SQL Server authentication credentials would result in an 'Authentication failed' or 'Login failed' error, not a 'Connection timeout'. Option B is wrong because if the Self-Hosted Integration Runtime were not installed, the pipeline would fail with a different error (e.g., 'Unable to connect to Integration Runtime' or 'Integration Runtime is offline'), not a timeout. Option D is wrong because the Azure Storage account firewall controls inbound access to the storage endpoint, not outbound connectivity from the SHIR to the on-premises SQL Server; a storage firewall issue would produce an error related to storage access (e.g., 403 Forbidden), not a timeout to the source.

235
MCQhard

You have a Data Factory pipeline that runs a U-SQL script in Azure Data Lake Analytics. The script processes terabytes of data and outputs to a CSV file. The pipeline is failing with the error: 'The job failed with UserError: Script execution failed.' You need to troubleshoot the issue. Which approach should you take first?

A.Change the output format to Parquet to reduce file size.
B.Review the job logs in Azure Data Lake Analytics to identify the specific script error.
C.Migrate the U-SQL script to Azure Synapse Spark pool.
D.Increase the degree of parallelism for the U-SQL job.
AnswerB

Job logs provide detailed error messages that pinpoint the issue.

Why this answer

The most effective first step is to examine the detailed job logs in Data Lake Analytics, which contain the actual script error. Increasing parallelism or changing output format may not address the underlying script error. Moving to Azure Synapse is a larger architectural change.

236
Multi-Selecteasy

You are developing a data pipeline in Azure Data Factory that ingests data from multiple on-premises SQL Server databases to Azure Data Lake Storage Gen2. The data volume is about 1 TB per day. You need to ensure the pipeline can handle the volume and provide monitoring and alerting. Which THREE components should you include?

Select 3 answers
A.Azure Synapse Analytics pipeline
B.Self-hosted integration runtime
C.Azure Monitor
D.Power BI
E.Data flow activity
AnswersB, C, E

Required to connect to on-premises SQL Server.

Why this answer

Correct answers: B, C, and E. Self-hosted integration runtime (B) is required for connectivity to on-premises SQL Server databases. Azure Monitor (C) provides monitoring and alerting for the pipeline.

Data flow activity (E) allows data transformation at scale. Option A is incorrect because Azure Synapse Analytics pipeline is not used in this context; the pipeline is in Azure Data Factory. Option D is incorrect because Power BI is for visualization, not data ingestion.

237
MCQmedium

A data engineer is tasked with optimizing a Spark job in Azure Synapse Analytics that processes 10 TB of data daily. The job currently uses 50 executors with 4 cores each. The performance is bottlenecked by shuffle operations. The engineer wants to reduce shuffle data size. Which technique should be applied?

A.Coalesce the number of partitions before the shuffle.
B.Increase the number of shuffle partitions.
C.Broadcast all tables to avoid shuffles.
D.Use column pruning to select only required columns before shuffle.
AnswerD

Reduces data volume.

Why this answer

Column pruning (selecting only required columns) reduces the amount of data shuffled, decreasing shuffle size. Option A is incorrect because coalescing reduces the number of partitions but does not reduce the data volume shuffled. Option B is incorrect because increasing the number of shuffle partitions can increase shuffle overhead, not reduce it.

Option C is incorrect because broadcasting all tables is impractical for large datasets (10 TB) and does not reduce shuffle data.

238
MCQhard

You are designing a data processing solution for a healthcare organization. The solution must process streaming data from IoT devices and store it in Azure Data Lake Storage Gen2. The data must be available for both real-time dashboards and historical analysis. You need to minimize operational overhead. What should you do?

A.Ingest data via Azure Functions and write to Data Lake Storage; use Power BI to query Data Lake
B.Use Azure Stream Analytics to output to both Power BI and Data Lake Storage
C.Use Azure Databricks with Structured Streaming to write to Data Lake Storage and use Power BI DirectQuery
D.Ingest data to Azure Event Hubs, then use Event Hubs Capture to store in Data Lake Storage; use Power BI with Event Hubs
AnswerB

Stream Analytics is serverless, supports real-time output to Power BI and batch writes to Data Lake.

Why this answer

Azure Stream Analytics can directly output to both Power BI (for real-time dashboards) and Azure Data Lake Storage Gen2 (for historical analysis) in a single job, minimizing operational overhead by avoiding the need for multiple services or custom code. This serverless, fully managed service handles streaming data processing with low latency and integrates natively with Azure IoT Hub or Event Hubs for ingestion, making it ideal for healthcare IoT scenarios.

Exam trap

The trap here is that candidates often overcomplicate the solution by choosing Databricks (Option C) for its flexibility, overlooking that Stream Analytics provides a simpler, fully managed approach with native dual-output support that minimizes operational overhead.

How to eliminate wrong answers

Option A is wrong because Azure Functions writing directly to Data Lake Storage introduces additional latency and operational complexity (e.g., managing function scaling, retries) and does not natively support real-time streaming output to Power BI without extra integration. Option C is wrong because Azure Databricks with Structured Streaming requires managing a cluster and incurs higher operational overhead compared to a fully managed service like Stream Analytics, and Power BI DirectQuery from Data Lake Storage is not optimized for real-time dashboards due to latency. Option D is wrong because Event Hubs Capture writes data in Avro format to Data Lake Storage in batches (e.g., every 5 minutes or 100 MB), which introduces latency unsuitable for real-time dashboards, and Power BI directly querying Event Hubs is not supported without an intermediary like Stream Analytics.

239
MCQmedium

Your organization uses Azure Synapse Analytics. You need to design a data transformation pipeline that processes streaming data from Azure Event Hubs, performs aggregations over a 5-minute tumbling window, and loads the results into a dedicated SQL pool table. Which Azure service should you use to implement the streaming transformation?

A.Azure Stream Analytics
B.Azure Data Factory
C.Apache Spark for Azure Synapse
D.Azure Functions
AnswerA

Azure Stream Analytics can ingest from Event Hubs, perform tumbling window aggregations, and output to Synapse SQL pool.

Why this answer

Azure Stream Analytics is the appropriate service for real-time stream processing with windowed aggregations. Option B is wrong because Azure Data Factory is for batch orchestration. Option C is wrong because Spark Structured Streaming is for big data workloads but less integrated with SQL pools.

Option D is wrong because Azure Functions is not designed for streaming windowed aggregations.

240
Multi-Selecteasy

A company uses Azure Data Lake Storage Gen2 as the data lake. The data engineering team needs to ensure that sensitive data such as credit card numbers are masked when queried by non-admin users. The solution must be implemented within the data lake without moving data to another store. Which TWO features should they use? (Choose two.)

Select 2 answers
A.Azure Policy to audit access
B.Azure SQL Database dynamic data masking
C.Azure Synapse Serverless SQL with dynamic data masking
D.Microsoft Purview data classification and labeling
E.Azure Storage blob-level access policies
AnswersC, D

Can mask data in queries over external tables.

Why this answer

Azure Synapse Serverless SQL can query data directly from Azure Data Lake Storage Gen2 and supports dynamic data masking (DDM) to obfuscate sensitive columns like credit card numbers from non-admin users. This allows masking to be applied at query time without moving or transforming the underlying data in the lake.

Exam trap

The trap here is that candidates often confuse dynamic data masking in Azure SQL Database (which requires data to be in a relational store) with the ability to mask data in place in the data lake using Azure Synapse Serverless SQL, and they may overlook Microsoft Purview's role in classifying and labeling sensitive data as a prerequisite for applying masking policies.

241
MCQmedium

You are designing a data lakehouse architecture in Azure using Delta Lake. The solution needs to process batch and streaming data from multiple sources, including IoT devices and CRM systems. You need to ensure data quality by enforcing schema validation and handling schema evolution. You also need to provide a unified catalog for querying. Which service should you use?

A.Azure Purview
B.Azure Data Lake Storage Gen2
C.Azure Synapse Analytics serverless SQL pool
D.Azure Databricks Unity Catalog
AnswerD

Azure Databricks Unity Catalog provides a central catalog with schema enforcement and evolution for Delta Lake, making it the correct choice for a data lakehouse with batch and streaming data.

Why this answer

Azure Databricks Unity Catalog provides a unified governance solution for data and AI, including schema enforcement and evolution for Delta Lake. Option A is wrong because Azure Purview is for data discovery and lineage, not for schema enforcement. Option B is wrong because Azure Data Lake Storage Gen2 is storage only, not a catalog or governance layer.

Option C is wrong because Azure Synapse Analytics serverless SQL pool is a query engine and does not provide the same schema management features as Unity Catalog.

242
MCQeasy

You are designing a data processing solution in Azure Databricks to transform streaming data from Azure Event Hubs. The data must be aggregated in 1-minute tumbling windows and written to Azure Synapse Analytics. Which Spark API should you use?

A.RDD API
B.Structured Streaming
C.Spark Streaming (DStreams)
D.DataFrame API with batch processing
AnswerB

Supports windowed aggregations and streaming sinks.

Why this answer

Structured Streaming is the correct choice because it provides native support for event-time-based aggregations, such as 1-minute tumbling windows, and integrates seamlessly with Azure Event Hubs as a streaming source and Azure Synapse Analytics as a streaming sink using the `foreachBatch` or `writeStream` API. It offers exactly-once semantics and automatic state management for windowed operations, which are essential for reliable streaming ETL.

Exam trap

The trap here is that candidates confuse the older Spark Streaming (DStreams) API with Structured Streaming, assuming both are equally capable for event-time windows, but DStreams lack native event-time support and are deprecated in favor of Structured Streaming.

How to eliminate wrong answers

Option A is wrong because the RDD API operates at a low level without built-in support for event-time windowing, stateful aggregation, or streaming sinks like Azure Synapse Analytics, requiring manual implementation of checkpointing and fault tolerance. Option C is wrong because Spark Streaming (DStreams) uses micro-batch processing with a DStream API that is now in maintenance mode and lacks native event-time handling, making tumbling window aggregations more complex and less efficient compared to Structured Streaming. Option D is wrong because the DataFrame API with batch processing is designed for static data, not continuous streaming; it cannot process unbounded data from Event Hubs in real time or maintain state for tumbling windows without additional custom orchestration.

243
Multi-Selectmedium

Which TWO are valid ways to process data in Azure Synapse Analytics?

Select 2 answers
A.Use Logic Apps to run data transformations.
B.Use Azure Functions to process data in a serverless manner.
C.Use Synapse SQL pool to run T-SQL queries.
D.Use Power BI to transform data.
E.Use Synapse Spark notebooks to run Scala code.
AnswersC, E

Synapse SQL pool provides distributed query processing.

Why this answer

Synapse SQL pool (formerly SQL DW) is a dedicated or serverless SQL engine within Azure Synapse Analytics that allows you to run T-SQL queries for data transformation, loading, and querying. It is a first-class compute resource designed for large-scale data warehousing workloads, making T-SQL queries a valid and primary method for processing data in Synapse.

Exam trap

The trap here is that candidates confuse general Azure services (Logic Apps, Functions, Power BI) with native Synapse Analytics processing capabilities, forgetting that only Synapse SQL and Synapse Spark are first-class compute engines within the service.

244
MCQmedium

You are developing a data processing pipeline in Azure Synapse Analytics. The pipeline uses a mapping data flow to transform data from Azure Data Lake Storage Gen2 to a dedicated SQL pool. The data flow includes a Derived Column transformation that uses the expression: `iif(isnull(Column1), 'Default', Column1)`. However, the transformation is not handling NULL values correctly. What is the most likely cause?

A.The Derived Column transformation does not support the iif function; use a Conditional Split instead.
B.The column data type is not string; convert Column1 to string first.
C.The function names in the expression are case-sensitive; use 'isNull' instead of 'isnull'.
D.The expression must use ternary operator syntax: `Column1 == null ? 'Default' : Column1`.
AnswerC

This is correct. The function names are case-sensitive; the expression uses `isnull` (all lowercase), but the correct function name is `isNull` with a capital 'N'. This causes the transformation to fail to interpret the function correctly.

Why this answer

The function names in Azure Synapse mapping data flows are case-sensitive. The expression uses `isnull` (all lowercase), but the correct function name is `isNull` (capital 'N'). When the function name is typed with incorrect casing, the Derived Column transformation fails to recognize the function and does not evaluate the null check correctly, leading to NULL values not being replaced with 'Default'.

Exam trap

The trap here is that candidates assume function names in Azure Synapse mapping data flows are case-insensitive like in SQL, but they are actually case-sensitive, causing a seemingly correct expression to fail due to a subtle casing error.

How to eliminate wrong answers

Option A is wrong because the Derived Column transformation fully supports the `iif` function for conditional logic; a Conditional Split is not required for simple null handling. Option B is wrong because the `iif` function can handle any data type, and converting to string is unnecessary; the issue is not about data type conversion. Option D is wrong because mapping data flows do not support ternary operator syntax (`? :`); they use the `iif` function for conditional expressions.

245
MCQeasy

You need to perform incremental data loading from Azure SQL Database to Azure Data Lake Storage Gen2 using Azure Data Factory. Which approach is the most efficient?

A.Use a lookup activity to retrieve the last watermark value, then copy only new records with a filter.
B.Use a tumbling window trigger with a data flow that processes all data each time.
C.Use a mapping data flow with a full load and then use Azure Databricks to deduplicate.
D.Copy the entire table every time and use Azure Synapse serverless SQL to filter duplicates.
AnswerA

Watermark pattern is efficient and well-supported.

Why this answer

It uses a lookup activity to retrieve the last watermark value (e.g., a timestamp or incrementing key), then copies only new or changed records via a filter in the Copy activity. This minimizes data movement and processing time, making it the most efficient incremental loading approach in Azure Data Factory.

Exam trap

Microsoft often tests the misconception that any trigger-based or full-load approach can be adapted for incremental loading, but the key is minimizing data movement; candidates may overlook the watermark pattern and choose a full-load option because they think deduplication later solves the problem.

How to eliminate wrong answers

Option B is wrong because a tumbling window trigger with a data flow that processes all data each time performs a full load on every run, ignoring incremental logic and wasting resources. Option C is wrong because performing a full load followed by deduplication in Azure Databricks is inefficient; it moves all data repeatedly and adds unnecessary compute overhead. Option D is wrong because copying the entire table every time and using Azure Synapse serverless SQL to filter duplicates still transfers all data, incurring high egress costs and defeating the purpose of incremental loading.

246
MCQeasy

You are a data engineer at a retail company. You need to develop a data processing solution in Azure Synapse Analytics that reads sales transactions from Parquet files stored in Azure Data Lake Storage Gen2, transforms the data by aggregating daily sales per store, and writes the results to a dedicated SQL pool table for reporting. The transformation logic must be reusable and maintained in a source control system. You want to minimize administrative overhead and leverage serverless resources where possible. Which approach should you recommend?

A.Use Azure Data Factory with Mapping Data Flows to read Parquet files, perform aggregations, and write to the dedicated SQL pool.
B.Use a serverless SQL pool to query the Parquet files via OPENROWSET, then use CETAS to write the aggregated results to the dedicated SQL pool using an external table.
C.Create an Azure Synapse Spark notebook that reads Parquet files, performs aggregation using PySpark, and writes the results to the dedicated SQL pool using the Spark Synapse connector.
D.Use PolyBase in a dedicated SQL pool to create external tables over the Parquet files, then use INSERT...SELECT to load aggregated data into the target table.
AnswerB

Serverless SQL pool can read Parquet natively, and CETAS allows writing to dedicated SQL pool via external table.

Why this answer

A serverless SQL pool can directly query Parquet files using OPENROWSET, and the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement can transform and store aggregated results in a dedicated SQL pool table via PolyBase. This approach uses serverless resources for transformation, minimizing administrative overhead and avoiding the need to manage Spark pools. Option A is incorrect because Azure Data Factory with Mapping Data Flows runs on Spark clusters, which adds administrative overhead and does not align with the goal of minimizing management.

Option C is incorrect because creating an Azure Synapse Spark notebook requires provisioning and managing a Spark pool, which increases administrative overhead. Option D is incorrect because using PolyBase in a dedicated SQL pool requires creating external tables and performing INSERT...SELECT, which is less flexible for reusable transformation logic compared to CETAS in serverless SQL pool.

247
MCQeasy

You need to process a large number of small files (each < 1 MB) from Azure Blob Storage in Azure Synapse Analytics. The processing is I/O-bound due to many small file operations. Which approach should you use to improve performance?

A.Use wildcard paths to read multiple files at once.
B.Enable optimized write on the Spark session.
C.Convert the files to a binary format like Avro before processing.
D.Use 'spark.sql.files.maxPartitionBytes' to coalesce small files into larger partitions.
AnswerD

This configuration merges small files into larger partitions, reducing overhead.

Why this answer

`spark.sql.files.maxPartitionBytes` controls the maximum number of bytes packed into a single partition when reading files. By increasing this value, Spark coalesces many small files into fewer, larger partitions, reducing the overhead of task scheduling and I/O operations. This directly addresses the I/O-bound bottleneck caused by processing numerous small files in Azure Synapse Analytics.

Exam trap

The trap here is that candidates confuse file format conversion (Avro) or write optimization with read-side partition coalescing, failing to recognize that the core issue is the number of partitions created during file scanning, not the data format or write behavior.

How to eliminate wrong answers

Option A is wrong because wildcard paths only simplify file selection but do not reduce the number of partitions or I/O operations; each small file still becomes a separate partition by default. Option B is wrong because 'optimized write' is a Delta Lake feature that improves write performance by reducing small file output, but it does not help when reading existing small files from Blob Storage. Option C is wrong because converting to Avro changes the serialization format but does not inherently reduce the number of file read operations; the small file problem persists regardless of format.

248
MCQmedium

You are designing a data pipeline to ingest streaming data from IoT devices into Azure Synapse Analytics. The data must be available for querying with minimal latency, but you also need to handle spikes in throughput without data loss. Which service should you use as the ingestion layer?

A.Azure Data Lake Storage Gen2
B.Azure Blob Storage
C.Azure IoT Hub
D.Azure Event Hubs
AnswerD

Event Hubs is optimized for high-throughput streaming data ingestion with buffering.

Why this answer

Azure Event Hubs is the correct choice because it is a fully managed, real-time data ingestion service optimized for high-throughput streaming data from millions of IoT devices. It provides at-least-once delivery, supports partitioning for massive scale, and integrates natively with Azure Synapse Analytics via the Synapse Pipeline or Event Hubs Capture to handle throughput spikes without data loss.

Exam trap

The trap here is that candidates often confuse Azure IoT Hub with Event Hubs, assuming IoT Hub is the default for all IoT streaming, but IoT Hub is designed for device management and lower-throughput telemetry, whereas Event Hubs is the dedicated high-throughput ingestion service for analytics pipelines.

How to eliminate wrong answers

Option A is wrong because Azure Data Lake Storage Gen2 is a hierarchical file storage service designed for batch and analytical workloads, not for real-time streaming ingestion; it lacks native event streaming and buffering capabilities. Option B is wrong because Azure Blob Storage is an object storage service for unstructured data, not built for low-latency, high-throughput event ingestion; it would require additional services like Event Hubs to capture streaming data. Option C is wrong because Azure IoT Hub is a managed service for bi-directional communication with IoT devices, but it is not optimized for high-volume streaming ingestion into Synapse; it is better suited for device management and command/control scenarios, and its default throughput is lower than Event Hubs for pure data ingestion.

249
Multi-Selectmedium

Which TWO strategies reduce data movement in Azure Synapse Analytics pipelines? (Choose two.)

Select 2 answers
A.Use Data Flow to transform data before loading
B.Use Stored Procedure activity to insert data
C.Use serverless SQL pool to query data in place
D.Use Copy activity to move data from source to staging
E.Use PolyBase to load data from external tables
AnswersC, E

Queries without moving data.

Why this answer

Serverless SQL pools in Azure Synapse Analytics allow you to query data directly from files in Azure Data Lake Storage or other external sources without moving the data into a dedicated SQL pool. This eliminates data movement entirely by using the compute resources of the serverless pool to process queries in place, leveraging the OPENROWSET or CREATE EXTERNAL TABLE syntax to read data from its original location.

Exam trap

The trap here is that candidates confuse 'reducing data movement' with 'optimizing data movement'—they may think Data Flow or Copy activity with staging reduces movement when in fact they still move data, whereas serverless SQL and PolyBase query data in place without relocation.

250
MCQmedium

You are building a data processing solution that requires exactly-once semantics when writing to Azure Event Hubs from Azure Stream Analytics. Which output format should you configure?

A.JSON
B.CSV
C.Parquet
D.Avro
AnswerD

Avro format in Stream Analytics provides exactly-once semantics when writing to Event Hubs.

Why this answer

Azure Stream Analytics supports exactly-once semantics when writing to Event Hubs only when using Avro serialization. This is because Avro provides a compact binary format with embedded schema, enabling Stream Analytics to track and deduplicate events precisely during output, which is required for exactly-once delivery. JSON, CSV, and Parquet do not support the necessary metadata and checkpointing mechanisms for exactly-once guarantees in this specific integration.

Exam trap

The trap here is that candidates often assume JSON is the default or most compatible format for streaming outputs, but they overlook that exactly-once semantics require a binary format with embedded schema support, which only Avro provides in this specific Azure Stream Analytics to Event Hubs integration.

How to eliminate wrong answers

Option A is wrong because JSON is a text-based format that does not support the schema evolution and binary encoding required for Stream Analytics to enforce exactly-once semantics to Event Hubs. Option B is wrong because CSV lacks schema information and binary encoding, making it impossible for Stream Analytics to guarantee exactly-once delivery due to potential data loss or duplication during serialization. Option C is wrong because Parquet is a columnar storage format optimized for analytics and batch processing, not for real-time streaming output to Event Hubs, and Stream Analytics does not support exactly-once semantics with Parquet output to Event Hubs.

251
MCQeasy

You are building a data pipeline in Azure Data Factory to copy data from an on-premises SQL Server database to Azure Blob Storage. The pipeline must run daily and handle incremental updates. The on-premises SQL Server table has a LastModifiedDate column that is updated when a row changes. What is the most efficient way to implement incremental loads?

A.Enable Change Data Capture (CDC) on the SQL Server database and use an ADF mapping data flow to read changes.
B.Use a Lookup activity to get the maximum LastModifiedDate from the destination, then use a Copy activity with a query that filters rows where LastModifiedDate > that value.
C.Use a tumbling window trigger with a window size of 1 day and copy all data from the source each time.
D.Perform a full load every day and use a Delete activity to remove duplicates.
AnswerB

This is the standard watermark pattern for incremental loads.

Why this answer

It uses a Lookup activity to retrieve the maximum LastModifiedDate from the destination (Azure Blob Storage), then passes that value as a parameter to a Copy activity that queries only rows where LastModifiedDate exceeds it. This minimizes data transfer by reading only new or changed rows, and it avoids the overhead of enabling Change Data Capture or performing full loads, making it the most efficient approach for incremental loads in Azure Data Factory.

Exam trap

The trap here is that candidates often overcomplicate the solution by choosing CDC (Option A) or full loads (Options C and D), when a simple watermark-based query using the existing LastModifiedDate column is the most efficient and straightforward approach for incremental loads in Azure Data Factory.

How to eliminate wrong answers

Option A is wrong because enabling Change Data Capture (CDC) on SQL Server introduces additional overhead and complexity, and using an ADF mapping data flow for CDC is less efficient than a simple query-based incremental load when a LastModifiedDate column exists. Option C is wrong because using a tumbling window trigger with a 1-day window and copying all data each time performs a full load every run, which is inefficient and wasteful for incremental updates. Option D is wrong because performing a full load daily and then using a Delete activity to remove duplicates is extremely inefficient, as it transfers all data every day and requires additional processing to identify and delete duplicates, negating the benefits of incremental loading.

252
MCQmedium

Refer to the exhibit. You have an Azure Data Factory pipeline that copies trade data from Azure Blob Storage to Azure SQL Database. The pipeline runs every hour and truncates the destination table before each copy. However, users report that data is missing during the copy window. What is the most likely cause?

A.The output dataset is not correctly configured
B.The writeBatchSize is too low, causing timeouts
C.The preCopyScript truncates the table before the copy completes, causing a period with no data
D.The source dataset is set to recursive, which includes unwanted files
AnswerC

Table is empty during copy.

Why this answer

The preCopyScript truncates the destination table before the copy operation begins. During the copy window, if the copy takes time or fails, the table remains empty, causing users to see missing data. Option A is incorrect because the output dataset configuration does not cause this issue.

Option B is incorrect because writeBatchSize affects performance but not data loss. Option D is incorrect because recursive setting affects source file selection, not data loss.

253
Multi-Selecthard

You are tuning a Spark job in Azure Synapse Analytics that processes large Parquet files. The job currently takes too long due to data skew. Which three actions can improve performance? (Choose three.)

Select 3 answers
A.Use bucketing on the join key when writing intermediate data.
B.Add a salt key to the join column to distribute the load.
C.Use coalesce to reduce the number of partitions.
D.Increase executor memory to handle larger partitions.
E.Repartition the data on the skewed column.
AnswersA, B, E

Bucketing pre-partitions data to avoid shuffle and reduce skew.

Why this answer

Options A, B, and E are correct. Repartitioning redistributes data. Salting adds a random key to break skew.

Bucketing pre-partitions data. Option C is wrong because coalesce reduces partitions but does not address skew. Option D is wrong because increasing executor memory may help but does not directly solve skew.

254
MCQeasy

Your team is developing a data processing solution using Azure Databricks. The data is stored in Delta Lake format in Azure Data Lake Storage Gen2. You need to ensure that when multiple jobs concurrently write to the same Delta table, the operations are atomic and consistent. Which Delta Lake feature should you use?

A.Enable Optimized Write on the Delta table.
B.Enable Auto Optimize on the Delta table.
C.Rely on Delta Lake's built-in ACID transactions.
D.Use Dynamic Partition Pruning in your Spark jobs.
AnswerC

Delta Lake provides ACID transactions, ensuring atomic and consistent concurrent writes.

Why this answer

Delta Lake provides built-in ACID (Atomicity, Consistency, Isolation, Durability) transactions that guarantee atomic and consistent concurrent writes. When multiple jobs write to the same Delta table, Delta Lake uses a transaction log (stored as JSON files in the `_delta_log` directory) to serialize writes, ensuring that each write is either fully committed or rolled back, preventing partial updates or data corruption.

Exam trap

The trap here is that candidates confuse performance-tuning features (Optimized Write, Auto Optimize, Dynamic Partition Pruning) with transactional guarantees, assuming they provide atomicity or consistency when they only address file layout or query speed.

How to eliminate wrong answers

Option A is wrong because Optimized Write is a performance feature that reduces the number of small files written by coalescing partitions, but it does not provide atomicity or consistency guarantees for concurrent writes. Option B is wrong because Auto Optimize is a Delta Lake feature that automatically compacts small files and optimizes file layout, but it does not handle transactional concurrency or atomicity. Option D is wrong because Dynamic Partition Pruning is a Spark SQL optimization that improves query performance by skipping irrelevant partitions during joins, not a mechanism for ensuring atomic or consistent concurrent writes.

255
MCQeasy

You are designing a data processing solution in Azure Databricks. The data is stored in Azure Data Lake Storage Gen2 and you need to perform transformations using Apache Spark. The security requirements mandate that all data in transit must be encrypted and that the storage account must not be accessible from the public internet. What should you configure?

A.Enable the storage account firewall and add a private endpoint for Azure Databricks to use.
B.Disable TLS on the storage account and use a shared access signature (SAS) token for authentication.
C.Use Azure Databricks with VNet injection and configure a service endpoint for the storage account.
D.Configure the storage account to use HTTPS only and enable firewall rules to allow only Azure services.
AnswerA

Private endpoint ensures private connectivity and encryption in transit.

Why this answer

It satisfies both security requirements: encrypting data in transit and preventing public internet access. A private endpoint uses Azure Private Link to connect Azure Databricks to the storage account over the Microsoft backbone network, ensuring all traffic stays within the Azure network and is encrypted via TLS. The storage account firewall is then configured to deny all public traffic, so only the private endpoint can access the storage account, meeting the 'not accessible from the public internet' mandate.

Exam trap

The trap here is confusing service endpoints (which still use public endpoints) with private endpoints (which use private IPs and fully isolate the resource from the internet), leading candidates to pick Option C thinking VNet injection plus service endpoints provides complete public internet isolation.

How to eliminate wrong answers

Option B is wrong because disabling TLS removes encryption in transit, violating the requirement that all data in transit must be encrypted; SAS tokens provide authentication but do not encrypt the channel. Option C is wrong because a service endpoint still exposes the storage account to the public internet (it only restricts source IPs via the firewall), so it does not meet the 'not accessible from the public internet' requirement; VNet injection alone does not enforce private connectivity. Option D is wrong because enabling HTTPS only and allowing only Azure services still leaves the storage account accessible from the public internet (Azure services can originate from public IPs), failing the requirement to block all public internet access.

256
MCQeasy

You need to execute a T-SQL stored procedure in Azure Synapse dedicated SQL pool that performs a large data load. The stored procedure takes approximately 45 minutes to run. You want to monitor the progress and see the statement text currently being executed. Which dynamic management view (DMV) should you query?

A.sys.dm_workload_management_workload_groups_stats
B.sys.dm_pdw_exec_requests (with status='running')
C.sys.dm_pdw_waits
D.sys.dm_pdw_exec_requests
AnswerB

Filtering by status='running' shows currently executing requests with the command text.

Why this answer

B is correct because `sys.dm_pdw_exec_requests` with the filter `WHERE status = 'running'` returns the currently executing requests in an Azure Synapse dedicated SQL pool, including the full statement text in the `command` column. This allows you to see the exact T-SQL being executed during the 45-minute data load, enabling progress monitoring.

Exam trap

The trap here is that candidates often choose D without the `status='running'` filter, thinking it shows current execution, but it returns all historical requests, requiring an additional filter to isolate the active one.

How to eliminate wrong answers

Option A is wrong because `sys.dm_workload_management_workload_groups_stats` provides statistics about workload group resource utilization (e.g., CPU, memory), not the currently executing statement text. Option C is wrong because `sys.dm_pdw_waits` shows wait information (e.g., locks, waits on resources) but does not include the statement text of running requests. Option D is wrong because `sys.dm_pdw_exec_requests` without the `status='running'` filter returns all requests (completed, failed, cancelled, etc.), not just the currently executing one, so you would not see the active statement text without additional filtering.

257
MCQhard

You are developing a data processing solution in Azure Synapse Analytics. The solution must support both batch and streaming data ingestion into a dedicated SQL pool. You need to ensure that data from streaming sources is available for queries within 5 seconds. Which approach should you use?

A.Use Azure Stream Analytics with a custom SQL function that writes directly to the dedicated SQL pool
B.Use Azure Databricks with Structured Streaming, write to Data Lake Storage, and then use PolyBase to load into SQL pool
C.Use Azure Data Factory with tumbling window triggers to load data from Event Hubs every 5 seconds
D.Use Event Hubs Capture to write to Data Lake Storage, then use PolyBase to load into the SQL pool every 5 seconds
AnswerA

Stream Analytics can achieve sub-second latency and write directly to SQL pool via a stored procedure.

Why this answer

Azure Stream Analytics can output directly to Azure Synapse Analytics dedicated SQL pool using the built-in Azure Synapse Analytics output adapter, which supports high-throughput, low-latency writes. This enables streaming data to be available for queries within seconds, meeting the 5-second latency requirement without intermediate storage or batch processing steps.

Exam trap

The trap here is that candidates often assume any pipeline involving Event Hubs or Data Lake Storage can achieve sub-5-second latency, but they overlook that batch-oriented components like PolyBase, Data Factory tumbling windows, or Capture introduce inherent delays that violate the strict latency requirement.

How to eliminate wrong answers

Option B is wrong because writing to Data Lake Storage and then using PolyBase adds batch latency (minutes) and does not guarantee sub-5-second availability. Option C is wrong because Azure Data Factory with tumbling window triggers is a batch-oriented orchestration tool, not designed for sub-5-second streaming latency, and Event Hubs integration via Data Factory does not support true streaming ingestion. Option D is wrong because Event Hubs Capture writes data in periodic batches (e.g., every 5 minutes or when a size threshold is met), and PolyBase loads are batch-based, both introducing latency far exceeding 5 seconds.

258
MCQeasy

You are using Azure Synapse Pipelines to perform an incremental load from Azure SQL Database to Azure Synapse Analytics. You need to identify rows that have changed since the last load. Which approach should you use?

A.Compare the current data with a snapshot using T-SQL MERGE.
B.Truncate and reload the entire table daily.
C.Use a watermark column such as LastModifiedDate.
D.Enable Change Data Capture (CDC) on the source table.
AnswerD

CDC captures all changes and supports incremental load.

Why this answer

Change Data Capture (CDC) on the source Azure SQL Database captures insert, update, and delete operations in change tables, enabling Azure Synapse Pipelines to efficiently identify only the changed rows since the last load. This approach minimizes data movement and processing overhead compared to full or snapshot-based comparisons, making it the recommended pattern for incremental loads in Synapse Pipelines.

Exam trap

The trap here is that candidates often choose the watermark column approach (Option C) because it seems simpler, but they overlook that CDC is the only option that natively captures all DML changes (including deletes) without requiring schema modifications or custom logic to handle edge cases like out-of-order updates.

How to eliminate wrong answers

Option A is wrong because comparing current data with a snapshot using T-SQL MERGE requires storing a full snapshot and performing row-by-row comparison, which is resource-intensive and does not leverage Synapse Pipelines' native incremental load capabilities. Option B is wrong because truncate and reload the entire table daily defeats the purpose of incremental loading, causing unnecessary full data transfer and processing, and is not a valid incremental approach. Option C is wrong because using a watermark column such as LastModifiedDate only captures updates to rows that have a timestamp updated, but it cannot detect deletes or changes to rows where the timestamp is not maintained, and it requires the source to reliably update the column on every change.

259
MCQhard

Refer to the exhibit. You created an external table in Azure Synapse Analytics serverless SQL pool to query Parquet files. Queries return no rows even though the files exist. What is the most likely issue?

A.The CREDENTIAL is missing
B.The FILE_FORMAT is incorrectly specified
C.The DATA_COMPRESSION is not supported for Parquet
D.The LOCATION path in the external table is relative to the data source, but the data source points to the wrong container or folder
AnswerD

The data source points to 'sales' container, table location adds 'parquet/sales', likely the files are not there.

Why this answer

The most likely issue is that the LOCATION path in the external table is relative to the data source. In the exhibit, the external data source points to the root container 'sales', and the external table LOCATION is 'parquet/sales/'. The combined path becomes 'sales/parquet/sales/', which is likely incorrect.

The correct data source should point to the container containing the 'parquet' folder, or the LOCATION should be adjusted. Option A is incorrect because CREDENTIAL is already defined. Option B is incorrect because FILE_FORMAT is specified and valid for Parquet.

Option C is incorrect because DATA_COMPRESSION is supported for Parquet.

260
MCQhard

You are using Azure Stream Analytics to process real-time temperature data from IoT devices. The output must be written to Azure SQL Database. The job has been running successfully for weeks, but recently you notice that the output data has duplicate rows. The input events are unique. The job uses a windowed aggregation (TumblingWindow). What is the most likely cause of duplicates?

A.The job is not handling late-arriving events.
B.The job is being restarted and reprocessing data.
C.The input event hub is receiving duplicate events.
D.The tumbling window size is too small.
AnswerB

Restart can cause reprocessing and duplicate output without idempotent writes.

Why this answer

When an Azure Stream Analytics job is restarted, it may reprocess data from the last checkpoint or from the beginning of the input stream, depending on the configured start time and output policy. This reprocessing can cause duplicate rows in the output, especially when using windowed aggregations like TumblingWindow, because the same events are aggregated again and written to Azure SQL Database without deduplication logic.

Exam trap

The trap here is that candidates often assume duplicates are caused by late-arriving events or input issues, but the core concept is that job restarts in Stream Analytics can reprocess data, and without idempotent output, duplicates are introduced.

How to eliminate wrong answers

Option A is wrong because late-arriving events are handled by the 'late arrival policy' in Stream Analytics, which can adjust window boundaries but does not inherently cause duplicates; duplicates arise from reprocessing, not from late data. Option C is wrong because the question explicitly states that input events are unique, so the input event hub is not receiving duplicate events. Option D is wrong because the tumbling window size being too small would affect the granularity of aggregations, not cause duplicate rows; duplicates are a result of reprocessing, not window size.

261
MCQeasy

You need to transform data in Azure Synapse Analytics using a language that supports procedural logic and error handling. Which option should you use?

A.T-SQL stored procedures
B.CREATE VIEW
C.PolyBase
D.CREATE EXTERNAL TABLE
AnswerA

Supports procedural logic and error handling.

Why this answer

T-SQL stored procedures are the correct choice because they support procedural logic (e.g., IF/ELSE, loops, TRY/CATCH) and error handling within Azure Synapse Analytics dedicated SQL pools. This allows you to encapsulate complex data transformation logic, handle runtime errors gracefully, and manage transactions, which is not possible with declarative objects like views or external tables.

Exam trap

The trap here is that candidates confuse PolyBase's ability to query external data with the ability to perform procedural transformations, overlooking that PolyBase is a query engine, not a programming construct for logic and error handling.

How to eliminate wrong answers

Option B is wrong because CREATE VIEW creates a read-only virtual table that cannot contain procedural logic or error handling; it is purely declarative. Option C is wrong because PolyBase is a data virtualization technology for querying external data sources (e.g., Azure Blob Storage) using T-SQL, but it does not support procedural logic or error handling itself. Option D is wrong because CREATE EXTERNAL TABLE defines a schema for external data but provides no procedural capabilities or error handling; it is a metadata object for PolyBase queries.

← PreviousPage 4 of 4 · 261 questions total

Ready to test yourself?

Try a timed practice session using only Develop data processing questions.