DP-203 Develop data processing Practice Test 21 — 15 Questions

Question 1

You are designing a batch processing solution for a data lake. Source files arrive daily in Parquet format in Azure Data Lake Storage Gen2. The data must be cleaned, aggregated, and loaded into an Azure Synapse SQL pool. The solution should minimize compute costs and management overhead. Which technology should you use for the transformation?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Azure Synapse Pipelines with mapping data flows.. Azure Synapse Pipelines with mapping data flows provide a serverless, code-free transformation service that runs on managed Spark clusters, minimizing management overhead and costs. Mapping data flows can directly read Parquet from ADLS Gen2, perform cleaning and aggregation, and load into Azure Synapse SQL pool without requiring cluster management. Option A (Azure HDInsight with Spark) requires manual cluster provisioning and management, increasing operational overhead. Option C (custom SSIS package) is legacy, not cloud-native, and requires an integration runtime for execution. Option D (Azure Databricks with Auto Loader) provides powerful stream and batch processing but incurs higher costs for a simple batch job due to cluster management and DBU consumption.

Answer

Azure HDInsight with Spark jobs scheduled in Azure Data Factory.

Answer

Azure Data Factory with a custom SSIS package.

Answer

Azure Databricks with an Auto Loader pipeline.

Question 2

You are building a real-time dashboard to monitor user activity on a website. The data is ingested via Azure Event Hubs and must be aggregated every minute with a 30-second late-arrival tolerance. The aggregated results should be stored in Azure Cosmos DB for low-latency reads. Which Azure service should you use to perform the windowed aggregation?

Accepted Answer

Azure Stream Analytics with a tumbling window of 1 minute and a late-arrival policy of 30 seconds.. Azure Stream Analytics is the correct choice because it natively supports windowed aggregations (tumbling, hopping, sliding, session) and allows you to define a late-arrival policy to handle out-of-order events. A tumbling window of 1 minute with a late-arrival tolerance of 30 seconds meets the requirement exactly, and the output can be directly written to Azure Cosmos DB for low-latency reads.

Answer

Azure Functions triggered by Event Hubs to aggregate data and write to Cosmos DB.

Answer

Azure Databricks with structured streaming and a sliding window.

Answer

Azure Analysis Services to process streaming data directly from Event Hubs.

Question 3

You are optimizing a data pipeline in Azure Synapse Analytics that loads data from a CSV file in ADLS Gen2 into a dedicated SQL pool using PolyBase. The load is slow and you need to improve performance. Which action would be MOST effective?

Accepted Answer

Combine the CSV files into fewer, larger files before loading.. Combining many small CSV files into fewer, larger files reduces the number of file open/close operations and minimizes the overhead of PolyBase's external file enumeration and parallel split logic. PolyBase performs best when each file is at least 256 MB, as it can then assign a full file to each distribution, avoiding the overhead of splitting tiny files across multiple threads.

Answer

Increase the service level (DWU) of the dedicated SQL pool.

Answer

Change the file format from CSV to Avro.

Answer

Use Azure Data Factory to stage the data in Azure Blob Storage before loading.

Question 4

You have a mission-critical pipeline that processes financial transactions in Azure Synapse Analytics. The pipeline uses Azure Data Factory with a mapping data flow to transform data. You need to ensure high availability and minimal data loss in case of a regional failure. What should you implement?

Accepted Answer

Store the source data in an RA-GRS storage account and use Azure Data Factory to copy from the secondary endpoint.. Using an RA-GRS storage account ensures geo-redundancy with a readable secondary endpoint, and Azure Data Factory can copy data from that secondary endpoint in case of a regional failure. This provides high availability and minimal data loss for the source data, while the mapping data flow transformation can proceed after the copy. Option D is incorrect because Azure Synapse Link for Cosmos DB changes the architecture (using Cosmos DB as source) and does not directly provide high availability for the existing ADF pipeline with mapping data flows; it introduces a different technology stack that may not meet the requirement of pipeline resilience.

Answer

Configure the pipeline to retry on failure and manually restore from backup.

Answer

Use Azure SQL Database active geo-replication as the source.

Answer

Use Azure Synapse Link for Cosmos DB to enable near real-time analytics with multi-region writes.

Question 5

You need to transform semi-structured JSON data into a tabular format for analysis in Azure Synapse Analytics. The data is stored in ADLS Gen2. Which feature should you use to query the JSON data directly without loading it into a table?

Accepted Answer

Use the OPENROWSET function in a serverless SQL pool.. OPENROWSET in Synapse serverless SQL can query JSON files directly. Option B (Azure Data Factory) flattens JSON but is an orchestration tool, not a direct query method. Option C (PolyBase) requires external tables. Option D (COPY INTO) loads data into a table, not direct query.

Answer

Use Azure Data Factory to flatten the JSON and store as Parquet.

Answer

Create an external table using PolyBase.

Answer

Use the COPY INTO command in a dedicated SQL pool.

Question 6

You are designing a data processing solution for an e-commerce company. The company receives millions of clickstream events per hour from their website and needs to aggregate the data by product category and windowed time intervals for real-time dashboards. You need to minimize latency and cost. Which service should you use?

Accepted Answer

Azure Stream Analytics. (Azure Stream Analytics) is the best choice because it is purpose-built for real-time stream processing, supports windowed aggregations, and integrates with Power BI for dashboards. Option A (Azure Databricks Structured Streaming) can handle streaming but is more complex and typically more expensive for simple aggregations. Option B (Azure Data Factory) is for batch data movement, not real-time. Option D (Azure Synapse Pipelines) is for orchestrating data movement, not real-time processing.

Answer

Azure Databricks Structured Streaming

Answer

Azure Data Factory

Answer

Azure Synapse Pipelines

Question 7

You are designing a real-time analytics solution for IoT devices that emit telemetry data every second. The data must be aggregated every minute and stored in Azure SQL Database for historical analysis. You need to minimize latency and operational overhead. Which approach should you recommend?

Accepted Answer

Use Azure Stream Analytics with a tumbling window of 1 minute and output to Azure SQL Database. Azure Stream Analytics natively supports real-time stream processing with tumbling windows, allowing you to aggregate IoT telemetry data every minute and output directly to Azure SQL Database with minimal latency. This approach avoids the overhead of managing clusters (Databricks) or orchestrating batch loads (Data Factory), directly meeting the requirement for low latency and operational simplicity.

Answer

Use Azure Databricks with Structured Streaming to aggregate and write to SQL Database

Answer

Use Event Hubs Capture to store raw data in blob storage, then use Azure Data Factory to load into SQL Database hourly

Answer

Use Azure Functions to process events and write to SQL Database

Question 8

You are troubleshooting a slow-running pipeline in Azure Data Factory that uses a Copy activity to transfer data from Azure Blob Storage to Azure Synapse Analytics. The pipeline processes about 100 GB of CSV files. The copy performance is poor even though the source and sink are in the same region. What is the most likely cause?

Accepted Answer

The copy activity is not using staging and PolyBase. The Copy activity in Azure Data Factory uses PolyBase or COPY statement (staging) to bulk load data into Azure Synapse Analytics. Without staging and PolyBase, the default insert method is row-by-row, which is extremely slow for large datasets like 100 GB of CSV files. Enabling staging with PolyBase allows parallel, high-throughput loading, which is essential for performance at this scale.

Answer

The source and sink are in different Azure regions

Answer

The Data Integration Unit (DIU) setting is too low

Answer

The source files are compressed

Question 9

You need to process streaming data from Azure Event Hubs and store the results in Azure Cosmos DB for a real-time dashboard. The solution must handle duplicate events and ensure exactly-once processing. Which Azure service should you use?

Accepted Answer

Azure Stream Analytics. (Azure Stream Analytics) is correct because it provides exactly-once processing when configured with Event Hubs and Cosmos DB output. Option A (Azure Data Factory) is batch-oriented. Option B (Azure Functions) may have at-least-once guarantees. Option D (Azure Databricks) can achieve exactly-once but requires more configuration.

Answer

Azure Data Factory

Answer

Azure Functions with Event Hubs trigger

Answer

Azure Databricks Structured Streaming

Question 10

Your team uses Azure Synapse Analytics serverless SQL pool to query Parquet files in Azure Data Lake Storage Gen2. The query performance is inconsistent, and some queries take a long time to execute. You need to improve query performance. What should you do?

Accepted Answer

Create statistics on the columns used in joins and filters. (Create statistics on the columns used in joins and filters) is correct because serverless SQL pool relies on statistics for optimal query plans. Option A (Increase the maximum degree of parallelism) is not directly applicable. Option C (Convert to CSV) would degrade performance. Option D (Use a dedicated SQL pool) may be an option but not the best immediate step.

Answer

Increase the MAXDOP setting in the query

Answer

Move the data to a dedicated SQL pool

Answer

Convert the Parquet files to CSV format

Question 11

You are designing a data processing pipeline that ingests data from a REST API endpoint every hour. The API returns JSON data with a varying schema. You need to store the raw data in Azure Data Lake Storage Gen2 and later process it using Azure Databricks. Which file format should you use for the raw data storage?

Accepted Answer

JSON. C is correct because the raw data arrives from a REST API with a varying JSON schema, and storing it in JSON format preserves the exact structure and schema variability without data loss or transformation. JSON is schema-on-read, meaning the raw data can be ingested as-is into Azure Data Lake Storage Gen2 and later processed by Azure Databricks, which natively supports JSON parsing. This avoids premature schema enforcement that would occur with columnar or binary formats.

Answer

Parquet

Answer

CSV

Answer

Avro

Question 12

Refer to the exhibit. You have an Azure Data Factory pipeline that copies trade data from Azure Blob Storage to Azure SQL Database. The pipeline runs every hour and truncates the destination table before each copy. However, users report that data is missing during the copy window. What is the most likely cause?

Accepted Answer

The preCopyScript truncates the table before the copy completes, causing a period with no data. The preCopyScript truncates the destination table before the copy activity begins writing data, creating a window where the table is empty or partially populated. Since the pipeline runs every hour and users query the table during the copy window, they see missing data until the copy completes. This is the classic 'truncate-then-load' anti-pattern in ADF that causes data availability gaps. The correct fix is to use a staging table with an atomic swap or to use incremental loading instead of truncation.

Answer

The output dataset is not correctly configured

Answer

The writeBatchSize is too low, causing timeouts

Answer

The source dataset is set to recursive, which includes unwanted files

Question 13

Refer to the exhibit. You have a mapping data flow in Azure Data Factory that aggregates sales data. The data flow runs successfully but the sink table contains only the total sum per run instead of per product. What is missing?

Accepted Answer

The aggregate transformation does not have a groupBy column. In an ADF mapping data flow, the Aggregate transformation computes aggregates based on the columns listed in the Group By tab. If no groupBy column is specified, the transformation treats the entire dataset as a single group, producing one row with the total sum. Adding the product column to groupBy makes the aggregation produce one row per product.

Answer

The source dataset is not filtering by date

Answer

The data flow is missing a filter transformation

Answer

The sink dataset is not configured to append

Question 14

Your organization uses Azure Synapse Analytics. You need to design a data transformation pipeline that processes streaming data from Azure Event Hubs, performs aggregations over a 5-minute tumbling window, and loads the results into a dedicated SQL pool table. Which Azure service should you use to implement the streaming transformation?

Accepted Answer

Azure Stream Analytics. Azure Stream Analytics is the appropriate service for real-time stream processing with windowed aggregations. Option B is wrong because Azure Data Factory is for batch orchestration. Option C is wrong because Spark Structured Streaming is for big data workloads but less integrated with SQL pools. Option D is wrong because Azure Functions is not designed for streaming windowed aggregations.

Answer

Azure Data Factory

Answer

Apache Spark for Azure Synapse

Answer

Azure Functions

Question 15

You are building a data processing pipeline in Azure Synapse Analytics. The pipeline should read data from Azure Data Lake Storage Gen2 (Parquet files), apply transformations using a mapping data flow, and write the results to a dedicated SQL pool table. The source data contains personally identifiable information (PII). You need to mask the PII columns (e.g., email) using a data masking function within the data flow. Which transformation should you use?

Accepted Answer

Derived Column transformation. The Derived Column transformation in mapping data flows allows you to create new columns or modify existing ones using expressions, including built-in data masking functions like `mask()`, `maskEmail()`, or `substring()`. This is the correct transformation to apply PII masking on columns such as email addresses within the data flow pipeline before writing to the dedicated SQL pool.

Answer

Join transformation

Answer

Aggregate transformation

Answer

Pivot transformation