- A
Convert data to columnar file formats such as Parquet.
Columnar formats compress data and allow predicate pushdown, reducing I/O.
- B
Move data to the cool tier to reduce storage costs.
Why wrong: Cool tier increases read costs and may not be cost-effective for analytics workloads.
- C
Enable soft delete to protect against accidental deletion.
Why wrong: Soft delete does not improve performance or reduce costs.
- D
Use blob index tags to partition data logically.
Index tags enable efficient filtering and reduce the amount of data scanned.
- E
Consolidate small files into larger files (e.g., 100 MB or more).
Reducing the number of files reduces metadata operations and improves read throughput.
Question 567 of 851
Optimizing Azure Data Lake Storage Gen2 for Small Files
This DP-203 practice question tests your understanding of monitor and optimize data storage and processing. Match the stated requirement to the specific cloud service, access model, or configuration option — many options are valid in isolation but not for this scenario. A key principle to apply: columnar file formats. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
A data engineer is optimizing an Azure Data Lake Storage Gen2 account used for big data analytics. The account contains billions of small files (under 1 MB). The analytics jobs are slow and cost more than expected. Which THREE actions should the engineer take to improve performance and reduce costs?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Convert data to columnar file formats such as Parquet.
Converting data to columnar formats like Parquet (A) reduces the amount of data read during analytics, improving performance and reducing I/O costs. Using blob index tags (D) enables logical partitioning for faster query pruning. Consolidating small files into larger files (E) reduces metadata overhead and improves throughput. Moving data to cool tier (B) does not improve performance and may add retrieval costs. Enabling soft delete (C) increases data protection but does not address performance or cost issues.
Key principle: Columnar file formats
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Convert data to columnar file formats such as Parquet.
Why this is correct
Columnar formats compress data and allow predicate pushdown, reducing I/O.
Related concept
Columnar file formats
- ✗
Move data to the cool tier to reduce storage costs.
Why it's wrong here
Cool tier increases read costs and may not be cost-effective for analytics workloads.
- ✗
Enable soft delete to protect against accidental deletion.
Why it's wrong here
Soft delete does not improve performance or reduce costs.
- ✓
Use blob index tags to partition data logically.
Why this is correct
Index tags enable efficient filtering and reduce the amount of data scanned.
Related concept
Columnar file formats
- ✓
Consolidate small files into larger files (e.g., 100 MB or more).
Why this is correct
Reducing the number of files reduces metadata operations and improves read throughput.
Related concept
Columnar file formats
Common exam traps
Common exam trap: answer the scenario, not the keyword
The trap here is that candidates may think moving data to a cooler tier or enabling soft delete directly improves performance, when in reality these actions address cost or protection, not the root cause of slow analytics jobs caused by small file overhead.
Detailed technical explanation
How to think about this question
Columnar formats like Parquet use techniques such as predicate pushdown and compression (e.g., Snappy, Gzip) to minimize data scanned. Consolidating small files into larger ones (e.g., 100 MB or more) reduces the number of metadata operations and improves Spark or Hadoop job parallelism, as each file split incurs overhead. In Azure Data Lake Storage, the optimal file size aligns with the block size of the processing engine (e.g., 128 MB or 256 MB for Spark).
KKey Concepts to Remember
- Columnar file formats
- Blob index tags
- File consolidation
TExam Day Tips
- Watch for words such as best, first, most likely and least administrative effort.
- Review why wrong options are wrong, not only why the correct option is correct.
Key takeaway
Columnar file formats
Real-world example
How this comes up in practice
A startup's cloud architect reviews their monthly bill and notices costs are higher than expected for a long-running batch job. Switching from on-demand instances to Reserved Instances — or using Spot/Preemptible VMs — can reduce compute costs by up to 72 %. Questions like this test whether you understand the tradeoffs between commitment, flexibility, and cost across cloud pricing models.
Quick reference
Azure Blob Storage Tier Comparison
| Tier | Storage Cost | Retrieval Cost | Latency | Use Case |
|---|---|---|---|---|
| Hot | Highest | Lowest | Immediate | Active data, frequent reads |
| Cool | Lower | Higher | Immediate | Data accessed < once / month |
| Cold | Lower still | Higher | Immediate | Data accessed < once / quarter |
| Archive | Lowest | Highest + rehydration delay | Hours | Long-term compliance retention |
What to study next
Got this wrong? Here's your next step.
Review columnar file formats, then practise related DP-203 questions on the same topic to reinforce the concept.
- →
Monitor and optimize data storage and processing — study guide chapter
Learn the concepts, then practise the questions
- →
Monitor and optimize data storage and processing practice questions
Targeted practice on this topic area only
- →
All DP-203 questions
851 questions across all exam domains
- →
Microsoft Azure Data Engineer Associate DP-203 study guide
Full concept coverage aligned to exam objectives
- →
DP-203 practice test guide
How to use practice tests most effectively before exam day
Related practice questions
Related DP-203 practice-question pages
Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.
Secure, monitor, and optimize data storage and data processing practice questions
Practise DP-203 questions linked to Secure, monitor, and optimize data storage and data processing.
Design and develop data processing practice questions
Practise DP-203 questions linked to Design and develop data processing.
Design and implement data security practice questions
Practise DP-203 questions linked to Design and implement data security.
Monitor and optimize data storage and processing practice questions
Practise DP-203 questions linked to Monitor and optimize data storage and processing.
Design and implement data storage practice questions
Practise DP-203 questions linked to Design and implement data storage.
Develop data processing practice questions
Practise DP-203 questions linked to Develop data processing.
DP-203 fundamentals practice questions
Practise DP-203 questions linked to DP-203 fundamentals.
DP-203 scenario practice questions
Practise DP-203 questions linked to DP-203 scenario.
DP-203 troubleshooting practice questions
Practise DP-203 questions linked to DP-203 troubleshooting.
Practice this exam
Start a free DP-203 practice session
Short sessions build daily habit. Longer sessions build exam-day stamina. Try a timed session to simulate real conditions.
FAQ
Questions learners often ask
What does this DP-203 question test?
Monitor and optimize data storage and processing — This question tests Monitor and optimize data storage and processing — Columnar file formats.
What is the correct answer to this question?
The correct answer is: Convert data to columnar file formats such as Parquet. — Converting data to columnar formats like Parquet (A) reduces the amount of data read during analytics, improving performance and reducing I/O costs. Using blob index tags (D) enables logical partitioning for faster query pruning. Consolidating small files into larger files (E) reduces metadata overhead and improves throughput. Moving data to cool tier (B) does not improve performance and may add retrieval costs. Enabling soft delete (C) increases data protection but does not address performance or cost issues.
What should I do if I get this DP-203 question wrong?
Review columnar file formats, then practise related DP-203 questions on the same topic to reinforce the concept.
What is the key concept behind this question?
Columnar file formats
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on DP-203
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. You are a data engineer for a retail company. The company uses Azure Data Lake Storage Gen2 to store raw transaction data partitioned by date. Each day, a folder is created with the format 'YYYY/MM/DD' containing thousands of small JSON files (each ~10 KB). An Azure Databricks job runs daily to read the previous day's folder, transform the data, and write to a Delta table for reporting. Over time, the job's execution time has increased from 15 minutes to over 2 hours. The job uses a cluster with 4 nodes (each 16 GB memory). Monitoring shows that the job spends most of its time in the 'listing files' stage. Which optimization should you implement to reduce the job duration?
hard- A.Increase the number of nodes in the cluster to 16.
- B.Change the output format from JSON to Delta and enable Delta caching.
- ✓ C.Pre-process the raw data to coalesce small JSON files into larger parquet files (e.g., 256 MB each).
- D.Use Azure Data Factory instead of Databricks to copy the raw data.
Why C: The job spends most of its time in the 'listing files' stage because reading thousands of small JSON files (each ~10 KB) from Azure Data Lake Storage Gen2 incurs high metadata operation overhead. Coalescing these small files into larger Parquet files (e.g., 256 MB each) reduces the number of files that Spark must list and process, dramatically cutting down the listing stage time and improving overall throughput.
Variation 2. You are a data engineer for a financial services company. You manage an Azure Data Lake Storage Gen2 account that stores real-time stock trade data ingested from Azure Event Hubs via Azure Stream Analytics. The data is partitioned by date and symbol. Each day, a downstream Azure Databricks job runs an ETL process to aggregate trades into 5-minute bars and writes the results to a separate container. The Databricks job runs on a cluster with 10 worker nodes (Standard_DS3_v2) using Auto-Scaling enabled (2-10 workers). Recently, the job has been taking longer than expected, and you observe that the cluster is often at 10 workers but still the job duration increased by 30%. The storage account shows high transaction costs. You suspect the issue is related to how data is read. What should you do to optimize the job's performance and reduce costs?
medium- A.Convert the data to Avro format to reduce file size.
- B.Increase the maximum number of workers to 20 and use a larger instance type.
- ✓ C.Modify the Stream Analytics job to output larger files (e.g., set the minimum file size to 100 MB) and use coalesce in Databricks to reduce the number of output partitions.
- D.Move the data to Azure Blob Storage Premium tier to reduce latency.
Why C: Option C is correct because the performance issue stems from reading many small files (small file problem) in Azure Data Lake Storage Gen2, which increases transaction costs and slows down Spark jobs. By configuring Stream Analytics to output larger files (e.g., minimum 100 MB) and using coalesce in Databricks to reduce output partitions, you minimize the number of files read/written, reducing overhead and transaction costs. This directly addresses the root cause—high transaction costs and cluster saturation at 10 workers—without unnecessary scaling or tier changes.
Keep practising
More DP-203 practice questions
- You are designing a data storage solution for IoT sensor data. The data is written thousands of times per second and req…
- A data processing job in Azure Synapse Analytics writes results to a table in the dedicated SQL pool. After a failure, t…
- A multinational corporation uses Azure Data Lake Storage Gen2 to store petabytes of parquet files partitioned by date an…
- A company ingests streaming data from IoT devices into Azure Event Hubs. The data must be processed in near real-time to…
- Which THREE factors should be considered when choosing between Azure Stream Analytics and Azure Databricks for a real-ti…
- You are designing a data lake on Azure Data Lake Storage Gen2. The data will be used by both batch processing (Spark) an…
Last reviewed: Jun 11, 2026
This DP-203 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DP-203 exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.