A data engineering team needs to build a batch processing pipeline that transforms large volumes of sales data stored in Azure Data Lake Storage Gen2. The transformations include aggregations and joins, and the output should be stored back in the data lake as Parquet files. The team wants a serverless compute option that automatically scales and charges per second. Which Azure service should they use?
Answer choices
Why each option matters
Good practice is not just finding the correct option. The wrong answers often show the exact trap the exam wants you to fall into.
Distractor review
Azure Synapse Analytics dedicated SQL pool
Dedicated SQL pool is provisioned compute, not serverless. It charges for the provisioned resources regardless of usage, and does not auto-scale seamlessly.
Best answer
Azure Databricks with auto-scaling clusters
Azure Databricks offers auto-scaling clusters and serverless compute options that scale down to zero, charging per second for the resources consumed, ideal for batch transformations on data lakes.
Distractor review
Azure Data Factory with mapping data flows
Mapping data flows are serverless and scale automatically, but they are best for ETL transformations with a visual interface. For complex custom logic (e.g., multi-step aggregations and joins), Databricks provides more flexibility.
Distractor review
Azure Stream Analytics
Stream Analytics is designed for real-time stream processing, not batch processing of large volumes of data stored in a data lake.
Common exam trap
Common exam trap: answer the scenario, not the keyword
Many certification questions include familiar terms but test a specific constraint. Read the exact wording before choosing an answer that is generally true but wrong for this case.
Technical deep dive
How to think about this question
This question should be treated as a scenario, not a definition check. Identify the problem, the constraint and the best action. Then compare each option against those facts.
KKey Concepts to Remember
- Read the scenario before looking for a memorised answer.
- Find the constraint that changes the correct option.
- Eliminate answers that are true in general but not in this case.
- Use explanations to understand the rule behind the answer.
TExam Day Tips
- Underline the problem statement mentally.
- Watch for words such as best, first, most likely and least administrative effort.
- Review why wrong options are wrong, not only why the correct option is correct.
Related practice questions
Related DP-900 practice-question pages
Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.
More questions from this exam
Keep practising from the same exam bank, or move into a focused topic page if this question exposed a weak area.
Question 1
A data engineer needs to process streaming data from IoT devices and store the results in Azure Data Lake Storage for long-term analytics. The data must be processed in near real-time to detect anomalies and trigger alerts. Which Azure service should the engineer use for stream processing?
Question 2
A data engineer needs to query data stored in CSV files in Azure Data Lake Storage Gen2 using T-SQL in Azure Synapse Analytics, without loading the data into the database. Which feature should they use?
Question 3
A data engineer needs to process raw clickstream data from multiple websites that is stored in Azure Blob Storage as JSON files. The processing must run automatically every hour, transform the data into a structured format for reporting, and handle schema changes in the source data without manual intervention. Which Azure service should be used?
Question 4
A data engineer is designing a data lake architecture in Azure. They plan to first ingest raw data from various sources into a landing zone in Azure Data Lake Storage Gen2. Then they will clean, validate, and deduplicate that data in a second zone. Finally, they will create aggregated, business-ready datasets in a third zone for analysts. This layered approach is known as which architecture?
Question 5
A data engineer needs to transform large datasets stored in Azure Data Lake Storage Gen2 using Python and Apache Spark. They want a serverless compute option that automatically scales and requires no cluster management. Which Azure service should they use?
Question 6
A company collects customer feedback forms. Each form contains always-present fields like CustomerID and SubmissionDate, but also a free-text Comments field and optional fields like Rating or ProductCategory that vary between forms. How should this data be classified?
FAQ
Questions learners often ask
What does this DP-900 question test?
Read the scenario before looking for a memorised answer.
What is the correct answer to this question?
The correct answer is: Azure Databricks with auto-scaling clusters — Azure Databricks with auto-scaling clusters (or serverless SQL warehouses) provides a serverless compute model for big data transformations. It automatically scales based on workload and charges per second (in the case of serverless SQL warehouses). Dedicated SQL pool is provisioned and does not charge per second; it requires sizing and management. Data Factory mapping data flows are serverless and charge per data flow execution, but they are less flexible for complex custom logic compared to Databricks. Stream Analytics is for real-time stream processing, not batch transformations. Therefore, the best fit for ad-hoc large-scale transformations with serverless compute is Azure Databricks.
What should I do if I get this DP-900 question wrong?
Then try more questions from the same exam bank and focus on understanding why the wrong options are tempting.
Discussion
Sign in to join the discussion.