DP-900 Describe an analytics workload on Azure Practice Question
A data engineering team needs to transform large datasets stored in Azure Data Lake Storage Gen2 using Apache Spark with Python code. They want a fully managed service that provides serverless Spark pools, meaning no clusters to manage and automatic scaling. Which Azure service should they use?
⚠ Common exam trap
A common mix-up: candidates confuse Azure Databricks as the only serverless Spark option, but Azure Synapse Analytics also offers serverless Spark pools that are fully managed and integrated with Azure Data Lake Storage Gen2, making it the correct answer for this specific scenario.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Azure Synapse Analytics with serverless Spark pools
Azure Synapse Analytics with serverless Spark pools is the correct choice because it provides a fully managed, serverless Apache Spark environment that automatically scales and eliminates the need to manage clusters. This service directly supports transforming large datasets in Azure Data Lake Storage Gen2 using Python code with Spark, meeting the team's requirement for a no-cluster-management, auto-scaling solution.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Azure HDInsight
Why it's wrong here
Azure HDInsight is a managed big data analytics service that provisions a cluster of VMs and requires you to configure, scale, and patch the cluster nodes yourself. It is not serverless because you are responsible for the cluster lifecycle and you continue to pay for the underlying VMs even when no jobs are running. For transforming large datasets on-demand, HDInsight's operational overhead and fixed cluster costs make it less suitable than a solution that offers automatic scaling and per-second billing.
- ✗
Azure Databricks
Why it's wrong here
Azure Databricks is a powerful Apache Spark-based platform that can be run in a 'serverless compute' mode, but it is a separate multi-cloud offering from the Databricks Lakehouse Platform with its own workspace, cluster policies, and pricing model. In Azure, Synapse Analytics provides serverless Spark pools natively within the same workspace, whereas Databricks often requires attaching an explicit compute resource and separate management. While Databricks can definitely transform large datasets, it is not the same as Azure Synapse's serverless Spark and is considered a distinct product.
- ✓
Azure Synapse Analytics with serverless Spark pools
Why this is correct
Azure Synapse Analytics with serverless Spark pools is correct because it provides on-demand Apache Spark compute that auto-starts, scales automatically, and shuts down when idle, so you do not provision or manage any cluster. You are billed only for the compute resources consumed during job execution (per second), which makes it well-suited for transformation and exploration of large datasets stored in Azure Data Lake Storage. The Spark engine executes directly on the data in place, enabling interactive, scalable pipelines without infrastructure management.
- ✗
Azure Machine Learning
Why it's wrong here
Azure Machine Learning is designed for the end-to-end machine learning lifecycle, including model training, deployment, and monitoring, not as a general-purpose data transformation engine. While you can run data preprocessing in Azure Machine Learning notebooks or pipelines, doing so requires attaching or creating separate compute resources and does not provide the simplified, on-demand Spark transformation experience of Synapse serverless pools. Therefore it is not the intended or most direct service for transforming large datasets at scale for a data engineering team.
Quick reference
Cloud Service Model Comparison
| Model | You Manage | Provider Manages | Examples |
|---|---|---|---|
| IaaS | OS, runtime, apps, data | Hardware, hypervisor, networking | EC2, Azure VMs, GCP Compute Engine |
| PaaS | Apps and data | OS, runtime, middleware, hardware | Elastic Beanstalk, Azure App Service |
| SaaS | Data and settings only | Everything else | Microsoft 365, Salesforce, Workday |
| FaaS / Serverless | Function code only | Infra, scaling, runtime | Lambda, Azure Functions, Cloud Run |
| CaaS | Containers and apps | Kubernetes, OS, hardware | EKS, AKS, GKE |
Go deeper
Related to this question
Learn chapter
Data Roles and Core Concepts
Key term
Data Lake Storage Gen2
Data Lake Storage Gen2 is a cloud-based storage service that combines a scalable data lake with enterprise-grade file system capabilities for big data analytics.
Key term
Data lake
A data lake is a centralized storage repository that holds vast amounts of raw data in its native format until it is needed for analysis.
About these practice questions
One of 820 original DP-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DP-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DP-900 exam.