You are a data architect at a global retail company. The company has an Azure Data Lake Storage Gen2 account that stores petabytes of clickstream data. They need to provide near real-time analytics dashboards for regional managers. The data arrives in batches every 5 minutes. Currently, they use Azure Databricks to transform the data and load it into Azure Synapse Analytics, but the dashboards show data that is 30 minutes old. The business requires dashboards to reflect data within 10 minutes of ingestion. You propose a new solution. Which approach should you recommend?
Stream Analytics provides low-latency streaming to Power BI.
Why this answer
Option C is correct because it uses Azure Event Hubs for low-latency ingestion and Azure Stream Analytics for real-time processing, enabling near real-time dashboards in Power BI with sub-minute latency. This architecture bypasses the batch-oriented pipeline that causes the current 30-minute delay, meeting the 10-minute requirement.
Exam trap
The trap here is that candidates may assume batch tools like Data Factory or Databricks can be tuned to meet near real-time SLAs, but they fundamentally operate on file-based or micro-batch paradigms that cannot match the sub-minute latency of a true streaming pipeline with Event Hubs and Stream Analytics.
How to eliminate wrong answers
Option A is wrong because replacing Synapse with Azure Analysis Services does not address the root cause of latency—the batch processing in Databricks—and Analysis Services is an OLAP engine that still requires data to be loaded, not a streaming solution. Option B is wrong because Azure Data Factory with tumbling window triggers is a batch-oriented orchestration tool that introduces inherent latency from window scheduling and data movement, failing to achieve sub-10-minute freshness. Option D is wrong because increasing Databricks clusters and using Auto Loader only accelerates the batch transformation step but does not eliminate the fundamental batch processing delay, and Auto Loader still operates on file arrival, not streaming.