How do you decide where to put all the different types of data a modern business generates—customer names, product images, sales logs, live chat messages—so it's both cheap to store and instantly available when needed? That is the core challenge of 'Designing Data Storage Solutions', a key exam objective for the DP-203. This chapter breaks down the choices you have in Microsoft Azure, so you can build a storage plan that is fast, secure, and cost-effective, just like a professional data engineer would.
Jump to a section
A simple way to picture Design Data Storage Solutions
Have you ever tried to organise a massive collection of old photos, school projects, and tax documents, only to realise that some things need to be kept forever, some need to be easy to search through, and some just need to be stored cheaply in the basement?
That is exactly the problem cloud data storage solves, but on a global scale. Think of a large office block with different types of storage rooms on each floor. On the ground floor, there is a public noticeboard where anyone can pin a note, but it gets crowded quickly and is only good for small, temporary messages. This is like 'Azure Blob Storage' for unstructured data such as images or videos. On the first floor, there are thousands of secure, lockable filing cabinets lined up in rows. Each cabinet has drawers labelled with precise names—'Customer-Orders-2024', 'Employee-Payroll', 'Product-Catalogue'. To find something, you need the exact drawer name and key. This is like 'Azure Cosmos DB', a database that stores structured data and lets you look it up by a unique ID, really fast. On the top floor, there is a giant, shared warehouse with shelves reaching the ceiling. You can store boxes of any shape or size here, but the warehouse manager will charge you per box per month, especially if you ask for a specific box quickly. This is 'Azure Data Lake Storage' for huge amounts of raw data like log files or sensor readings.
When you design a data storage solution, you decide which kind of 'room' to put each type of data into, how to arrange the keys (access permissions), and whether you need a backup copy in a second building in another city to protect against a fire. Just like you wouldn't store your tax returns on a public noticeboard, you wouldn't store live chat messages in a slow warehouse. The skill is matching the data's needs—speed, size, structure, and security—to the right virtual 'room'.
When you design a data storage solution in Azure, you are really answering one fundamental question: what kind of 'container' does each piece of data need? The answer depends on three factors: the structure of the data, how often you need to read or write it, and how much you are willing to pay.
Let us start with the main types of data storage in Azure. The first is 'Azure Blob Storage'. 'Blob' stands for 'Binary Large Object'. Think of it as a giant, online hard drive for files like photos, videos, log files, and backups. It is great for unstructured data—data that has no fixed format or internal structure that a database understands. You can store billions of blobs, and you can access them from anywhere over the internet. Azure Blob Storage offers three 'access tiers' to save money: Hot (for data you use every day, costs more to store but less to retrieve), Cool (for data you use maybe once a month), and Archive (for data you keep for years, costs very little to store but takes hours to retrieve).
The second major type is 'Azure Data Lake Storage Gen2'. This is a superset of Blob Storage, specially built for 'big data' analytics—huge datasets that you process using tools like Apache Spark or Hadoop. It combines the cheap storage of blob with the hierarchical folder structure of a file system, so a data engineer can organise terabytes of raw data into folders like '/raw/year=2024/month=01/'. It also supports a 'POSIX' permission system, meaning you can set very fine-grained access controls for users and applications, just like on a Linux server.
The third category is 'Azure Files'. This provides a fully managed 'Server Message Block' (SMB) or 'Network File System' (NFS) file share in the cloud. Imagine a shared drive on your office network that multiple people can access at the same time. Azure Files lets you do exactly that, but without managing the physical server. It is perfect for 'lift-and-shift' migrations—moving an old on-premises application to the cloud without rewriting its code—because the app can just mount the cloud drive as if it were local.
The fourth type is 'Azure Disks'. These are 'block-level' storage volumes that are attached to virtual machines (VMs). When you create a VM in Azure, you assign it a disk for the operating system and one for data. This is not a file share; it is a raw disk that the VM's operating system sees as a local hard drive. Disks are either managed by Azure ('Azure Managed Disks') or unmanaged. They come in different performance tiers: Standard HDD (cheapest, slowest), Standard SSD (good for general use), Premium SSD (fast, for production databases), and Ultra Disk (extremely fast, for high-performance applications like SAP HANA).
Now, what about databases? The exam also focuses on choosing the right 'database platform' for your data. Azure offers two main types: 'relational' and 'NoSQL'. Relational databases, like 'Azure SQL Database', store data in tables with rows and columns. They enforce a schema (a strict definition of what data goes in each column), and they use 'Structured Query Language' (SQL) to query data. They are perfect for transactional systems—like a banking app where you must ensure that an account balance updates accurately.
On the other side, you have 'Azure Cosmos DB', which is a 'NoSQL' database. NoSQL means 'Not Only SQL'. It is designed for massive scale and global distribution. You can replicate your data across multiple Azure regions around the world, and users get low-latency access from the closest region. Cosmos DB supports multiple 'data models': key-value (like a giant dictionary), document (JSON files), graph (relationships between data points), and column-family (similar to Cassandra). You choose a model based on how your application queries the data. For example, a gaming app that needs to store player profiles and look them up by player ID is a perfect use case for the key-value model.
Finally, there is 'Azure Synapse Analytics' (formerly SQL Data Warehouse). This is a 'massively parallel processing' (MPP) system for 'data warehousing'—combining data from multiple sources to run complex analytical queries. While Azure SQL Database is for transactional workloads (many small, fast reads and writes), Synapse is for analytical workloads (scanning billions of rows to answer questions like 'what was our total sales per region last quarter?'). It separates 'compute' (processing power) from 'storage' (where the data lives), meaning you can pause the compute to save money when nobody is running queries, while the data remains safely stored.
Choosing between all these options is called 'data storage design'. As an Azure data engineer, your job is to map business requirements to the right Azure service. If an app needs to store billions of IoT sensor readings and query them by time range, you might pick Cosmos DB with a time-to-live (TTL) setting to automatically delete old data. If a finance team needs a monthly report that joins data from sales, inventory, and HR systems, you would build a data pipeline into Synapse Analytics. If a team of data scientists needs to experiment with terabytes of raw CSV files, you would land them in Data Lake Storage Gen2 with proper folder structure and access controls.
Analyse Data Characteristics
First, identify the nature of each dataset. Is it structured (rows and columns like a spreadsheet), semi-structured (JSON logs), or unstructured (images, videos)? What is the expected volume (gigabytes or petabytes)? How frequently is it accessed (every second or once a year)? This analysis determines the next step's service selection.
Match Service to Pattern
Based on the analysis, choose the appropriate Azure service. For relational transactional data, pick Azure SQL Database. For globally distributed NoSQL data, pick Cosmos DB. For raw big data files, pick Azure Data Lake Storage Gen2. For shared file access, pick Azure Files. This step is critical because picking the wrong service leads to poor performance or high costs.
Configure Redundancy and Performance
Decide on the storage account's redundancy level (LRS, ZRS, GRS, RA-GRS) based on your disaster recovery requirements. For blob storage, choose an access tier (Hot, Cool, Archive). For databases, choose a service tier and compute size (DTUs or vCores for SQL, Request Units per second for Cosmos DB). These decisions directly affect uptime guarantees and monthly costs.
Implement Security and Access Controls
Set up Azure RBAC to grant only the necessary permissions to users and applications. Use managed identities for Azure services to avoid storing connection strings in code. Enable encryption at rest (Azure Storage Service Encryption) and encryption in transit (HTTPS/TLS). Configure firewalls and virtual network service endpoints or private endpoints to restrict network access.
Define Data Lifecycle and Management
Create lifecycle management policies to automatically move blobs between Hot, Cool, and Archive tiers based on age. Set up backup policies (e.g., Azure SQL Database point-in-time restore, Blob soft delete). Implement data retention policies to delete old data that is no longer needed for compliance. This step prevents storage costs from spiralling out of control.
Monitor and Optimise Continuously
After deployment, use Azure Monitor to track storage metrics (latency, throughput, errors). Set up alerts for thresholds. Use Azure Advisor to get recommendations for right-sizing disks, changing access tiers, or tuning Cosmos DB RU/s. Periodically review cost reports in Azure Cost Management. Data storage is not a 'set and forget' task; it requires ongoing optimisation.
Let us walk through a concrete scenario: a fictional online retail company called 'ShopWave' that is migrating its entire data platform to Azure. The team has three core data needs.
First, ShopWave's website runs a product catalogue. When a customer searches for 'blue running shoes', the website must return results in under 200 milliseconds. The data for each product—name, description, price, stock level—is structured and relational. The team chooses 'Azure SQL Database' for this. They create tables for products, categories, and inventory. They enable 'auto-tuning' so Azure automatically optimises indexes based on query patterns. They set up 'geo-replication' by creating a readable secondary replica in a second Azure region. If the primary region goes down, the app can failover to the secondary in seconds.
Second, ShopWave captures every click, search, and page view from its 10 million monthly visitors. This is 'clickstream data'—unstructured, high volume, and generated in real time. The data engineers decide to stream this data into 'Azure Event Hubs' (a data ingestion service) and then land it into 'Azure Data Lake Storage Gen2'. They create a folder structure: '/clickstream/raw/yyyy/mm/dd/hh/'. This keeps the data organised and makes it easy for data scientists to later run machine learning models on it. They set the storage tier to 'Cool' because they only query clickstream data from the last two months for ad-hoc analysis. Older data is automatically moved to 'Archive' tier using a 'lifecycle management policy'.
Third, ShopWave needs a central dashboard for the CEO that shows total revenue, top-selling products, and customer acquisition costs across all channels. This data comes from the SQL database (transactions), Data Lake (clickstream), and a separate CRM system. The engineering team builds an 'extract, transform, load' (ETL) pipeline using 'Azure Data Factory'. Every night, they copy and transform the data into 'Azure Synapse Analytics'. They design a star schema with fact tables (sales, clicks) and dimension tables (product, customer, time). The CEO's dashboard tool connects to Synapse using a SQL query that joins these tables. Because Synapse uses MPP, queries that scan billions of rows return in seconds rather than minutes.
Throughout this process, the team also implements security and governance. They use 'Azure Private Endpoints' to ensure data never traverses the public internet. They assign 'Azure RBAC' (role-based access control) roles: data engineers get 'Storage Blob Data Contributor' to write to Data Lake, while business analysts get 'Storage Blob Data Reader' on a specific folder. They apply 'Azure Policy' to enforce encryption for all data at rest and in transit. They also set up 'diagnostic logging' to monitor who accesses which storage account and when.
In daily operations, an Azure data engineer working at ShopWave would:
Monitor storage costs using Azure Cost Management and adjust access tiers when data ages.
Optimise Cosmos DB RU/s (Request Units per second) to balance performance and cost, especially during peak shopping seasons.
Test failover for the geo-replicated SQL database every quarter to ensure the recovery plan works.
Review Azure Advisor recommendations for right-sizing disks attached to virtual machines.
Automate the creation of new storage accounts using ARM templates or Terraform, ensuring naming conventions (e.g., 'shopwavedatalakeweu001') are consistent.
The DP-203 exam will test your ability to 'design and implement data storage' through scenario-based multiple-choice questions. You will not be asked to click through the Azure portal. Instead, you will read a business requirement and pick the correct Azure service or configuration from a list.
Here is what you need to memorise and recognise:
First, the exam loves testing the differences between 'Azure SQL Database', 'Azure SQL Managed Instance', and 'SQL Server on Azure VM'. The key distinction is who manages the underlying operating system and instance. Azure SQL Database is a fully managed 'Platform as a Service' (PaaS) offering—Microsoft handles everything, but you lose some features like SQL Agent Jobs or cross-database queries. Azure SQL Managed Instance gives you more compatibility with on-premises SQL Server (including SQL Agent, linked servers, and CLR) while still being PaaS. SQL Server on Azure VM is 'Infrastructure as a Service' (IaaS)—you manage the OS and SQL Server patches, but you have full control. A typical exam question: 'You need to migrate a legacy SQL Server application that uses SQL Agent jobs. Which service should you use?' The answer is SQL Managed Instance.
Second, the exam will test 'storage redundancy' options. For any storage account (Blob, Files, Data Lake), you must choose a 'redundancy' level: 'Locally Redundant Storage' (LRS) — three copies within a single data centre; 'Zone-Redundant Storage' (ZRS) — copies across three availability zones in the same region; 'Geo-Redundant Storage' (GRS) — copies in a secondary region hundreds of miles away; and 'Read-Access Geo-Redundant Storage' (RA-GRS) — same as GRS but with read-only access to the secondary copy. The trap is that exam questions often say 'you need to protect against a regional disaster and allow read access from the secondary region' — the answer is RA-GRS, not GRS.
Third, 'access tiers' for Blob Storage. You need to know when to use Hot, Cool, and Archive. A common trap question: 'Your application must access blobs within 30 seconds of a request, and you want to minimise storage costs. Storage latency is not critical. Which tier?' The answer is Cool (not Archive) because Archive has a retrieval time of hours, not seconds. Another trap: 'You need to query data that is older than 90 days. Which tier?' The answer is Cool, with a lifecycle policy to move it automatically.
Fourth, 'Azure Cosmos DB consistency levels'. The exam tests the five levels: Strong, Bounded Staleness, Session, Consistent Prefix, and Eventual. For example: 'You are building a global e-commerce application. Your users must always see the latest product price. Which consistency level?' The answer is Strong, because it guarantees linearizability—all reads return the most recent write. But the trap is that Strong reduces write performance and availability. The exam often asks you to balance consistency vs. performance. If the question says 'highest performance with low latency', Session or Eventual is correct.
Fifth, 'data partitioning' in Cosmos DB and Synapse. For Cosmos DB, you must choose a 'partition key'. The exam expects you to know that a good partition key has high cardinality (many unique values) and evenly distributes request units (RU/s). A bad partition key leads to 'hot partitions' where most traffic hits a single partition. For example: 'You store IoT device telemetry. Which partition key is best?' The answer is 'deviceId', not 'status' (which has only a few values).
Finally, 'Azure Files' vs. 'Azure NetApp Files'. Both provide file shares, but Azure NetApp Files is a high-performance, enterprise-grade service built on NetApp technology, supporting advanced features like snapshots and clones. The exam might ask: 'You need to run a high-performance computing (HPC) workload that requires a shared file system with sub-millisecond latency.' The answer is Azure NetApp Files. For standard shared folders accessed by 10-20 employees, Azure Files suffices.
Memorise these distinctions, and practise with sample questions. The exam will try to confuse you with services that sound similar (e.g., 'Azure Table Storage' vs. 'Cosmos DB Table API'—the latter is globally distributed and faster). Always map the requirement first: 'do I need global distribution, low latency, relational queries, or cheap archival?' Then eliminate the wrong answers.
Azure offers four main storage types for raw data: Blob (objects), Files (SMB/NFS shares), Disks (VM block storage), and Data Lake Gen2 (hierarchical big data storage).
Azure Cosmos DB is a globally distributed NoSQL database with multiple consistency models, while Azure SQL Database is a relational PaaS database for transactional workloads.
Storage redundancy levels (LRS, ZRS, GRS, RA-GRS) determine how many copies of your data exist and where, directly affecting durability and disaster recovery capability.
Access tiers for Blob Storage (Hot, Cool, Archive) let you optimise cost by moving infrequently accessed data to cheaper storage, but Archive tier requires hours to retrieve data.
Azure Synapse Analytics separates compute and storage, enabling you to pause compute resources to save costs while data persists in the underlying Data Lake.
For the DP-203 exam, always map the business requirement (latency, scalability, structure, cost) to the correct Azure service, not the one with the most similar name.
Azure Files is ideal for lift-and-shift migrations of legacy apps, while Azure NetApp Files is for high-performance computing workloads needing sub-millisecond latency.
Data partitioning in Cosmos DB requires a partition key with high cardinality to avoid hot partitions; a bad partition key destroys write performance.
These come up on the exam all the time. Here's how to tell them apart.
Azure SQL Database
Relational (tables, rows, columns) with fixed schema
Uses SQL for queries, supports complex JOINs and transactions
Strongly consistent by default (ACID compliance)
Azure Cosmos DB
NoSQL (key-value, document, graph, column-family)
Uses SQL-like syntax or NoSQL API, limited JOINs
Offers five consistency levels, from Eventual to Strong
Azure Blob Storage
Flat namespace (all files at root level unless you fake folders)
No POSIX permissions; uses storage account keys or RBAC
Designed for general object storage (images, backups, logs)
Azure Data Lake Storage Gen2
Hierarchical namespace (real folders and subfolders)
POSIX-compliant access control lists for fine-grained permissions
Optimised for big data analytics tools (Spark, Hive, Hadoop)
Azure Files (SMB)
Fully managed SMB and NFS file shares for general use
Performance up to 100,000 IOPS per share
Ideal for shared folders, lift-and-shift migrations, and dev/test
Azure NetApp Files
Enterprise-grade NFS and SMB with advanced features (snapshots, clones)
Performance up to 4.5 million IOPS with ultra-low latency
Designed for HPC, SAP HANA, and critical database workloads
Hot Access Tier (Blob)
Optimised for frequent reads (multiple times per day)
Higher storage cost per GB, but no retrieval cost
Instant access to data (milliseconds latency)
Archive Access Tier (Blob)
Optimised for rarely accessed data (once or twice per year)
Lowest storage cost per GB, but high retrieval cost and latency
Data must be rehydrated first (hours, not seconds) before access
Geo-Redundant Storage (GRS)
Data replicated to a secondary region for disaster recovery
Secondary copy is not readable during normal operations
Failover to secondary requires a manual or automated process
Read-Access Geo-Redundant Storage (RA-GRS)
Same replication as GRS, plus read-only access to secondary region
Secondary copy is always readable, enabling local read requests
Ideal for applications that need low-latency reads from multiple regions
Azure Synapse Analytics Dedicated SQL Pool
Massively parallel processing (MPP) for large-scale analytics
Separates compute from storage; can pause compute to save costs
Best for data warehousing (aggregating billions of rows)
Azure SQL Database Hyperscale
Single-instance architecture with read scale-out for OLTP
Storage and compute are tightly coupled; no pausing of compute
Best for transactional workloads that need to scale beyond 4 TB
Mistake
Azure Blob Storage is only for storing backups and media files.
Correct
Azure Blob Storage can store any type of unstructured data, including logs, IoT telemetry, and big data files. It is also the foundation for Azure Data Lake Storage Gen2.
Beginners often think of blob storage as just a 'dumping ground' because they have only used it for simple file uploads in tutorials. They do not realise that many data pipelines use it as the primary landing zone for raw data.
Mistake
If I use Azure SQL Database, I have full control over the operating system and can install custom software on the server.
Correct
Azure SQL Database is a Platform-as-a-Service (PaaS) offering. You get a database endpoint but no access to the underlying OS, no ability to install software, and limited ability to configure server-level settings like trace flags or SQL Agent jobs.
People with a background in managing on-premises SQL Server assume 'a database is a database' and think they will have the same administrative control. The cloud PaaS model removes that control in exchange for automatic patching and high availability.
Mistake
Cosmos DB is just a faster version of Azure SQL Database.
Correct
Cosmos DB is a NoSQL database designed for massive scale and global distribution, not for complex relational queries with JOINs. Azure SQL Database is a relational database for transactional workloads. They solve different problems.
Both are databases, so beginners assume they are interchangeable. They do not grasp the fundamental difference between SQL (relational, ACID) and NoSQL (non-relational, eventually consistent, flexible schema).
Mistake
Geo-Redundant Storage (GRS) always allows you to read from the secondary region.
Correct
Standard GRS provides read-only access to the secondary region only if you explicitly enable it with 'Read-Access Geo-Redundant Storage' (RA-GRS). Without RA-GRS, the secondary copy is only used for failover and is not readable during normal operations.
The name 'Geo-Redundant' sounds like it gives you access to both copies. Beginners do not read the documentation carefully enough to see the distinction between GRS and RA-GRS.
Mistake
You can change the access tier of a blob from Archive to Hot instantly.
Correct
Changing a blob from Archive to Hot (or Cool) triggers a 'rehydration' process that can take up to 15 hours. During rehydration, the blob is not accessible.
People see 'storage tier' as a simple setting like a light switch. They do not realise that Archive tier stores data on offline tape or slow disk, so retrieving it takes significant time and is a deliberate, high-cost operation.
Mistake
Azure Files and Azure NetApp Files are the same thing; you can use either one interchangeably.
Correct
Azure Files is a fully managed SMB/NFS file share for general use. Azure NetApp Files is a high-performance, enterprise-grade file service with advanced features like snapshots, clones, and low-latency access suitable for HPC workloads.
Both services provide file shares in the cloud, so the names sound interchangeable. Beginners do not understand the performance and feature differences, leading them to choose the wrong service for latency-sensitive applications.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Azure Blob Storage is object storage for unstructured data, organised in a flat namespace. Azure Data Lake Storage Gen2 builds on top of Blob Storage by adding a hierarchical namespace (folders and subfolders) and POSIX permissions, making it suitable for big data analytics tools like Apache Spark.
Use Cosmos DB when you need global distribution, low latency (single-digit milliseconds) at any scale, and your data does not require complex relational joins or strict ACID transactions. Use Azure SQL Database for traditional transactional workloads that require relational queries, foreign keys, and strong consistency.
Use the 'Cool' or 'Archive' access tiers. Set up a lifecycle management policy to automatically move blobs from Hot to Cool after 30 days, then to Archive after 90 days. Remember that Archive tier has a retrieval time of up to 15 hours, so it is only suitable for data you almost never need.
LRS (Locally Redundant Storage) keeps three copies within a single data centre. ZRS (Zone-Redundant Storage) keeps copies across three availability zones in the same region. GRS (Geo-Redundant Storage) copies data to a secondary region hundreds of miles away. RA-GRS is GRS but also gives read-only access to the secondary copy.
No. Changing from Archive to Hot triggers a 'rehydration' process that can take up to 15 hours. During this time, the blob is not accessible. Plan accordingly and only use Archive for data you can tolerate being offline.
Choose a partition key that has high cardinality (many distinct values), such as a 'userId' or 'deviceId'. Avoid keys with few values (like 'status' with only 3 options) because that creates 'hot partitions' where all traffic goes to one node, limiting scale. A good partition key evenly distributes Request Units across physical partitions.
Azure SQL Database is a fully managed PaaS offering with limited SQL Server features (no SQL Agent, no cross-database queries). Azure SQL Managed Instance provides higher compatibility with on-premises SQL Server, including SQL Agent, linked servers, and CLR integration, while still being PaaS. Choose Managed Instance when migrating legacy apps that depend on these features.
You've finished Design Data Storage Solutions. Continue through the DP-203 study guide to build a complete picture of the exam.
Done with this chapter?