Azure Blob Storage

Azure Blob Storage is Microsoft's object storage solution for the cloud. It is designed to store massive amounts of unstructured data—data that doesn't fit neatly into relational tables. Examples include text files, images, videos, backups, log files, and virtual machine disks. Before cloud storage, businesses had to manage their own on-premises storage area networks (SANs) or network-attached storage (NAS). This required upfront capital expenditure, physical space, power, cooling, and IT staff to maintain. Scaling meant buying more hardware, which could take weeks. Blob Storage eliminates these headaches: you pay only for what you use, scale instantly from gigabytes to petabytes, and access data from anywhere via the internet.

Blob Storage works on a flat namespace model. You create a Storage Account (the top-level container), then within it you create containers (like folders, but they don't nest), and then you upload blobs (files) into those containers. Each blob has a unique URL: https://<storage-account>.blob.core.windows.net/<container>/<blob-name>. When you upload a blob, Azure breaks it into chunks (blocks) and stores them across multiple servers. For retrieval, Azure reassembles the blocks and streams the blob to you. This allows parallel uploads and downloads, improving performance.

- Storage Account: The Azure resource that contains all your Blob Storage data. It defines the global namespace, replication strategy, and performance tier. You can have up to 250 storage accounts per subscription per region. - Container: A grouping of blobs. There is no nesting; all blobs are at the same level within a container. However, you can simulate a folder hierarchy by using naming conventions (e.g., logs/2024/01/log.txt). - Blob: The actual file. There are three types: - Block Blobs: For text and binary files (up to 4.75 TB). Most common type. - Append Blobs: Optimized for append operations (e.g., logging). - Page Blobs: For random read/write operations (up to 8 TB). Used for virtual machine disks.

Blob Storage offers three access tiers to optimize cost based on access frequency:

You can set the tier at the blob level or the storage account level (default for all blobs). You can also automatically move blobs between tiers using lifecycle management policies.

You pay for: - Storage capacity: Per GB per month (varies by tier). - Data operations: Per 10,000 operations (read, write, list). - Data transfer: Outbound data (egress) is charged; inbound is free. - Early deletion fees: For Cool and Archive tiers if deleted before minimum duration.

On-premises, you might have a file server with RAID arrays. Scalability requires buying new drives or servers. Backup typically involves tape or external drives. With Blob Storage, you get: - Elastic scaling: No capacity planning. - Geo-redundancy: Replicate data to another region for disaster recovery. - Access control: Azure AD integration, shared access signatures (SAS), and firewall rules. - No hardware maintenance: Microsoft handles all patching and hardware failures.

In the Azure portal, you can create a storage account by navigating to "Storage accounts" and clicking "Create". You then configure basics (subscription, resource group, name), networking (public or private), data protection (soft delete, versioning), and advanced settings (hierarchical namespace for Data Lake Storage Gen2). Once created, you can add containers and upload blobs directly from the portal.

Using Azure CLI, you can automate these tasks. For example:

# Create a storage account
az storage account create --name mystorageaccount --resource-group myResourceGroup --location eastus --sku Standard_LRS --kind StorageV2

# Create a container
az storage container create --name mycontainer --account-name mystorageaccount --auth-mode login

# Upload a blob
az storage blob upload --container-name mycontainer --name myblob.txt --file /path/to/file.txt --account-name mystorageaccount --auth-mode login

A law firm needs to retain client documents for 7 years for compliance. They previously used external hard drives, which were unreliable and required manual rotation. They migrate to Azure Blob Storage. They create a storage account with geo-redundant storage (GRS) so data is replicated to a secondary region. They use lifecycle management: Hot tier for the first 30 days (frequent access), Cool tier for months 1-6, Archive tier for years 1-7. They set up Azure Backup to automatically backup their on-premises file server to Blob Storage daily. They also enable soft delete (retention of 30 days) to recover from accidental deletions. Cost: they pay ~$0.018/GB/month for Cool and ~$0.002/GB/month for Archive. If they need to restore, they rehydrate from Archive (takes ~15 hours). They test recovery quarterly. Mistake: If they forget to set lifecycle policies, they pay Hot tier prices for 7 years, drastically increasing costs.

An e-commerce company has product images, CSS, and JavaScript files. They host them in Blob Storage with a CDN. They create a storage account with a globally unique name. They set the container access level to 'Blob' (anonymous read for blobs) so users can directly access images via URLs. They integrate Azure CDN (from Microsoft) to cache blobs at edge nodes worldwide. The CDN pulls from Blob Storage on cache miss. They use versioning to keep previous versions of images. Cost: storage cost is low; CDN egress cost is ~$0.01/GB. Problem: If they set container access to 'Private', the CDN cannot fetch blobs unless they use a SAS token, which complicates caching. They must also enable 'Secure transfer required' to enforce HTTPS.

A smart building company collects temperature and humidity data from thousands of sensors every minute. They use Azure IoT Hub to ingest data, which writes to Blob Storage as JSON files. They use Append Blobs for real-time log streaming. They set up an Azure Function that triggers when a new blob is created, processing the data and storing it in a database. They use lifecycle management to delete raw blobs after 30 days. Scale: They write ~100 MB per minute. Blob Storage can handle this easily with block sizes of 4 MB. Mistake: If they use Block Blobs instead of Append Blobs, each write creates a new blob, leading to millions of small blobs, which degrades performance and increases transaction costs. They should use Append Blobs or batch writes.