Azure Table Storage and Blob Data

Azure Table Storage and Azure Blob Storage are both part of Azure Storage, a massively scalable, durable cloud storage service. They are non-relational (NoSQL) and store data differently from a relational database like Azure SQL Database.

Both services are accessed via REST APIs and are highly available, with at least 99.9% SLA for standard tiers. They are replicated across multiple storage nodes for durability.

Blob Storage - Blobs are stored in a flat namespace within a container. Containers are like directories but cannot be nested. - Each blob is identified by a URL: https://<storageaccount>.blob.core.windows.net/<container>/<blobname> - Blobs have metadata (key-value pairs) and can have properties like content type, content encoding, and lease state. - Three types of blobs: Block blobs (for most data, up to 4.75 TB per blob), Append blobs (optimized for append operations, like logging), and Page blobs (for random read/write, up to 8 TB, used for Azure VM disks). - Block blobs are composed of blocks (up to 100 MB each). You upload blocks individually and commit them to finalize the blob. This allows parallel uploads and retries. - Replication options: LRS (locally redundant, 3 copies in one datacenter), GRS (geo-redundant, 6 copies across 2 regions), RA-GRS (read-access geo-redundant), ZRS (zone-redundant), and GZRS. - Access tiers: Hot (frequent access, higher cost per GB, lower access cost), Cool (infrequent access, lower storage cost, higher access cost), Archive (offline, lowest storage cost, hours to rehydrate).

Table Storage - Data is stored in tables within a storage account. Each table can have billions of entities. - Each entity has a Partition Key (PK) and Row Key (RK) that together form the primary key. The PK enables partitioning across storage nodes for scalability. - Entities are schema-less: each entity can have different properties (columns). The only required properties are PK, RK, and Timestamp. - Queries are efficient only when using PK and RK. Queries that scan without PK are slow and expensive. - Maximum entity size is 1 MB, maximum property count is 252 (excluding PK, RK, Timestamp). - Table Storage is not the same as Azure Cosmos DB Table API, which offers higher throughput and lower latency but is a separate service.

Blob Storage - Default blob type: Block blob (if not specified). - Maximum block size: 100 MB per block (for block blobs). - Maximum blob size: 4.75 TB for block blobs (using 50,000 blocks of 100 MB). - Maximum container name length: 63 characters, lowercase letters, numbers, and hyphens. - Maximum blob name length: 1024 characters. - Storage account types: General-purpose v2 (GPv2) supports all features; Blob storage accounts are legacy. - Soft delete: Enabled by default for blobs, with a retention period of 7 days (configurable up to 365 days). - Versioning: Can be enabled to keep previous versions of blobs. - Access tier can be set at the blob level or account level.

Table Storage - Maximum table name length: 63 characters. - Maximum entity size: 1 MB. - Maximum number of properties: 255 (including PK, RK, Timestamp). - Partition key string length: 1-1024 characters. - Row key string length: 1-1024 characters. - Timestamp is automatically maintained by Azure and used for optimistic concurrency. - Maximum storage account capacity: 500 TB total (shared across blobs, tables, queues, files). - Throughput targets: Up to 20,000 entities per second per partition (depending on partition design).

Using Azure CLI:

Scenario 1: Storing User Profile Images in a Social Media App

A social media platform uses Azure Blob Storage to store millions of user profile pictures. They use a GPv2 storage account with RA-GRS replication for high durability and read access during regional failures. Each user uploads a JPEG image, which is stored as a block blob in a container named 'profiles'. The blob name is the user's ID (e.g., 'user12345.jpg'). They set the blob access tier to Hot because images are accessed frequently. To improve performance, they use Azure CDN to cache images at edge locations. Common misconfiguration: forgetting to set the public access level to 'Blob' if images need to be publicly accessible via URL. If set to Private, the app must generate SAS tokens for each request, adding overhead. They also enable soft delete with a 30-day retention to recover from accidental deletions. Scale: They store 10 TB of images, with 50,000 uploads per day. Performance is fine as Blob Storage scales automatically.

Scenario 2: IoT Sensor Data Logging

A manufacturing company collects temperature and humidity readings from thousands of sensors. Each sensor sends a JSON payload every minute. They use Azure Table Storage because the data is structured (sensor ID, timestamp, value) and they need fast lookup by sensor and time. They design the partition key as SensorId and row key as a reversed timestamp (e.g., DateTime.MaxValue.Ticks - ticks) to enable querying the most recent data first. They use a GPv2 account with LRS (since data is not critical). Throughput is about 10,000 writes per second, which Table Storage handles easily. Queries filter by partition key (SensorId) and row key range. Trap: If they had used a single partition key for all sensors, they would hit the 20,000 ops/sec limit and see throttling. Also, they cannot run complex analytics on Table Storage; they use Azure Stream Analytics to read from Table Storage and aggregate into a SQL database for reporting.

Scenario 3: Storing Large Backup Files

An enterprise backs up on-premises databases to Azure Blob Storage nightly. Each backup file is 500 GB, so they use block blobs with large block sizes (100 MB). They upload using AzCopy for parallel transfer. They set the access tier to Cool because backups are accessed infrequently (only for restores). After 30 days, they move blobs to Archive tier using lifecycle management policies to reduce costs. Restore process: When a restore is needed, they change the blob tier from Archive to Hot, which takes up to 15 hours. They must plan for this delay. Misconfiguration: If they accidentally set the default access tier to Archive, the first upload fails because Archive does not support immediate writes. They also enable blob versioning to keep multiple backup versions. Scale: They store 100 TB across multiple containers, with each container holding backups for a specific database.