Blob Storage Events with Event Grid

Azure Event Grid is a fully managed event routing service that enables event-driven, reactive programming in the cloud. It decouples event publishers from event subscribers, allowing services like Blob Storage to emit events without knowing who is listening. Event Grid then reliably delivers those events to subscribers (handlers) such as Azure Functions, Logic Apps, Webhooks, or other Azure services. This eliminates the need for polling or custom message brokers, reducing latency and cost.

Blob Storage can emit events for operations like blob creation, deletion, or modification. These events are published to an Event Grid system topic (automatically created for the storage account) or a custom topic. The flow is: 1. An operation occurs on a blob (e.g., PutBlob, DeleteBlob). 2. Blob Storage generates an event and sends it to Event Grid. 3. Event Grid matches the event against all subscriptions (filters based on event type, subject, etc.). 4. Event Grid delivers the event to each matching subscriber endpoint. 5. The subscriber processes the event and returns an HTTP 200 OK to acknowledge receipt.

- Event Schema: Event Grid uses CloudEvents v1.0 schema by default. Each event includes id, topic, subject, eventType, eventTime, data, and dataVersion. - Event Types for Blob Storage: - Microsoft.Storage.BlobCreated – emitted when a blob is created or replaced (PutBlob, PutBlockList, CopyBlob). - Microsoft.Storage.BlobDeleted – emitted when a blob is deleted (DeleteBlob). - Additional types like BlobTierChanged exist but are less common. - Subject Filtering: The subject field follows the pattern /blobServices/default/containers/<container-name>/blobs/<blob-name>. You can filter subscriptions by subject prefix or suffix (e.g., subjectBeginsWith: /blobServices/default/containers/images/). - Delivery Retry Policy: Event Grid retries delivery for up to 24 hours (default) with exponential backoff. Maximum retry count is 30 (default). You can configure maxDeliveryAttempts and eventTimeToLive (in minutes, max 1440). - Dead-Lettering: If delivery fails after all retries, events can be sent to a dead-letter storage blob container. This is configured per subscription. - Filters: Subscriptions can filter by event type, subject, and advanced filters (e.g., blob size > 1 MB). Advanced filters support operators like NumberGreaterThan, StringContains, etc. - Endpoint Validation: For webhook endpoints, Event Grid validates the endpoint by sending a validation handshake (HTTP POST with validationCode). The endpoint must respond with the validation code in the body or URL within 10 seconds.

To enable blob events, you don't need to enable anything on the storage account – it's automatic. However, you must create an Event Grid subscription. You can do this via Azure Portal, Azure CLI, or PowerShell.

Azure CLI Example:

# Create an Event Grid subscription for blob created events in a specific container
az eventgrid event-subscription create \
  --name MyBlobEventSub \
  --source-resource-id /subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/{account} \
  --included-event-types Microsoft.Storage.BlobCreated \
  --subject-begins-with /blobServices/default/containers/mycontainer/ \
  --endpoint https://myfunctionapp.azurewebsites.net/api/trigger

PowerShell Example:

New-AzEventGridSubscription `
  -EventSubscriptionName MyBlobEventSub `
  -ResourceId (Get-AzStorageAccount -ResourceGroupName MyRG -Name MyStorageAccount).Id `
  -IncludedEventType Microsoft.Storage.BlobCreated `
  -SubjectBeginsWith '/blobServices/default/containers/mycontainer/' `
  -Endpoint https://myfunctionapp.azurewebsites.net/api/trigger

Verification:

# List all event subscriptions for a storage account
az eventgrid event-subscription list --source-resource-id /subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/{account}

# Get details of a specific subscription
az eventgrid event-subscription show --name MyBlobEventSub --source-resource-id ...

A media company uploads user-submitted images to Azure Blob Storage. They need to generate thumbnails and watermark each image automatically. They configure an Event Grid subscription for BlobCreated events on the 'images' container, with a subject filter for common image extensions (.jpg, .png). The event triggers an Azure Function (consumption plan) that reads the blob, processes it using ImageMagick, and writes the thumbnail to a separate container. The function is idempotent: it checks for existing thumbnail metadata before processing. At scale, they handle thousands of uploads per second; Event Grid can handle millions of events per second. A common misconfiguration: forgetting to filter by file extension, causing the function to be triggered for non-image files (e.g., .txt), wasting compute. They also configure dead-lettering to a storage container to capture failed events for later analysis.

A financial services company needs to archive audit logs from multiple applications into Azure Blob Storage. Each application writes logs to a specific container. They want to be notified when new log files arrive to trigger a compliance check. They create an Event Grid subscription for BlobCreated events on the 'audit-logs' container, with the endpoint being a Logic App that validates the log format and, if valid, copies the blob to a long-term retention container with immutability policy. If validation fails, the Logic App sends an alert to the security team. The challenge: logs arrive in bursts, and Event Grid's at-least-once delivery causes duplicate validation. The Logic App uses a deduplication step (checking blob metadata for a processed flag). They also set maxDeliveryAttempts to 5 and eventTimeToLive to 60 minutes to avoid processing stale events.

An e-commerce platform uses Blob Storage for product images. When a new image is uploaded, they need to: (1) generate multiple sizes, (2) update a search index, (3) invalidate a CDN cache. Instead of a single handler doing everything, they create multiple Event Grid subscriptions (one per handler) all pointing to different Azure Functions. This decouples the processing and allows independent scaling. However, they must ensure idempotency across all handlers. A common issue: if the CDN purge function fails, the image may be served stale. They configure dead-lettering and a monitoring alert on the dead-letter container. The exam tests the concept of multiple subscriptions for the same event.