Azure Functions Workflows: Orchestration Patterns

Azure Functions is a serverless compute service that lets you run event-driven code without managing infrastructure. However, a single function execution has limits: it must complete within a timeout (default 5 minutes for the Consumption plan, up to 10 minutes for Premium, and unlimited for Dedicated/ASE). For workflows that span hours, days, or involve multiple steps with conditional logic, a single function is insufficient. This is where Durable Functions come in. Durable Functions is an extension of Azure Functions that allows you to define stateful workflows (orchestrations) using code. The orchestrator function coordinates the execution of other functions (activity functions) and manages state, retries, and waiting for external events.

Durable Functions rely on the Durable Task Framework (DTF), which uses Azure Storage (queues, tables, and blobs) to persist execution state. The core idea is that the orchestrator function is replayed on each execution. When an orchestrator function calls an activity, it yields control, and the runtime persists the call to a history table. On replay, the orchestrator reads the history and skips already-completed activities, ensuring deterministic behavior. This replay mechanism allows the orchestrator to be stateless in memory while maintaining durable state.

Key Components: - Orchestrator function: Defines the workflow using context object. Must be deterministic (no random, no I/O except via activity calls). - Activity function: Performs actual work (e.g., calling an API, processing a file). Can be non-deterministic. - Client function: Starts an orchestration instance (typically an HTTP-triggered function). - Orchestration instance: A single running workflow, identified by an InstanceId.

Storage Resources: - Task Hub: A logical container for storage resources. Named in host.json. All instances share the same task hub. - Control Queue: Contains messages that drive the orchestration (e.g., timer messages, activity responses). - Work-item Queue: Contains messages for activity tasks. - History Table: Stores orchestration history as rows (events). - Instance Table: Stores runtime status of instances. - Blob Leases: Used for partition management.

Execution Flow: 1. Client function starts an orchestration by calling starter.StartNewAsync("OrchestratorFunction", input). 2. The runtime creates an orchestration instance and enqueues an execution message to the control queue. 3. The orchestrator function is triggered. It executes until it calls an activity or waits. At that point, it yields, and the runtime persists the call to history. 4. The activity function is triggered via the work-item queue. It executes and returns a result to the control queue. 5. The orchestrator is replayed: it reads history, sees that the activity has completed, and continues from that point. 6. This continues until the orchestrator completes or is terminated.

#### 1. Function Chaining

Executes a sequence of activities in a specific order. Each activity's output is passed as input to the next. This is the simplest pattern.

[FunctionName("Chaining")]
public static async Task<object> Run(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var x = await context.CallActivityAsync<object>("F1", null);
    var y = await context.CallActivityAsync<object>("F2", x);
    var z = await context.CallActivityAsync<object>("F3", y);
    return z;
}

#### 2. Fan-Out/Fan-In

Executes multiple activities in parallel (fan-out) and then aggregates results (fan-in). Useful for batch processing.

[FunctionName("FanOutFanIn")]
public static async Task Run(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var tasks = new List<Task<int>>();
    var workBatch = await context.CallActivityAsync<List<string>>("GetWorkBatch", null);
    foreach (var workItem in workBatch)
    {
        tasks.Add(context.CallActivityAsync<int>("ProcessWorkItem", workItem));
    }
    await Task.WhenAll(tasks);
    var sum = tasks.Sum(t => t.Result);
    await context.CallActivityAsync("SendResult", sum);
}

#### 3. Async HTTP APIs

Long-running operations that return an HTTP 202 Accepted response with a status endpoint. The client polls the endpoint to check completion. This pattern solves the problem of HTTP timeouts for long-running processes.

[FunctionName("HttpStart")]
public static async Task<HttpResponseMessage> Run(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestMessage req,
    [DurableClient] IDurableOrchestrationClient starter,
    ILogger log)
{
    string instanceId = await starter.StartNewAsync("MyOrchestration", null);
    return starter.CreateCheckStatusResponse(req, instanceId);
}

The response contains statusQueryGetUri, sendEventPostUri, terminatePostUri, etc. The client can call statusQueryGetUri to get status: Running, Completed, Failed, Terminated.

#### 4. Monitor

Repeatedly checks a condition until it is met. Uses durable timers to avoid infinite loops and to control polling frequency.

[FunctionName("Monitor")]
public static async Task Run(
    [OrchestrationTrigger] IDurableOrchestrationContext context,
    ILogger log)
{
    int jobId = context.GetInput<int>();
    int pollingInterval = 15; // seconds
    DateTime expiry = context.CurrentUtcDateTime.AddHours(6);

    while (context.CurrentUtcDateTime < expiry)
    {
        var jobStatus = await context.CallActivityAsync<string>("GetJobStatus", jobId);
        if (jobStatus == "Completed")
        {
            await context.CallActivityAsync("SendAlert", $"Job {jobId} completed.");
            return;
        }
        var nextCheck = context.CurrentUtcDateTime.AddSeconds(pollingInterval);
        await context.CreateTimer(nextCheck, CancellationToken.None);
    }
    await context.CallActivityAsync("SendAlert", $"Job {jobId} timed out.");
}

#### 5. Human Interaction

Waits for an external event (e.g., approval) with a timeout. Uses WaitForExternalEvent and CreateTimer for race conditions.

[FunctionName("HumanInteraction")]
public static async Task Run(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    await context.CallActivityAsync("RequestApproval", null);
    using (var timeoutCts = new CancellationTokenSource())
    {
        DateTime due = context.CurrentUtcDateTime.AddHours(72);
        Task durableTimeout = context.CreateTimer(due, timeoutCts.Token);
        Task<bool> approvalEvent = context.WaitForExternalEvent<bool>("ApprovalEvent");
        if (approvalEvent == await Task.WhenAny(approvalEvent, durableTimeout))
        {
            timeoutCts.Cancel();
            if (approvalEvent.Result)
                await context.CallActivityAsync("ProcessApproval", null);
            else
                await context.CallActivityAsync("Escalate", null);
        }
        else
        {
            await context.CallActivityAsync("Escalate", null);
        }
    }
}

- host.json: Configure durableTask section. - hubName: Task hub name (default: TestHubName for local, AzureFunctionsHub for Azure). - storageProvider: Azure Storage connection string (default: AzureWebJobsStorage). - maxConcurrentActivityFunctions: Max concurrent activity functions (default: 10 for single VM, 10x number of partitions for scale). - maxConcurrentOrchestratorFunctions: Max concurrent orchestrator functions (default: 10 for single VM). - extendedSessionsEnabled: Enable session reuse (default: false). - extendedSessionIdleTimeoutInSeconds: Idle timeout for extended sessions (default: 30).

A smart building company uses Durable Functions to roll out firmware updates to thousands of IoT devices. The orchestration: (1) queries devices to update (activity), (2) fans-out to each device to download firmware (activity), (3) waits for each device to report installation status (external events), (4) aggregates success/failure, (5) sends a report. The monitor pattern is used to check device health after update. If a device fails to respond within 30 minutes, the orchestration marks it as failed and retries later. This system handles 50,000 devices. Key consideration: use ContinueAsNew to keep the orchestration alive for weeks without hitting history limits. Misconfiguration: not using context.CreateTimer for delays; using Thread.Sleep causes replay issues and unnecessary cost.