Durable Functions: Fan-Out, Chaining, Human Interaction

Durable Functions is an extension of Azure Functions that allows you to write stateful functions in a serverless compute environment. The core problem it solves is that standard Azure Functions are stateless — they execute in response to an event and then terminate, with no built-in mechanism to maintain state across multiple invocations or to orchestrate complex workflows that involve waiting, retries, and parallel execution. Durable Functions addresses this by introducing a durable orchestration framework that automatically manages state, checkpoints, and replays.

Durable Functions introduces two main function types: orchestrator functions and activity functions. Orchestrator functions define the workflow using procedural code (e.g., C# async methods) and coordinate the execution of activity functions. Activity functions are the actual units of work — they perform one specific action, such as calling an external API, writing to a database, or processing a file.

The orchestrator function is deterministic: every time it runs, it must produce the same sequence of calls to activity functions given the same input. This determinism is enforced by the Durable Functions runtime through a mechanism called replay. The orchestrator function's execution history is persisted in Azure Storage (specifically in a table named DurableFunctionsHistory). If the orchestrator is interrupted (e.g., due to a host restart), it can be replayed from the beginning by reading the history and skipping already-completed steps. This ensures that the orchestrator is resilient to failures without requiring explicit checkpointing code.

To enable Durable Functions in a function app, you must add the DurableTask extension. In the host.json file, you can configure the storage connection string and task hub name:

{
  "version": "2.0",
  "extensions": {
    "durableTask": {
      "hubName": "MyTaskHub",
      "storageProvider": {
        "connectionStringName": "DurableFunctionsStorage",
        "partitionCount": 4
      }
    }
  }
}

To verify correct configuration, you can monitor the storage account: check that queues (e.g., MyTaskHub-control-00, MyTaskHub-workitems) and tables (e.g., DurableFunctionsHistory, DurableFunctionsInstances) are created. Use Azure Storage Explorer or CLI:

az storage table list --connection-string "<connection-string>" --query "[?contains(name, 'DurableFunctions')]"

Function chaining executes a sequence of activities in a specific order. The orchestrator calls each activity one after another, and the output of one can be passed as input to the next. This is implemented using await in C#:

[FunctionName("Chaining")]
public static async Task<List<string>> RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var input = context.GetInput<string>();
    var result1 = await context.CallActivityAsync<string>("Activity1", input);
    var result2 = await context.CallActivityAsync<string>("Activity2", result1);
    var result3 = await context.CallActivityAsync<string>("Activity3", result2);
    return new List<string> { result1, result2, result3 };
}

The orchestrator replays after each activity completes. If the host crashes after Activity1 but before Activity2, the orchestrator replays from the start, sees that Activity1 has already completed (via history), and skips directly to calling Activity2.

Fan-out/fan-in executes multiple activities in parallel and then aggregates their results. This is done using Task.WhenAll:

[FunctionName("FanOutFanIn")]
public static async Task<int> RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var workBatch = await context.CallActivityAsync<string[]>("GetWorkBatch", null);
    var tasks = new Task<int>[workBatch.Length];
    for (int i = 0; i < workBatch.Length; i++)
    {
        tasks[i] = context.CallActivityAsync<int>("ProcessItem", workBatch[i]);
    }
    var results = await Task.WhenAll(tasks);
    var total = results.Sum();
    return total;
}

Key points: The orchestrator issues all activity calls immediately (fan-out) and then awaits all of them (fan-in). The number of concurrent activities is limited by the maxConcurrentTaskActivityWorkItems setting (default 10 per host). For large fan-outs, consider batching or using a sub-orchestrator pattern.

Human interaction involves waiting for an external event (e.g., approval) with a timeout. The orchestrator creates a durable timer and waits for either the timer to expire or an external event to be raised:

[FunctionName("ApprovalWorkflow")]
public static async Task RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    await context.CallActivityAsync("RequestApproval", null);
    using (var timeoutCts = new CancellationTokenSource())
    {
        DateTime deadline = context.CurrentUtcDateTime.AddHours(24);
        Task timeoutTask = context.CreateTimer(deadline, timeoutCts.Token);
        Task approvalTask = context.WaitForExternalEvent("ApprovalResult");
        var winner = await Task.WhenAny(timeoutTask, approvalTask);
        if (winner == approvalTask)
        {
            // Approval received
        }
        else
        {
            // Timeout occurred
            await context.CallActivityAsync("Escalate", null);
        }
    }
}

The external event is raised from another function (e.g., an HTTP-triggered function) using:

await client.RaiseEventAsync(instanceId, "ApprovalResult", approvalValue);

Important: The WaitForExternalEvent call is durable — if the host restarts while waiting, it will resume waiting after replay. The timer is also durable.

Activity functions can throw exceptions. The orchestrator can catch them and implement retry logic using CallActivityWithRetryAsync:

try
{
    await context.CallActivityWithRetryAsync("FlakyActivity", 
        new RetryOptions(TimeSpan.FromSeconds(5), 3)
        {
            BackoffCoefficient = 2.0,
            MaxRetryInterval = TimeSpan.FromMinutes(1)
        }, input);
}
catch (Exception ex)
{
    // Handle after retries exhausted
}

A large e-commerce company uses Durable Functions to process orders from submission to fulfillment. The orchestration chains activities: validate payment, check inventory, reserve stock, ship, and send confirmation. Fan-out is used to check inventory across multiple warehouses in parallel. Human interaction is required when a payment fails — an email is sent to the customer with a link to update payment, and if not completed within 48 hours, the order is cancelled and inventory is released. In production, this system handles 10,000 orders per hour. The key configuration is setting the maxConcurrentTaskActivityWorkItems to 50 per host to balance throughput and resource usage. A common misconfiguration is using too few control queue partitions (default 4), which limits parallelism — increasing to 16 improved throughput by 3x.

A financial services firm uses Durable Functions for document approval. When a document is uploaded, an orchestrator starts: first, it runs an activity to classify the document, then fans out to multiple reviewers (each reviewer is an activity that sends an email with an approval link). The orchestrator waits for all reviewers to approve or reject. If any reviewer rejects, the entire workflow is terminated. A durable timer enforces a 7-day deadline; after that, the document is escalated to a manager. The system processes 500 documents per day. The main challenge is managing the large number of concurrent timers — each timer creates a storage queue message. The team had to increase the storage account's queue capacity and use the Netherite provider for better timer performance.

A media company processes video uploads: transcode to multiple formats (fan-out), generate thumbnails, and extract metadata. Each activity runs in parallel. After all complete, the orchestrator aggregates results and updates a database. The fan-out can involve 10–20 parallel activities. The orchestrator uses CallActivityWithRetryAsync for each transcode activity with 3 retries and exponential backoff. In production, they noticed that if one activity fails and exhausts retries, the entire orchestration fails. They implemented a custom error-handling pattern: catch exceptions and store partial results, then continue. This required careful design of the orchestrator's state management.