Custom Telemetry with Application Insights SDK

Application Insights automatically collects a wealth of telemetry: requests, exceptions, dependencies, page views, and performance counters. However, auto-collection cannot capture business-specific events or custom metrics that are unique to your application. For example, an e-commerce app might need to track 'CheckoutCompleted' or 'ItemAddedToCart' events with custom properties like product ID and price. This is where custom telemetry comes in. By using the Application Insights SDK, you can programmatically send: - Custom Events: Named occurrences (e.g., 'UserLoggedIn', 'PaymentFailed') with optional metrics and properties. - Custom Metrics: Numeric values that can be aggregated (e.g., queue length, processing time). - Traces: Diagnostic log messages with severity levels. - Page Views: Custom page view tracking for non-web contexts.

The Application Insights SDK provides a TelemetryClient object that acts as the primary interface for sending custom telemetry. When you call a method like TrackEvent(), the SDK creates a telemetry item (e.g., EventTelemetry) that includes: - Timestamp: Automatically set to UTC now. - Sequence number: For ordering. - Custom properties: A dictionary of string key-value pairs. - Custom measurements: A dictionary of string-double pairs for numeric data. - IKey: The instrumentation key (or connection string) identifying the Application Insights resource.

The telemetry item is then passed through a telemetry processor chain (if configured) where you can modify or filter items. After processing, it enters the telemetry channel, which buffers items and sends them to the ingestion endpoint. The channel uses a default buffer size of 500 items and a flush interval of 30 seconds (configurable). If the buffer overflows, items are dropped (by default) or you can configure a custom overflow handler.

var telemetryClient = new TelemetryClient(new TelemetryConfiguration(connectionString));

telemetryClient.TrackEvent("OrderPlaced", new Dictionary<string, string> { {"Currency", "USD"} }, new Dictionary<string, double> { {"Revenue", 99.95} });

telemetryClient.TrackTrace("Processing started", SeverityLevel.Information);

public class MyTelemetryProcessor : ITelemetryProcessor
{
    private ITelemetryProcessor Next { get; set; }
    public MyTelemetryProcessor(ITelemetryProcessor next) => Next = next;
    public void Process(ITelemetry item)
    {
        if (item is EventTelemetry eventTelemetry && eventTelemetry.Name == "SensitiveEvent")
            return; // Drop the event
        Next.Process(item);
    }
}

public class MyTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        telemetry.Context.Cloud.RoleName = "MyRole";
        telemetry.Context.GlobalProperties["Environment"] = "Production";
    }
}

var channel = new InMemoryChannel { MaxTelemetryBufferCapacity = 1000, SendingInterval = TimeSpan.FromSeconds(10) };

To configure the SDK in an ASP.NET Core app, add the following to ConfigureServices:

services.AddApplicationInsightsTelemetry(connectionString);

For custom telemetry, inject TelemetryClient:

public class MyService
{
    private readonly TelemetryClient _telemetryClient;
    public MyService(TelemetryClient telemetryClient) => _telemetryClient = telemetryClient;
}

To verify telemetry is flowing, use Application Insights Search in the Azure portal or the Live Metrics blade for real-time validation. You can also use Diagnostic Search to query for custom events.

The channel handles retries with exponential backoff (up to 10 retries). If the endpoint is unreachable, items are stored in a local buffer (default 1 MB) and sent when connectivity resumes. You can implement a custom ITelemetryChannel for persistent storage (e.g., to disk) to prevent data loss during extended outages.

using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.DataContracts;

var config = new TelemetryConfiguration("InstrumentationKey=...;IngestionEndpoint=https://...");
var client = new TelemetryClient(config);

// Add a telemetry initializer
config.TelemetryInitializers.Add(new MyTelemetryInitializer());

// Add a telemetry processor
var builder = config.TelemetryProcessorChainBuilder;
builder.Use((next) => new MyTelemetryProcessor(next));
builder.Build();

// Track a custom event
client.TrackEvent("VideoStarted", new Dictionary<string, string> { {"VideoId", "123"} });

// Track a custom metric
client.GetMetric("VideoDuration").TrackValue(120.5);

// Flush before shutdown
client.Flush();
Task.Delay(5000).Wait(); // Allow time for flush

A large online retailer needs to track every step of the checkout process to identify bottlenecks. They use TrackEvent with properties like StepName, CartValue, and PaymentMethod. They also track custom metrics like TimeSpentOnStep using GetMetric for pre-aggregation. The SDK is configured with a custom telemetry initializer that adds SessionId and UserId to all events. In production, the application handles thousands of checkouts per minute. To avoid overwhelming the ingestion endpoint, they enable adaptive sampling (default target 5 events/sec). However, they mark critical events (e.g., 'PaymentFailed') with SamplingPercentage = 100 by using a telemetry processor that sets item.SamplingPercentage = 100 for those events, ensuring no sampling. They also set up alerts on custom metrics to trigger when the checkout abandonment rate exceeds 20%.

A manufacturing company collects sensor data from thousands of IoT devices. Each device runs a .NET Core application that sends custom metrics (temperature, pressure, vibration) using TrackMetric. To reduce cost and storage, they configure fixed-rate sampling at 10% in the SDK. They also use a telemetry processor to drop metrics that are within normal range (e.g., temperature between 20-30°C) and only send anomalies. The telemetry channel is configured with a larger buffer (MaxTelemetryBufferCapacity=2000) to handle bursts. They also implement a custom channel that writes to a local file if the network is down, then replays when connectivity returns. This ensures no data loss during network outages.

A SaaS provider has a microservices architecture with 20 services. They use Application Insights for distributed tracing. Each service sends custom events for business operations (e.g., 'UserRegistered', 'SubscriptionUpgraded'). They use a telemetry initializer to set Cloud.RoleName to the service name, so they can filter telemetry by service. They also use a telemetry processor to add CorrelationId from the request headers to all custom events, enabling end-to-end tracking. The ingestion endpoint is configured with a custom endpoint for compliance reasons (e.g., https://dc.applicationinsights.azure.com). They monitor the live metrics to ensure no data loss and set up alerts on the rate of custom events per second to detect anomalies.