CloudWatch Embedded Metric Format (EMF)

CloudWatch Embedded Metric Format (EMF) is a JSON specification that enables you to embed custom metrics inside log events sent to CloudWatch Logs. Instead of making separate PutMetricData API calls to publish custom metrics, your application writes structured JSON log messages that CloudWatch automatically parses to extract metrics. This approach reduces the number of API calls, lowers latency, and simplifies your code—you only need to write logs, and metrics are derived from them.

Before EMF, developers had two choices for custom metrics: - Option A: Use the PutMetricData API directly. This requires explicit API calls, adds latency, and incurs costs per metric data point. - Option B: Log metric values as plain text and then create metric filters to extract them. Metric filters are processed asynchronously, incur additional costs, and have a limit of 100 metric filters per log group.

EMF eliminates these trade-offs. By embedding metrics in a structured log event, you get the simplicity of logging (no extra API calls) and the real-time availability of metrics (no filter delay). The metrics are extracted at ingestion time, so they appear in CloudWatch Metrics within seconds.

When your application sends a log event that conforms to the EMF schema, the CloudWatch Logs ingestion pipeline detects the special JSON structure and performs the following steps:

An EMF payload is a JSON object with the following required and optional keys:

- `_aws` (required): An object containing metadata. - `Timestamp` (optional): Integer in Unix epoch milliseconds. Defaults to log event timestamp. - `CloudWatchMetrics` (required): Array of metric definition objects. Each object has: - `Namespace` (required): String, the CloudWatch Metrics namespace. Must be non-empty and <= 255 characters. - `Dimensions` (required): Array of arrays of strings. Each inner array is a dimension set. For example, [["Service", "Region"], ["Service"]]. The first set defines two dimensions, the second defines one. Maximum 9 dimension sets, each with up to 10 dimensions. - `Metrics` (required): Array of metric objects. Each object has: - `Name` (required): String, the metric name. Must match a root-level key. - `Unit` (optional): String, the unit of the metric. Default is "None". Valid values include "Seconds", "Count", "Bytes", "Percent", etc. - `StorageResolution` (optional): Integer, either 1 (high resolution) or 60 (standard). Default is 60.

{
  "_aws": {
    "Timestamp": 1625443200000,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyApp",
        "Dimensions": [["Service", "Region"], ["Service"]],
        "Metrics": [
          { "Name": "ProcessingTime", "Unit": "Milliseconds", "StorageResolution": 1 },
          { "Name": "ErrorCount", "Unit": "Count" }
        ]
      }
    ]
  },
  "Service": "OrderService",
  "Region": "us-east-1",
  "ProcessingTime": 150.5,
  "ErrorCount": 0
}

This payload publishes two metrics (ProcessingTime and ErrorCount) in the MyApp namespace. The first dimension set is Service and Region; the second is only Service. So you get data points for: - Service=OrderService, Region=us-east-1 - Service=OrderService (aggregated over all regions)

EMF is enabled by default for all log groups. There is no explicit toggle. To verify that EMF metrics are being published:

EMF is designed for high throughput. Since metrics are extracted at ingestion time, there is no additional latency for metric processing. However, the log event must be a single line of JSON. If your application generates large JSON objects, ensure they stay under 256 KB. Also, be mindful of CloudWatch Metrics quotas: you can have up to 10,000 metrics per account per region. Each unique combination of namespace, metric name, and dimension values counts as a separate metric.

A large e-commerce company processes millions of orders daily using a microservices architecture. Each service (Order Service, Payment Service, Inventory Service) needs to emit metrics like processing time, error count, and throughput. Previously, each service made separate PutMetricData calls, resulting in thousands of API calls per second, high costs, and added latency. The team adopted EMF by integrating the AWS Lambda Powertools library into their Lambda functions. Now, each function logs a single EMF payload per invocation. For example, the Order Service logs:

{
  "_aws": {
    "CloudWatchMetrics": [
      {
        "Namespace": "ECommerce",
        "Dimensions": [["Service", "Stage"]],
        "Metrics": [
          { "Name": "ProcessingTime", "Unit": "Milliseconds" },
          { "Name": "OrderCount", "Unit": "Count" }
        ]
      }
    ]
  },
  "Service": "OrderService",
  "Stage": "prod",
  "ProcessingTime": 234,
  "OrderCount": 1
}

This reduced API costs by 90% and eliminated the need for metric filters. The team now has real-time dashboards showing order processing times across services. The main challenge was ensuring all services used the same namespace and dimension naming conventions to avoid metric fragmentation.

A smart building company collects temperature, humidity, and occupancy data from thousands of sensors. Each sensor sends data every minute. Using traditional PutMetricData would exceed API rate limits and incur high costs. The company uses an AWS IoT Core rule to forward sensor data to a Lambda function, which aggregates readings over 5-minute windows and logs an EMF payload. For example:

{
  "_aws": {
    "CloudWatchMetrics": [
      {
        "Namespace": "SmartBuilding",
        "Dimensions": [["Building", "Floor", "SensorType"]],
        "Metrics": [
          { "Name": "Temperature", "Unit": "Celsius" },
          { "Name": "Humidity", "Unit": "Percent" }
        ]
      }
    ]
  },
  "Building": "TowerA",
  "Floor": "5",
  "SensorType": "temperature",
  "Temperature": 22.5,
  "Humidity": 45
}

This approach allows them to publish hundreds of metrics per log event, staying well within CloudWatch Logs limits. However, they had to be careful about the 256 KB payload limit—aggregating too many sensors in one payload could exceed it. They also learned that if a dimension value is missing, the metric is still published but without that dimension, which can cause unexpected aggregation in dashboards.

In production, EMF payloads should be constructed efficiently. Avoid including unnecessary keys or large string dimensions. Use short dimension names and values to reduce payload size. For high-throughput systems, consider batching metrics into a single payload (up to 100 metrics) to minimize log events. Monitor CloudWatch Logs ingestion rate limits (5,000 requests per second per account per region by default) to avoid throttling.