CloudTrail Log File Validation

CloudTrail log file validation is a feature that allows you to verify that CloudTrail log files have not been modified, deleted, or tampered with after they were delivered by AWS. It provides cryptographic assurance that the log files you are analyzing are exactly as AWS created them. This is crucial for compliance, forensic investigations, and security auditing because logs are often the primary source of truth for detecting unauthorized activity.

When log file validation is enabled on a trail, CloudTrail performs the following:

Enabling Log File Validation (CLI):

aws cloudtrail create-trail --name my-trail --s3-bucket-name my-bucket --enable-log-file-validation

For an existing trail:

aws cloudtrail update-trail --name my-trail --enable-log-file-validation

Validating Log Files: AWS provides a Python script (validate-logs.py) that automates the validation process. You can download it from the AWS documentation or use the AWS CLI command:

aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-1:123456789012:trail/my-trail --start-time 2025-03-15T00:00:00Z --end-time 2025-03-15T23:59:59Z

The script: 1. Downloads the digest files for the specified time range. 2. Verifies the digital signature of each digest file using the public key. 3. Checks that the hash of the previous digest file matches the stored hash in the current digest. 4. Computes the hash of each log file listed in the digest and compares it to the stored hash. 5. Reports any mismatches or errors.

Manual Validation: You can also manually validate using OpenSSL and command-line tools. For example, to verify a digest file's signature:

# Extract the digest file (gzipped JSON)
gunzip -c digest.json.gz > digest.json

# Extract the signature (base64-encoded) from the JSON using jq
SIGNATURE=$(jq -r '.digestSignature' digest.json)

# Verify the signature using the public key
openssl dgst -sha256 -verify public-key.pem -signature <(echo $SIGNATURE | base64 -d) digest.json

Let's break down the validation process into discrete steps as a network engineer would observe:

{
  "digestVersion": 1,
  "digestPublicKeyFingerprint": "...",
  "digestSignature": "base64-encoded-signature",
  "previousDigestHash": "sha256-of-previous-digest",
  "previousDigestHashAlgorithm": "SHA-256",
  "digestStartTime": "2025-03-15T11:00:00Z",
  "digestEndTime": "2025-03-15T12:00:00Z",
  "logFiles": [
    {
      "s3Bucket": "my-bucket",
      "s3ObjectKey": "AWSLogs/123456789012/CloudTrail/us-east-1/2025/03/15/123456789012_CloudTrail_us-east-1_20250315T1100Z_abc123.json.gz",
      "hash": "sha256-of-logfile",
      "hashAlgorithm": "SHA-256"
    }
  ]
}

A financial services company must retain API activity logs for seven years and prove their integrity during audits. They enable CloudTrail log file validation on all trails across multiple accounts using AWS Organizations. The logs are stored in a centralized S3 bucket with a bucket policy that restricts write access to CloudTrail service and read access only to auditors. The company uses the validate-logs script weekly to generate an integrity report. They also set up an S3 lifecycle policy to transition older log files to Glacier, but they ensure digest files are retained with the same lifecycle. A common issue is that if a digest file is accidentally deleted (e.g., by a misconfigured lifecycle rule), the chain is broken for that hour, and auditors may flag it as a gap. To mitigate, they enable S3 versioning on the bucket to recover deleted digest files.

A large e-commerce platform uses CloudTrail to monitor all management events. After a security incident, the incident response team needs to verify that the logs have not been tampered with. They have log file validation enabled. They download the digest files for the period of the incident and run the validation script. The script confirms that all log files are intact. They then use the logs to trace the attacker's actions. If validation had failed, they would know the logs were compromised and would need to rely on other sources (e.g., VPC Flow Logs, application logs). In production, they run validation automatically as part of a CI/CD pipeline that triggers on new log delivery using S3 events and Lambda.

A global company uses a multi-region trail that delivers logs from all regions to a single S3 bucket in us-east-1. Log file validation is enabled. Digest files are also stored in the same bucket under the CloudTrail-Digest prefix, but only for the home region (us-east-1). The digest files contain log files from all regions. When validating, the script retrieves the digest files from the home region and then fetches log files from the same bucket (which may include objects from multiple regions). Performance considerations: validating millions of log files can take hours and generate significant S3 GET requests. They use the --start-time and --end-time parameters to narrow the scope. They also use S3 Inventory to list objects efficiently.