S3 Server Access Logs and Request Monitoring

S3 Server Access Logs provide detailed records of requests made to an S3 bucket. Each log entry captures information such as the requester (AWS account or IAM user), bucket name, request type (GET, PUT, DELETE, etc.), HTTP status code, error codes, and timestamps. These logs are essential for security audits, compliance (e.g., PCI DSS, HIPAA), and troubleshooting access issues.

When you enable server access logging on a source bucket, S3 automatically generates log objects for every request to that bucket. The logs are written to a target bucket that you specify. The process is asynchronous and best-effort—S3 aggregates log records for a period (typically 5-15 minutes) and then delivers them as a single log object. This means logs are not real-time. The log object is a sequence of newline-delimited records, each representing a single request.

To enable server access logging via AWS CLI:

aws s3api put-bucket-logging --bucket source-bucket --bucket-logging-status '{"LoggingEnabled": {"TargetBucket": "target-bucket", "TargetPrefix": "logs/"}}'

Using the AWS Management Console: Go to the source bucket's Properties > Server access logging > Enable, select target bucket and prefix.

To verify logging is enabled:

aws s3api get-bucket-logging --bucket source-bucket

This returns the logging configuration if enabled.

Logs can be analyzed using Amazon Athena, Amazon EMR, or custom scripts. Athena is the most common exam-tested method. You create a table in Athena with the log format, then query with SQL. Example:

CREATE EXTERNAL TABLE s3_access_logs (
  BucketOwner STRING,
  Bucket STRING,
  RequestDateTime STRING,
  RemoteIP STRING,
  Requester STRING,
  RequestID STRING,
  Operation STRING,
  Key STRING,
  RequestURI_Operation STRING,
  HTTPStatus STRING,
  ErrorCode STRING,
  BytesSent BIGINT,
  ObjectSize BIGINT,
  TotalTime STRING,
  TurnAroundTime STRING,
  Referrer STRING,
  UserAgent STRING,
  VersionId STRING,
  HostId STRING,
  SignatureVersion STRING,
  CipherSuite STRING,
  AuthenticationType STRING,
  HostHeader STRING,
  TlsVersion STRING,
  AccessPointArn STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
  'input.regex' = '([^ ]*) ([^ ]*) \\[([^]]*)\\] ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) \\"([^\"]*)\" ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*)')
LOCATION 's3://target-bucket/prefix/';

Then query:

SELECT requester, operation, key, httpstatus, totaltime
FROM s3_access_logs
WHERE bucket = 'source-bucket'
  AND requestdatetime LIKE '2025-04-01%';

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "logging.s3.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::target-bucket/prefix/*",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:s3:::source-bucket"
        }
      }
    }
  ]
}

This ensures only the specified source bucket can write logs to the target.

A healthcare company stores patient records in S3 and must comply with HIPAA. They enable server access logging on all buckets containing protected health information (PHI). Logs are delivered to a centralized target bucket in a separate account with strict access controls. Every month, the compliance team runs Athena queries to identify any access from unauthorized IPs or users. They also set up a lifecycle policy to transition logs older than 90 days to Glacier for long-term retention. A common issue is missing logs due to misconfigured target bucket permissions—the logging service fails silently. The team uses CloudWatch metrics to monitor the size of log objects; if no new logs appear for 24 hours, they trigger an alert.

A financial services firm detects unusual activity: a large number of DELETE requests from an unexpected IP. They enable server access logging (if not already enabled) but realize that logs are not available for past events—logging must be enabled beforehand. They use the existing logs to trace the source IP, the IAM user (if any), and the objects deleted. They find that the requests came from a compromised access key. The logs also show the signature version and TLS version, helping them determine the client software used. The incident response team uses Athena to correlate with CloudTrail logs to identify the exact time the key was used. The limitation of log delivery delay (up to several hours) means they cannot respond in real-time, so they rely on CloudWatch alarms for immediate anomalies.

A large e-commerce company uses S3 for static assets and user uploads. They enable server access logging to analyze request patterns. By querying logs with Athena, they identify which departments (via requester) consume the most bandwidth and which objects are most frequently accessed. They use this data to optimize storage classes (e.g., moving cold data to S3 Glacier) and to charge back costs to business units. They also discover that bots are scraping their public bucket, causing high GET requests. They then implement a bucket policy to restrict access based on User-Agent. The logs help them tune their caching strategy via CloudFront. A common mistake is forgetting to set a lifecycle policy on the target bucket, leading to accumulating log storage costs.