S3 Replication: CRR and SRR

Amazon S3 Replication is a feature that automatically and asynchronously copies objects from a source bucket to one or more destination buckets. The destination buckets can be in the same AWS Region (Same-Region Replication, SRR) or in different AWS Regions (Cross-Region Replication, CRR). Replication is used to meet data residency requirements, improve data durability, reduce latency for geographically distributed users, and provide a backup in case of accidental deletion or regional failure.

When you enable replication on a source bucket, you define a replication rule that specifies:

Once the rule is active, S3 monitors the source bucket for object uploads and other events that trigger replication. When an object is uploaded (PUT), S3 asynchronously copies it to the destination bucket. The replication is performed internally by the S3 service; you do not need to manage any infrastructure. The time to replicate depends on object size and network conditions, but S3 provides a 15-minute Service Level Agreement (SLA) for most objects. For large objects (multi-part uploads), replication can take longer.

To configure replication using the AWS CLI, you first create an IAM role with the necessary permissions. Then you apply a replication configuration to the source bucket. Example JSON for a replication rule:

{
  "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
  "Rules": [
    {
      "Status": "Enabled",
      "Priority": 1,
      "Filter": {
        "Prefix": "logs/"
      },
      "Destination": {
        "Bucket": "arn:aws:s3:::destination-bucket",
        "StorageClass": "STANDARD_IA",
        "EncryptionConfiguration": {
          "ReplicaKmsKeyID": "arn:aws:kms:us-west-2:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab"
        }
      },
      "DeleteMarkerReplication": { "Status": "Enabled" },
      "SourceSelectionCriteria": {
        "ReplicaModifications": {
          "Status": "Enabled"
        }
      }
    }
  ]
}

To apply this configuration, use:

aws s3api put-bucket-replication --bucket source-bucket --replication-configuration file://replication.json

To verify replication status, you can check the ReplicationStatus metadata on objects using:

aws s3api head-object --bucket source-bucket --key logs/2023/01/01/app.log

The response includes ReplicationStatus which can be COMPLETED, PENDING, FAILED, or REPLICA.

By default, delete markers are not replicated. If you enable delete marker replication, when a delete marker is added to the source (i.e., an object is deleted), the delete marker is replicated to the destination. This means the object appears deleted in both buckets. However, if you delete an object version (permanent deletion), it is not replicated because version deletion is not supported by replication.

Replication copies the object’s metadata (system-defined and user-defined) and tags by default. With replica modification sync (SRR only), any subsequent changes to metadata or tags on the source are also replicated.

If replication fails (e.g., due to permissions, network issues, or bucket configuration), S3 will retry for up to 15 days. After that, the object is marked as FAILED in the replication metrics. You can use S3 Batch Operations to retry replication for failed objects. Common reasons for failure include incorrect IAM role permissions, missing bucket policy on the destination (cross-account), or the destination bucket being in a different region without proper VPC endpoints (if using VPC endpoints).

S3 Replication is a powerful feature for data durability, compliance, and performance. Understanding the differences between CRR and SRR, the prerequisites (versioning, IAM roles), and the configuration options (delete markers, replica modification sync) is essential for the SOA-C02 exam. Remember that replication is asynchronous and best-effort unless RTC is enabled. Always verify your setup by checking replication status on objects.

A financial services company needs to ensure that their transaction logs are available even if the entire us-east-1 region goes down. They configure CRR from their source bucket in us-east-1 to a destination bucket in us-west-2. They enable versioning on both buckets and set up a replication rule with a prefix filter to replicate only objects under the 'transactions/' prefix. They also enable Replication Time Control (RTC) to guarantee a 15-minute SLA for replication. The IAM role is created with least privilege permissions. They test the setup by uploading sample files and monitoring replication status. In production, they handle large volumes of log files (several TB per day) and find that CRR with RTC meets their RPO of 15 minutes. They also set up lifecycle policies on the destination to transition older logs to Glacier for cost savings.

A SaaS company runs microservices across multiple AWS accounts, each generating access logs stored in separate S3 buckets in the same region (us-east-1). They want to aggregate all logs into a central bucket for analysis. They configure SRR from each source bucket to the central destination bucket. Since SRR does not incur data transfer costs, this is cost-effective. They use prefix filters to replicate only log files. They enable delete marker replication so that if a log is deleted from the source, it is also deleted from the central bucket to maintain consistency. They also enable replica modification sync so that any tag changes on the source are reflected in the replica. The central bucket has a lifecycle policy to transition logs to S3 Standard-IA after 30 days.

For high-volume replication, consider using S3 Replication Time Control (RTC) to meet strict SLAs. Monitor replication metrics in CloudWatch. If you have many small objects, the number of PUT requests to the destination can become costly. Use S3 Batch Operations to replicate existing objects after initial setup. If you need to replicate to multiple destinations, you can create multiple rules, but each rule adds overhead. For large-scale scenarios, consider using S3 Multi-Region Access Points for a simpler configuration.