AWS

How to enable the cross region replication on S3?

There are times when we just want more speed as well as redundancy of data stored in an S3 bucket. This is where the “cross region replication on S3 ” comes.

Question: What is actually “cross region replication“?

Answer: In short, it is nothing but creating multiple copies of data stored in an S3 bucket, which can be helpful in case of disaster recovery and high availability.

cross region replication on s3 bucket
cross region replication on s3 bucket

From the picture above, you can see that the bucket stored in “America = S3 bucket 2” is being replicated in the “Asia = S3 bucket 1” for high availability and redundant data.

Note: For enabling cross region replication on an S3 bucket, versioning must be enabled on the bucket.

Before going further, make sure you know -> How to create an S3 bucket

Also -> How to enable versioning on an S3 bucket

Step 1: Create an S3 bucket

In this, we have already created a bucket with the name “versioning-geekylane“, as shown in the picture below.

Moreover, you can notice that the “objects can be public” in this bucket.

Using the bucket created previously with versioning enabled
Using the bucket created previously with versioning enabled

Step 2: Add a cross region replication rule

For this, we have to go under the “Management” section of the “versioning-bucket” and after that, we have to go to “Replication” tab and from there select “+ Add rule” in order to add a cross region replication rule.

Add a cross region replication rule
Add a cross region replication rule

Step 3: Set Source Bucket

In this, we will be specifying which bucket we want to create a replication of, we are using “versioning-geekylane” as the source bucket, for reference check image below. After that click on the “Next” icon.

Set a source for the replication rule
Set a source for the replication rule

Step 4: Set Destination Bucket

Now, in this step, we have to specify in which bucket we want to save/copy the contents/objects of “versioning-geekylane” bucket.

Simply click on the “Create new bucket“.

For destination we are going to create a new bucket for cross region replication on S3
For destination we are going to create a new bucket

We are going to create a new destination bucket, with a name “geekylane-crr” in the “Region = Asia Pacific (Mumbai)“, for reference check the screenshot below.

After that, click on “Next” icon.

A new bucket geekylane crr in Asia mumbai is created for destination
A new bucket “geekylane-crr” in Asia mumbai is created for destination

Step 5: Set an IAM role

Here, we are going to create a new IAM rule for the cross region replication rule, the name of the “rule = geekylane-crr-rule” after that tick on the “Enabled” icon as shown below and click on “Next” icon.

Create a new IAM role name as geekylane crr rule
Create a new IAM role name as “geekylane-crr-rule

Step 6: Review the cross region replication rule

Before saving the changes being made to the rule, review them and then click on the “Save” icon.

Review the replication rule
Review the replication rule

Step 7: Cross region replication is created

From the screenshot below, we can confirm that our crr rule has been created successfully.

Cross Region Replication updated successfully
Cross Region Replication updated successfully

Step 8: Both the buckets are available – s3 cross region replication in multiple regions

From the image below, you can see that:

  • Source Bucket = versioning-geekylane (US East (N. Virginia)
  • Destination Bucket = Asia Pacific (Mumbai)

Both the buckets are situated in different regions.

Both the buckets are in different regions
Both the buckets are in different regions

Step 9: Check the Destination Bucket

Question: Wait! what? Why is the destination bucket empty?

Answer: It is because the cross region replication rule is only going to apply on the newly created files and not for already present files.

The new geekylane crr bucket is empty
The new “geekylane-crr” bucket is empty

Step 10: We are going to create a file

Note: We are using the file which we have used in the -> How to enable versioning on an S3 bucket

And this is going to be the “version 3” of the “geekylane.txt“, and have the following contents.

Updated version 3 of geekylane txt file
Updated “version 3” of “geekylane.txt” file

Step 11: Upload a file to Source bucket

In this, we are going to upload a file, which we have created in the “Step 10“, for reference check the image below.

After uploading the file, you will see a “100% Successful” message at the bottom of your S3 dashboard.

 Upload the new version to versioning geekylane
Upload the new version to “versioning-geekylane

Step 12: Check the Destination bucket

From below, we can see that the “geekylane.txt” file has bee replicated (copied) from the “versioning-geekylane” bucket to -> “geekylane-crr” bucket.

The geekylane txt is replicated to the the new bucket
The “geekylane.txt” is replicated to the the new bucket

Step 13: Try to access the file

We are still getting an ERROR on accessing the “geekylane.txt” using the Object URL from Browser.

Question: Why is that? What is happening?

Answer: Only the contents are being copied from “Source to -> Destination” bucket, and not the “Bucket Policies” and “Access Control List“.

File is not accessible
File is not accessible

Step 14: Make the file/object in the destination bucket public

In this, we are going to give the public access to the file “geekylane.txt” which is being copied from the source bucket.

To do so, follow the image below.

Make the file public
Make the file public

Step 15: Access the file Again

Now, the file “geekylane.txt” can be easily accessible from the “web browser” and do note that file is stored in the “destination bucket = geekylane-crr“.

 The file is now accessible
The file is now accessible

Question: Does deleting a file from the “Source bucket = versioning-geekylane” is going to delete the file from “Destination bucket = geekylane-crr“?

Answer: No, by default on the new file/object creations are going to be replication and not anything else and not even the deletion. The file is deleted from the “source” is not going be deleted from “destination“.

Step 16: Delete file from Source bucket

Delete the “geekylane.txt” file from the “source bucket = versioning-geekylane“.

 We delete the latest version of file from the source bucket
We delete the latest version of file from the source bucket

From the picture below, we can confirm that the latest version of the “geekylane.txt” file is being deleted from the “Source bucket = versioning-bucket“.

Source bucket is now left with two versions
Source bucket is now left with two versions

Step 17: Check the Destination bucket

From the contents of the “Destination bucket = geekylane-crr“, we can confirm that the latest version of the “geekylane.txt” is still available in it.

Destination bucket still have the latest version
Destination bucket still have the latest version

Step 18: Access “geekylane.txt” from both the buckets

From the image below, we can see that the “Destination bucket = geekylane-crr” is still showing the latest version (version 3) of the “geekylane.txt” but the “Source bucket = versioning-geekylane” is now showing the version 2 of the “geekylane.txt” file.

Which, as conclusion states that the deletion of objects stored in an S3 bucket is not going to be replication from the source bucket to the destination bucket if the cross region replication is enabled on s3 bucket.

Both have different versions because delete is not replicated
Both have different versions because delete is not replicated

More on S3:

More on AWS:

Comment here