Managing Versioning

Versioning allows to maintain multiple versions of an object in a bucket, providing protection against accidental overwrites and deletions.

By default, S3 Versioning is disabled on buckets, and you must explicitly enable it.

Enabling Versioning on a Bucket

To enable versioning, you first need to specify the bucket where you want it applied. Let’s say we want to enable versioning on an example bucket.

# First, let's create it.
aws --profile astran s3 mb s3://example
# Then, run the following command:
aws --profile astran s3api put-bucket-versioning --bucket example --versioning-configuration Status=Enabled

You should get no output if everything goes well.

See the versioning-enabled section to see how to manage objects for now on.

Disabling Versioning on a Bucket

If you no longer need versioning, you can suspend it.

First, we will explore the versioning-enabled bucket and then we will se how it works when you suspend versioning.

Checking the Versioning Status of a Bucket

You can check the versioning state of a bucket by running:

aws --profile astran s3api get-bucket-versioning --bucket example

For our bucket example, you should get the following output:

{
  "Status": "Enabled"
}

note

If versioning has never been enabled, there will be no output.

Managing Objects in a Versioning-Enabled Bucket

Uploading Objects in a Versioning-Enabled Bucket

When versioning is enabled, every time you upload a file, a new version is created. Let’s test this by uploading a file to our example bucket:

echo "Version 1 of my document" > document.txt
aws --profile astran s3 cp document.txt s3://example/document.txt

Now, modify the content of the file and upload it again:

echo "Version 2 of my document" > document.txt
aws --profile astran s3 cp document.txt s3://example/document.txt

Two versions of document.txt now exist in the example bucket.

Listing Object Versions in a Versioning-Enabled Bucket

To list all versions of the document.txt object, run:

aws --profile astran s3api list-object-versions --bucket example --prefix document.txt

The output of our example bucket should look similar to:

{
  "Versions": [
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_2",
      "LastModified": "2024-09-27T15:35:20+00:00",
      "IsLatest": true,
      "Size": 28
    },
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_1",
      "LastModified": "2024-09-27T15:34:50+00:00",
      "IsLatest": false,
      "Size": 28
    }
  ]
}

note

Objects that existed in the bucket before versioning was enabled are initially assigned a version ID of null. Enabling versioning doesn’t alter these existing objects. However, you can still reach them. We will see how it works in the suspended-versioning bucket section

Downloading Objects in a Versioning-Enabled Bucket

When versioning is enabled, you can retrieve either :

• The latest version (1)
• A specific version (2)

1. Downloading the Latest Version

By default, when you download an object without specifying a version ID, the most recent version is fetched. Let’s see how this works.

aws --profile astran s3 cp s3://example/document.txt downloaded_document.txt

This command will retrieve “Version 2 of my document” since it’s the latest.

2. Downloading a Specific Version

To download a specific version, you first need to list all versions of the object to obtain its version ID. Once you have the version ID, you can retrieve the desired version using the --version-id parameter, replacing version_id_1 with the version ID of the first uploaded version of document.txt obtained from the list.

aws --profile astran s3 cp s3://example/document.txt downloaded_version1.txt --version-id version_id_1

This command will retrieve the earlier version (“Version 1 of my document”).

Deleting an Object in a Versioning-Enabled Bucket

When versioning is enabled, deleting objects works differently compared to non-versioned buckets. Instead of being permanently deleted, a "delete marker" is placed, which makes the object appear deleted while still retaining the previous versions. This DeleteMarker makes the object inaccessible unless a specific previous version is targeted.

Deleting the Latest Version

If you delete an object in a versioning-enabled bucket, it will not be accessible anymore.

aws --profile astran s3 rm s3://example/document.txt

This command places a delete marker on document.txt, making any attempt to download it fail. However, the object’s previous versions remain in the bucket and can still be accessed by specifying the version-id parameter.

Thus, this command:

aws --profile astran s3 cp s3://example/document.txt document.txt

Should result in a 404 Not Found.

You can confirm the existence of the delete marker by listing all versions.

aws --profile astran s3api list-object-versions --bucket example --prefix document.txt

{
  "Versions": [
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_2",
      "LastModified": "2024-09-27T15:35:20+00:00",
      "IsLatest": true,
      "Size": 28
    },
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_1",
      "LastModified": "2024-09-27T15:34:50+00:00",
      "IsLatest": false,
      "Size": 28
    }
  ],
  "DeleteMarkers": [
    {
      "Key": "document.txt",
      "VersionId": "delete_marker_version_id",
      "LastModified": "2024-09-27T15:47:58+00:00",
      "IsLatest": true
    }
  ]
}

Here, we see that a delete marker has been added and is now the latest, and thus the current, version.

Here is a visualisation of how it works:

2. Deleting a Specific Version

If you want to delete a specific version of an object, you must use the --version-id parameter. For example:

aws --profile astran s3 rm s3://example-bucket/document.txt --version-id version_id_2

This command permanently deletes the specified version of document.txt from the bucket.

3. Removing the Delete Marker

If you want to restore access to an object deleted with versioning enabled, you can do so by removing the delete marker using its version-id.

aws --profile astran s3 rm s3://example-bucket/document.txt --version-id delete_marker_version_id

Once the delete marker is removed, the object will once again be accessible as if it was never deleted, with the most recent version becoming the current version.

4. Back to the first version !

Now that we’ve deleted both the latest version and the delete marker, the original version of document.txt has resurfaced as the current version. This means when you access document.txt without specifying a version ID, you’ll be retrieving the very first version you uploaded, as if no changes or deletions ever happened.

You can confirm this by running:

aws --profile astran s3 cp s3://example/document.txt restored_document.txt

This command will fetch the content of “Version 1 of my document,” showing that it’s back to being the active version.

Managing Objects in a Versioning-Suspended Bucket

Okay, so now that we’ve seen how versioning works when it’s enabled, let’s imagine you want to pause it. In this case, you can suspend versioning on your bucket.

When a bucket’s versioning status is set to Suspended, any new objects uploaded will no longer have version IDs, but existing object versions will remain intact, allowing you to retain historical versions even as new uploads occur.

aws --profile astran s3api put-bucket-versioning --bucket example --versioning-configuration Status=Suspended

Suspending versioning doesn’t delete existing versions. What changes is how Astran S3 handles objects in future requests. Let's see how it works.

Uploading an Object in a Versioning-Suspended Bucket

When versioning is suspended, no new version IDs are assigned to uploaded objects. More precisely, any non-versioned object will receive a "null" version ID, meaning that any new upload will overwrite the existing object with the same key and null version ID.

note

Objects that were uploaded before versioning was set in a bucket will also have a null version ID.

Let's test this by first suspending versioning on our example bucket:

aws --profile astran s3api put-bucket-versioning --bucket example --versioning-configuration Status=Suspended

Now, upload an object:

echo "Version 3 of my document" > document.txt
aws --profile astran s3 cp document.txt s3://example/document.txt

If you list the object versions, you will see that the IsLatest flag is now set to the newly uploaded version, which has a null version ID:

aws --profile astran s3api list-object-versions --bucket example --prefix document.txt

{
  "Versions": [
    {
      "ETag": "some_etag",
      "Size": 35,
      "Key": "document.txt",
      "VersionId": "null",
      "IsLatest": true,
      "LastModified": "2024-09-26T12:00:00.000Z"
    },
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_1",
      "LastModified": "2024-09-27T15:34:50+00:00",
      "IsLatest": true,
      "Size": 28
    }
  ]
}

Deleting an Object in a Versioning-Suspended Bucket

When working with a versioning-suspended bucket:

• A simple DELETE will target objects with a null version ID, replacing them with a delete marker.
• Without a null version, DELETE only adds a delete marker without affecting existing objects.
• You can still manage and permanently delete specific versions if needed.

Deleting the Latest Version

In a versioning-suspended bucket, a simple delete request (i.e. without specifying a version ID):

• Can only remove an object whose version ID is null.
• Inserts a delete marker with a null version ID.

Let's see what happens in our example bucket:

aws --profile astran s3 rm s3://example/document.txt

If you list object versions now:

aws --profile astran s3api list-object-versions --bucket example --prefix document.txt

You will see:

{
  "DeleteMarkers": [
    {
      "Key": "document.txt",
      "VersionId": "null",
      "IsLatest": true,
      "LastModified": "2024-09-28T12:30:00.000Z"
    }
  ],
  "Versions": [
    {
      "ETag": "etag_value",
      "Key": "document.txt",
      "VersionId": "version_id_1",
      "LastModified": "2024-09-27T15:34:50+00:00",
      "IsLatest": false,
      "Size": 28
    }
  ]
}

The null version ID object is now replaced by a delete marker, and the non-null version remain intact.

note

Content Loss: When a delete marker replaces the null version, you lose access to that content. The delete marker is essentially a placeholder indicating the object has been deleted, but it doesn’t store any data.

Deleting a specific version

If you want to permanently remove a specific version of an object in a versioning-suspended bucket, you can do so by providing the VersionId in the delete request.

Example:

aws --profile astran s3api delete-object --bucket example --key document.txt --version-id version_id_1

This action will permanently remove the specified version from the bucket.