In this tutorial, we will, as the title suggests, look at the ways to reduce the AWS S3 storage costs. Now, before we venture into reducing S3 costs, let us just understand how storage is charged.
S3 charges users for timed storage. The unit is GB-Mo. In the standard tier, the users, at the time of writing this post, incur $0.023 for every GB of data stored for a month. You can find the latest S3 pricing information here.
If your object is deleted after 2 days of creation, you will be charged only for those 2 days. Thus, the time for which the object stays in the S3 bucket is as important as the size of the object when determining the cost. While the size of the object may not be in your hands, you can save a lot if you keep the object in S3 only for as long as it is required. That’s what we will primarily focus on in this tutorial.
Given below are a couple of ways of making sure that your S3 objects don’t last longer in the bucket than required.
Add Lifecycle Expiration Rules
If you have objects that are needed only temporarily, you can define lifecycle rules to automatically expire and delete objects some X number of days after they are created. This will ensure that you will not pay for objects after they are no longer needed.
You can create a lifecycle rule by going to the Management tab of your bucket and clicking on ‘Create Lifecycle Rule’. The images below show how to set a rule to expire all objects in the my_folder folder 7 days after their creation.
Please note that the above rule will not delete all objects in the folder at once. Objects created 7 days ago will be deleted today. Objects created today will be deleted 7 days later. Each objects will stay in the bucket for 7 days.
The cron of AWS that executes these lifecycle rules runs at 00:00 UTC every day. Since it runs only once a day, it is only possible to have expiration time in days, and not in hours or minutes.
Add Lifecycle Transition Rules
Some objects, while important, may be accessed very infrequently some time after creation, say after a month. In such cases, you can consider transitioning the objects from Standard storage to Glacier storage. While standard storage costs you $0.023/GB-Mo, glacier storage costs you $0.004/GB-Mo, 17% of the standard storage cost.
Please note that there is a transition cost associated with the movement of objects from standard to glacier storage. This cost is on a per-object basis (it is essentially a PUT request into glacier storage). At the time of writing this post, this cost was $0.005 for every 1000 objects. Thus, if you are moving a large number of small objects, you will incur a higher cost as compared to moving a small number of large objects, even if the combined size of the objects is the same in both cases.
The images below show how to create a lifecycle rule for transitioning all objects in the my_folder folder to Glacier storage 2 months after creation.
Please note that AWS will provide a warning every time you choose to transition objects to Glacier storage. They will quote an approximate number for the amount it will take to transition all objects in the entire bucket to Glacier/ Glacier Deep Archive. If you are going to transition only objects of a specific folder, like in the example above, you can ignore the number above, and make your own calculations, using the number of objects in that folder.
Also, note that Glacier is not the right storage class if you need to frequently access objects. Use it only when you are sure that you will not access the objects for months at a stretch.
Suspend Versioning
Did you know that each version of an object adds to the total storage? If you have no use of versioning (i.e., there is 0 probability of you restoring a past version), then simply suspend versioning in your bucket. If you are creating a new bucket, better keep versioning disabled right from the time of creation.
You can find the option to Suspend versioning by going to the Properties tab of your bucket.
Also, you can add lifecycle rules to delete non-current versions of the objects as well. If you absolutely need versioning, then you can add a rule to delete the previous versions, say, after 2 days of creation. This way, you will pay for the versions for only 2 days. The image below shows the settings for adding such a lifecycle rule.
I hope this tutorial helped. Know any other ways of reducing S3 costs? Then let me know in the comments below. If you liked this tutorial, then check out other tutorials related to AWS here.
Also, if you are planning to become a certified AWS Solutions Architect, I recommend that you check out this course on Udemy. I took this course and found the lectures to be lucid, to-the-point, and fun.