Set Up S3 Bucket for Backup With Automatic Expiration
When using S3 for offsite backups, you generally want to only maintain a certain number of backup sets, unless you happen to have an infinite AWS budget. We use S3 as a destination for Drupal’s Backup & Migrate module, and we generally keep around 2 months of snapshots in S3.
Fortunately, S3 has a number of options for automating the lifecycle of your backup sets, including changing the storage class, timed deletion, and archiving to Glacier.
Automatic Expiration
S3 allows you to set an expiration period on objects within a bucket, so that they will be automatically purged after that period. This works well for rolling backups sets. To set this up:
- Go to the S3 Management Console
- Create a new bucket to hold your backups.
- Add a Lifecycle Rule to purge objects after a specified period of time.
As you can see, there are a number of other options you can set via lifecycle rules, such as moving backups to Infrequent Access (a step between regular S3 and Glacier), move to Glacier, etc. If you need to keep backups indefinitely, you could instead use the lifecycle rule to move them to Glacier, without deleting them. Glacier is much less expensive on a monthly basis, but can be quite expensive when you need to get data back in a hurry.