Obviously, we need to backup all this data. And obviously, a consumer product like Dropbox or SugarSync is not an option. We needed something fast, reliable & scalable. We needed cloud storage. So we chose Amazon S3 and I decided to write this simple step-by-step guide for anyone interested in automating S3 backups on a Windows server:
1: Create an Amazon AWS account
If you don't already have an AWS account - create it here, it's free. Amazon's "free usage tier" on S3 gives you 5GB free storage from scratch, so after registering, sign in to your "AWS Management Console", select the "S3" tab and create one or more "buckets" on the left.

2: Get your access keys
You will need security credentials to access your online storage from the server, so click your account name - "Security Credentials" - "Access Keys" and copy your Key ID and Secret.
3: Download "S3Sync"
"S3Sync" is a great free command-line application from SprightlySoft. It is .NET-based and even comes with the source codes. At the time of writing this post their website was down, so I published the tool on Google Docs here: S3Sync.zip.
The tool syncs a given folder with your S3 bucket. And the best part - unlike similar scripts and utilities it performs a "smart" differential sync that detects additions, deletions and file-modifications.
4: Write a backup script
Create a batch file and paste this code into it:
S3Sync.exe -AWSAccessKeyId xxxxxxx -AWSSecretAccessKey xxxxxxx -SyncDirection upload -LocalFolderPath "C:\inetpub\wwwroot" -BucketName YOURBUCKETNAMEThe code above is pretty self-explanatory. Just replace the "xxxxxx" with your access codes from #2, "YOURBUCKETNAME" with the name of your S3 bucket, and "C:\inetpub\wwwroot" - with the folder you want to backup. Then create a scheduled task that runs the batch file every 24 hours, and you're all set.
5: Pricing
Amazon gives you a 5 GB free storage during the first year. After the year ends the pricing is still very moderate. For instance, a 10GB storage will cost you about 60 cents a month which makes it a no-brainer.

6 comments:
There is another way to back up the data to S3. Try CloudBerry Backup. It is powered by Amazon S3 reliable and cost efficient storage. Unlike other vendors we have one time fee and no recurring charges. Besides, there is no proprietary data format and you can access your data using other Amazon s3 tools. Supports all Amazon S3 regions and Reduced Redundancy Storage.
Andy, CloudBerry Backup is not free. S3 Sync is.
What about uploading with Reduced Redundancy?
Does this still work for you? I always get a 403 error when S3Sync tries to connect.
This works but tries to follow junctions and errors out. Is the code open sourced?
Getting local files.
Error in application.
Message=Error listing local files. CurrentFolder=C:\Usersuser\AppData\Loca
l\Application Data\ ErrorMessage=Access to the path 'C:\Users\user\AppData\
Local\Application Data' is denied.
StackTrace= at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String
CurrentFolder, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslAr
rayList, String IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFiles
RegularExpression, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.Main(String[] args)
Unhandled Exception: System.Exception: Error listing local files. CurrentFolde
r=C:\Users\user\AppData\Local\Application Data\ ErrorMessage=Access to the
path 'C:\Users\user\AppData\Local\Application Data' is denied.
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.PopulateLocalHashtable(String BaseFolder, String CurrentFol
der, Boolean IncludeLocalSubFolders, ArrayList ExcludeLocalFolderslArrayList, St
ring IncludeOnlyLocalFilesRegularExpression, String ExcludeLocalFilesRegularExpr
ession, Hashtable& LocalHashtable)
at S3Sync.Program.Main(String[] args)
Is there a way to specify a subfolder under s3 bucket?
Post a Comment