We recently bought a site at work, which we’re in the process of revamping and relaunching. The original developers stored a few terabytes of user data in S3 buckets, which they were (ab)using as a CDN-lite. A large chunk of the data will be dumped due to dubious provenance, but that still left some seven million user images to be recovered from S3.
And fairly unsurprisingly, it turned out that storing many small files in S3, without local backups, is a complete pain in the neck. There are no mechanisms for manipulating data within S3 - no tarring, archiving, or heading over to the CoLo and ripping a hard disc out. Just “key goes in, value comes out”.
And so, the only way to pull the data out en masse was to make Seven. Million. HTTP. Requests. Three long weeks watching a very dumb process crawl down the screen.
If by some mischance I ever end up designing a system that stores data “in the cloud”, I’m making damn sure there’s a local copy.