Well, Google is competing a bit differently - they are offering to replace not just the servers, but also the software (mail, document sharing, calendar, etc.) that runs on the servers with their cloud/web services.
Still Java-Developer here, with all the bad habits: wouldn't you properly encapsule the use of something like S3? I haven't used it yet, but I imagine I would create some kind of EncodeResourceLink class, that creates a link to a resource an can be configured for a variety of storage solutions. Then some kind of open source clone of S3 could run on my own server. If the amount of data got overwhelming, I could easily shift everything over to S3, or whatever competitors emerge.