The main bandwidth savings I'm thinking about would come from heavy media files like images, which would be hosted on S3. The user's browser would request the HTML from my site, which would then serve them the HTML. Then their browser would parse the HTML, see the image links pointing to S3, and the browser would make a request directly to S3 for the image. Then S3 would serve the image directly to their browser. Nothing really special going on here; this is just the way the web plumbing works.

You do have to pay at S3 for the upload bandwidth, a one-time event, as well as for storage, (and obviously for the download bandwidth) but those are pretty cheap (pennies).

Then there might be a speed issue for some applications. Recently I noticed a site that was loading a bit slow, and the browser status bar showed it was waiting for images from S3. It was only about a one-second delay, but long enough to cause me to glance down at the status bar. Amazon claims their latency is OK (pretty vague, I forget their exact wording) but I'm skeptical. When I use them I'm planning to use CSS preloading of images on subsequent pages, in cases where it's likely those pages will be viewed.