Unless I mis-understand, LWP has nothing to do with arbitrary URL fetching of it's own accord. To impose a caching mechanism on it implies permanent storage of something, somewhere to use it. I'm sure I wouldn't be using it for much of anything if I had to provision some permanent file/folder/database just to use it.

My knee jerk theory is that 99.99% of the traffic problem is caused by web bots and spiders that simply strip any URL they find out of the page and keep on trucking. I'll even go so far as to say W3C did it to themselves by specifying a "http://" URL in the first place. I really don't think they've got any choice now but to live with the consequences.

On the other hand if they can embed some google adds in those DTDs they're probably sitting on a gold mine!