I havent seen this idea in your list, no idea whether it is viable:
What about providing a zipped download of the whole site, updated weekly or so?
Maybe stripped down from unnecessary/redundant info, so it is actually easier to use the download than scraping?