I just walked to the grocery store and back while thinking about it -- and I suspect you are correct. I'm able to log in and navigate using Mech, even though I see some Javascript on the page, but I think this download is asynchronous, thus it relies on JS, thus Mech can't catch it. Ugh.

I'd love to be able to try something like headless Chrome, but I don't think I'll be able to persuade my client to take that route. The low-tech alternative would be to have someone manually download the file every couple of days, and then upload it to the local server (which I manage). Thanks for your reply, and your original post -- it's always good to read about success stories.

It is also possible to capture the entire exchange (the dev panel(s) can do it) and see exactly what the JS is doing for headers and cookies and such. It should be pretty easy to then emulate the requests with plain HTTP in Perl code. It might be fragile but it's probably not too hard. Write a test and put it on a nightly cron to alert you if/when it breaks.

This is a brilliant idea, and I should have thought of it myself. Thanks!

Alex / talexb / Toronto

Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.