You should really copy&paste not retype such information. It would be interesting to see the real timestamps before and after the download. That might give an indication of how much more data was downloaded when it runs out of memory relative to when it doesn't.
The obvious--though not necessarily feasible--solution would be to populate the database as you download the data, rather than accumulating it in a hash and then populating. LWP has a callback interface, see :content_cb that would allow you to get the data in chunks and store it on the fly, thereby avoiding the accumulation of data in memory.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply]
[d/l] |