in reply to Re^4: Are there any memory-efficient web scrapers?
in thread Are there any memory-efficient web scrapers?

Bottleneck? By that I assume you are referring to processing speed.

No. We're talking about memory usage. How did you determine the link parsing porting, is responsible for your 200MB process size? And that a solution that incrementally parses the HTML is the answer to reducing memory usage?

  • Comment on Re^5: Are there any memory-efficient web scrapers?

In Section Seekers of Perl Wisdom