I do have experience using perl to scrape lots of things, having done so for years. I don't believe there's anything Python brings to the party that perl can't already do.

Python is better at doing analysis or perhaps I should say 'machine learning' 'on the fly'. For example it may be better at following nodes (or leads) through a twitter feed. Python might be able to learn how to find the best pages. Otherwise your probably right, Perl would be faster than Python for most text processing. The regular expression capabilities of Perl, as build in, are better I am sure.

Python is better at doing analysis or perhaps I should say 'machine learning' 'on the fly'. For example it may be better at following nodes (or leads) through a twitter feed. Python might be able to learn how to find the best pages.

However, I doubt this is a feature of the Python language itself. It's more likely the libraries that you can load into Python that provide this functionality. Many libraries have APIs for multiple languages, so it'd probably be possible to use the exact same, or at least similar, libraries from Perl instead of Python, and then IMHO the advantages shift towards using only one programming language.

Python is better at doing analysis or perhaps I should say 'machine learning' 'on the fly'.

I still disagree.

Python is better at doing ...

... space research, nasa uses Python a lot to explore space :) sorry for the useless comment, I could not resist.

I think marto's use of the phrase "arms race" referred to the commercial sites you seem to want to interact with and their terms of service, and believe me, that's a tar-baby you don't want to start a fight with!