How can I use perl as an internet browser ?

palkia has asked for the wisdom of the Perl Monks concerning the following question:

Hi
(kinda new at perl programming)

I'm looking for a module that given an url will allow me to get to a point where I have an array,
the elements of which are strings of the same (as lose as possible at least) text lines,
that I would see in my internet browser as it opened the web page's url.
so basically in the point if I would print the array I would see the same (text only of course) thing as my browser would.
kinda like using perl as my internet browser (if I don't mind there are no images etc.. ^^).

thx

oh yeah almost forgot:
The general Idea is that I could have perl analyze a webpage's text content automatically, without me having to know html or javascript or anything else other than perl and the page's content's form.

Also it would be gr8 if the module could also handle tables (even if only tables of text).

And I would prefer a module that would handle any text languages,
not only English (if possible), or alternatively a 2nd module that could translate the character between languages,
as if the 1st module could do it in all languages (unicode thing ? never mind ^^).

The cpan search engine was as always no help without a specific module's name...

I tried LWP & LWP::simple (& some other LWP:***),
but if there is a way to use them 4 that, I couldn't figure it out myself.

thx again ;-)

Comment on How can I use perl as an internet browser ?

Replies are listed 'Best First'.
Re: How can I use perl as an internet browser ? by wind (Priest) on Mar 10, 2011 at 19:17 UTC
Sounds like you just need WWW::Mechanize and HTML::Parser HTML::Parser even has an example for pulling all the plain text out of an html file: htext Also, for a non-perl solution, check out lynx or just google text based browser	[reply]
Re: How can I use perl as an internet browser ? by ww (Archbishop) on Mar 10, 2011 at 19:33 UTC
Sounds (to me) like the 5 year old who "wants a pony" but doesn't want to have to feed or groom it, or muck out the stall. If you don't want to "know html or javascript," how are you going to deal with unexpected output -- if that happens and it's likely to -- from the script you're supposedly interested in writing? Does "`the page's content's form`" mean that you believe (mistakenly, if so) that all web content is contained in forms? And re "`I couldn't figure it out myself`," does that mean you did - - or, did not - - read the documentation for LWP::Simple ... and did you make any other attempt to find an appropriate module? So, sounds to me like you need to do some of your own homework -- starting with On asking for help & Company, and then write some code... after which you will generally get a warm welcome if you explain how/why/where you're failing to attain your objective.	[reply]
Re: How can I use perl as an internet browser ? by Anonymous Monk on Mar 10, 2011 at 20:14 UTC
If you can get WWW::Mechanize::Firefox up and running, then its just a SMOP :)	[reply]
Re: How can I use perl as an internet browser ? by zentara (Cardinal) on Mar 11, 2011 at 15:41 UTC
Try Perl Web Browser using Gtk2 and WebKit I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply]
Re: How can I use perl as an internet browser ? by JavaFan (Canon) on Mar 11, 2011 at 19:41 UTC
my @lines = `lynx -dump $URL`; [download] Note that where a browser puts its line breaks will depend on the widget size and the font.	[reply] [d/l]
Re: How can I use perl as an internet browser ? by palkia (Monk) on Mar 21, 2011 at 22:56 UTC
I used Mechanize in the en and it worx gr8. regardless thank you all (all but the spammer jerk on the 2nd comment) very much.	[reply]