I want to be a perl web browser -- what are my best options?

taint has asked for the wisdom of the Perl Monks concerning the following question:

Greetings,
Over the years, I've written several, ahem... PHP, web scripts to communicate
with my HTTPd server(s). I would use it for all sorts of tasks to discover it's
responses. It's helpful for debugging perl scripts, the server, etc. Making
sure the server, or script, is posting the correct response(s) -- HEAD, for
example, ensures the server configuration(s) I'm experimenting with, ultimately
respond the way I want it to -- not necessarily the way it is intended to
(thwarting miscreants, for example). Anyway, I love perl (tho FAR from proficient),
so I should really be writing this in perl not in that
other script/language. ;)
So I guess my question is, given the multitude of available HTTP::, HTML::,
LWP::, etc available. I was hoping others here that have dealt with similar
circumstances might share their experiences. So that I can hopefully find/use
the right/best one the first time. :)
The script(s) I've written in the past, were simply forms, that provided
fields that I would use, depending on the task I needed to perform (loaded
into a web browser, of course).

Thank you for all your time, and consideration.

use perl::always;
my $perl_version = "5.12.4";
print $perl_version;

Comment on I want to be a perl web browser -- what are my best options?

Replies are listed 'Best First'.
Re: I want to be a perl web browser -- what are my best options? by frozenwithjoy (Priest) on Jun 22, 2012 at 04:43 UTC
I can't tell you how to be a web browser, but if you are interested in web development, you should take a look at Mojolicious and Dancer. They both are reasonably easy to learn, fun, and pretty full featured web frameworks. They both have guides and tutorials. Mojolicious also has some screencasts which make it nice to get an overview of some of what's possible. There is also Catalyst, some of the developers of which made Mojolicious.	[reply]
Re^2: I want to be a perl web browser -- what are my best options? by taint (Chaplain) on Jun 22, 2012 at 05:06 UTC
Crap. It appears that I didn't explain my request well enough. What I'm looking for is a module, of bundle, that permits me to communicate with my web server(s), as though I was a web client (browser). Allowing me to post (speak) HTTP to the web server. Or, speak in the HTTP protocol. Sending request(s) && replies to the server, and seeing it's (servers) response(s). I hope this helps clarify my request better, and thank you for taking the time to respond. use perl::always; my $perl_version = "5.12.4"; print $perl_version;	[reply]
Re^3: I want to be a perl web browser -- what are my best options? by BrowserUk (Patriarch) on Jun 22, 2012 at 05:14 UTC
LWP::Simple serves most of my (admittedly simple) requirements. Even if your requirements are more complicated, it would be a good starting point while you get used to Perlish way. Beyond that, there is LWP proper and all it's secure connection extensions. And then WWW::Mechanize and similar beyond that. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]
Re^3: I want to be a perl web browser -- what are my best options? by zentara (Cardinal) on Jun 22, 2012 at 09:13 UTC
See Automating-Web-based-Data-Retrieval-with-Perl.htm for an introduction to WWW::Mechanize If you really want to BE a web browser, use Gtk2::WebKit, see Perl Web Browser using Gtk2 and WebKit for an example. I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply]
Re^4: I want to be a perl web browser -- what are my best options? by taint (Chaplain) on Jun 23, 2012 at 01:27 UTC
Re^5: I want to be a perl web browser -- what are my best options? by ww (Archbishop) on Jun 23, 2012 at 01:37 UTC
Re^3: I want to be a perl web browser -- what are my best options? by Corion (Patriarch) on Jun 22, 2012 at 07:12 UTC
Mojolicious also has a HTTP client. There also are Web::Magic and the Mechanize (and WWW::Scripter) family of modules which implement HTTP clients that behave like a web browser to various degrees. Personally, I like the API that WWW::Mechanize gives, but none of the modules is inherently bad.	[reply]
Re^3: I want to be a perl web browser -- what are my best options? by Ea (Chaplain) on Jun 22, 2012 at 15:28 UTC
There is a nice chapter on just this topic in Perl Testing by Lanworth and chromatic (see http://shop.oreilly.com/product/9780596100926.do ) and gives clear examples using WWW::Mechanize. You'll be up and testing in no time. To save you some time, here's a couple of hints from my brief experience: `# check the html on the page - skip this for brevity $mech->page_links_ok(); html_tidy_ok( $mech->content() );` [download] This does a lot of work in just two little lines. `$mech->content_contains( 'Sorry, we couldn’t find anything that +matched your search.' );` [download] This one took me forever to work out how to match the apostrophe. I went with the html entity in the end. perl -e 'print qq(Just another Perl Hacker\n)' # where's the irony switch?	[reply] [d/l] [select]
Re^3: I want to be a perl web browser -- what are my best options? by frozenwithjoy (Priest) on Jun 22, 2012 at 05:11 UTC
Ah, ok. Well, I'm definitely not an expert on this, so hopefully someone else can help you out. Also, I'm not sure if this is of use, but the things I linked do POST and well as GET, etc.	[reply]