Get HTML content

spikeinc has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

Am trying something new. I want to collect some stats from a webpage using perl and parse it, and display it as output. The webpage contains a table of stats so when I run the script it should call the method invoked by html get (i can find it using fiddler tool) and then get the content of the table in a string and parse it.

I am kinda new to perl so can anyone help me get started? How can I achive this... thanks.

Comment on Get HTML content

Replies are listed 'Best First'.
Re: Get HTML content by kennethk (Abbot) on Aug 01, 2013 at 18:38 UTC
There is a large number of ways to attack this, but I'd suggest using LWP::Simple combined with Mojo::DOM for parsing the tree. HTML::Parser or HTML::TreeBuilder are also options for the parse. how to parse html perl should give you some basic codes. #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.	[reply]
Re: Get HTML content (imagine) by Anonymous Monk on Aug 02, 2013 at 03:42 UTC
I am kinda new to perl so can anyone help me get started? How can I achive this... thanks. Imagine you're not the first to ask :) See also htmltreexpather.pl and xpather.pl htmltreexpather.pl , Parsing HTML / Re^4: Parsing HTML, A regex question , NASA's Astronomy Picture of the Day / Re: NASA's Astronomy Picture of the Day , Re: Extracting HTML content between the h tags, Re^2: Help With Online Table Scraper, Re^4: web::scraper using an xpath, .... HTML Parser suggestions xpather.pl Re: Get Node Value from irregular XML (xpather.pl) Re: Having trouble with siblings Re^2: XML parsing and Lists Re: Counting number of child nodes based on element value (typos) Re^3: Extracting specific childnodes (xpath whitespace) Re^3: Extracting specific childnodes (play xmllint --shell ) Re: How do i get value of an element if the next elememnt has specific value in XML::LibXML using Xpath? Re: How do i get value of an element if the next elememnt has specific value in XML::LibXML using Xpath? Re: How to parse xml with namespase vale in XMl:LibXML? ( XPath error : Undefined namespace prefix ) Re^2: How to parse xml with namespase vale in XMl:LibXML? (xmllint --shell setns / xpathtester) There is a better way :) because xml::parser is low level, you should parse html/xml with xpath/twig/dom, Re: How to grab a portion of file with regex (don't), HTML Parser suggestions See also the real discouragement Oh Yes You Can Use Regexes to Parse HTML! and the real encouragement Re^2: parsing XML fragments (xml log files) with... a regex How do I match XML, HTML, or other nasty, ugly things with a regex? How do I remove HTML from a string? Re: Parsing webpages See htmltreexpather.pl , Parsing HTML / Re^4: Parsing HTML, A regex question , NASA's Astronomy Picture of the Day / Re: NASA's Astronomy Picture of the Day , Re: Extracting HTML content between the h tags, Re^2: Help With Online Table Scraper, Re^4: web::scraper using an xpath, .... HTML Parser suggestions	[reply]
Re^2: Get HTML content (imagine) by spikeinc (Acolyte) on Aug 02, 2013 at 19:00 UTC
Thankyou very much everyone! I love perlmonks :) definitely helps me to start shall post questions as I proceed :)	[reply]
Re: Get HTML content by PerlSufi (Friar) on Aug 01, 2013 at 21:23 UTC
Look into the modules kennethk recommended. I also like to use WWW::Mechanize. The `$mech->dump_text;` or `print $mech->$content;` approaches may work for you. The first method displays all of the text on the page, the second displays all of the HTML content of the page- including HTML elements.	[reply] [d/l] [select]