Parse HTML content

mailmeakhila has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I am using WWW::Mechanize to interact with a website. I have used 'get' method to get the html content of the website. Now i am trying to use HTML::Parser to remove few of the tags that are in the html content. Does anyone has any example of how can i pass the $mech->content to parse method in HTML::Parser. I am trying to use the below code.

$parser->parse($mech->content)
[download]

i also tried this approach

while($mech->content) {

$parser->parse($_);

}
[download]

Any suggestions?

Comment on Parse HTML content Select or Download Code

Replies are listed 'Best First'.
Re: Parse HTML content by Eliya (Vicar) on Mar 09, 2012 at 17:54 UTC
I am trying to use the below code. `$parser->parse($mech->content)` That should work. What happened when you tried?	[reply] [d/l]
Re^2: Parse HTML content by mailmeakhila (Sexton) on Mar 09, 2012 at 18:03 UTC
i am getting 500 internal server error. As this is a cgi script. The script goes fine in the terminal but as the html file is too large i have to see whats wrong with it as cgi page.	[reply]
Re^3: Parse HTML content by bitingduck (Deacon) on Mar 09, 2012 at 18:14 UTC
Check your server logs to see what the error is-- it's likely a permissions problem, but depending on the server could also be a Perl version problem. For example, Mac OS X has multiple Perl installs and you can write something that works fine from the terminal, then the CGI handler uses a different version of Perl that doesn't have any of your modules installed. Particularly annoying, since OS X depends on having the version it wants, so you end up with 2 or 3 (or more) installs and having to sort between them. But it's most likely permissions, since the CGI will run as a different user than you run as in command line mode. There are other possibilities, as well, and your server's error log should give you a clue where to look EDIT: also, if you really think it's the size of the target HTML, an easy check would be to put a mini-version of the target page up (or something similar) that has just enough data for your CGI to act on. You could even do it as dummy data embedded in the Perl code itself-- that would let you test that possibility quickly. But look at the error logs first- it might tell you exactly what the problem is	[reply]
Re^3: Parse HTML content by Eliya (Vicar) on Mar 09, 2012 at 18:13 UTC
The first step when getting an internal server error would typically be to check the error log of the webserver. The script goes fine in the terminal but as the html file is too large... How large is the HTML? Maybe the webserver process is running under a memory limit lower than your terminal user? (assuming you've verified that the CGI script runs in principle...)	[reply]
Re^4: Parse HTML content by mailmeakhila (Sexton) on Mar 09, 2012 at 20:36 UTC
Re^5: Parse HTML content by Eliya (Vicar) on Mar 09, 2012 at 20:44 UTC