I tried to write a text-based perl web browser called PROWL: Perl->Request->Object->Web->Link().

My goal was to create a web browser to help automate the generation of web data mining code.

I used these modules:

• LWP::Simple
• Curses
• Curses::Widgets
• Text::Format
• HTML::TreeBuilder
• HTML::FormatText
• HTML::TableExtract

The biggest challenge was getting the HTML layout to work. I started to get tables to work, but I gave up when I ran into frames. If you ignore these issues, it is easy to write a browser.

To make the HTML easy to parse, I suggest Dave Ragget's tidy program. Tidy converts hard-to-parse HTML into easy-to-parse XHTML. A perl module to interface to the tidy C code would be useful. In addition to being helpful for writing a perl web browser, a tidy perl module would be useful for web site maintenance and data mining.

It should work perfectly the first time! - toma