in reply to LWP::Simple::get($url) does not work for some urls

I think that LWP::Simple isn't picking-up on XHTML. What's happening is that Wikipedia is using a wiki markup dialect called MediaWiki, and the best way that I have found to get that url is to use HTML::WikiConverter. Try this:

#!/usr/bin/perl use strict; use warnings; use HTML::WikiConverter; my $wc = new HTML::WikiConverter( dialect => 'MediaWiki' ); print $wc->html2wiki( uri => 'http://en.wikipedia.org/wiki/Hotel' ), " +\n";
Updated: fixed typo

Replies are listed 'Best First'.
Re^2: LWP::Simple::get($url) does not work for some urls
by moritz (Cardinal) on Jul 07, 2008 at 08:59 UTC
    Maybe I misunderstood your reply, but I'm quite sure that LWP::Simple is ignorant to the content-type and just returns whatever the server sends. It doesn't complain when the content-type is xhtml and not html.
      I agree. The problem isn't with LWP::Simple, but with Wikipedia. Sorry if there was a misunderstanding.