huklberry has asked for the wisdom of the Perl Monks concerning the following question:

Is there a way to open the source of a remote webpage so that I can parse it?

Replies are listed 'Best First'.
Re: opening a webpage's source
by Zaxo (Archbishop) on Jun 26, 2005 at 06:31 UTC

    LWP::Simple's get() is one good way.

    Update: tlm++ corrected my memory of which LWP module defines get().

    After Compline,
    Zaxo

Re: opening a webpage's source
by ghenry (Vicar) on Jun 26, 2005 at 08:13 UTC

    This is quite handy:

    perl -MLWP::Simple -e 'getprint "http://www.google.com"'

    Walking the road to enlightenment... I found a penguin and a camel on the way.....
    Fancy a yourname@perl.me.uk? Just ask!!!
      GET 'http://www.google.com'

      works just as well from the prompt, but requires less typing. GET comes with LWP.

        GET doesn't come with every LWP installation, you're generally asked if you want it or not.

        Flavio
        perl -ple'$_=reverse' <<<ti.xittelop@oivalf

        Don't fool yourself.
Re: opening a webpage's source
by CountZero (Bishop) on Jun 26, 2005 at 08:34 UTC
    When going to a web-page, you always look at the source. It's the browser that makes it into something visual. The LWP-modules will help you.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

      Thanks for the help. The LWP module is exactly what I need.