gary441 has asked for the wisdom of the Perl Monks concerning the following question:

Hello all, Newbie here, just trying to learn perl. I'm trying to download a PDF file from a automotive site. Actually a Volvo site. It's a https site and I’ve been able to download the page with the link to the PDF I want, but now I can't parse the table with the link. I tried using HTML::TableExtract and other modules, but I need to parse out: CE1B85BDDDE8F58A852573BD003B1FBC/$file/vcmorarorp346601226200712262007944204.pdf or "CIS Report for 12/26/2007". I don't think I can parse by headers, depth or tag, because sometimes the information is not in the Volvo page. I want to put this perl script in a cron job and download the PDF every morning, but some days the PDF is late, so it might not be there or it will be another order. That's why I don't think I can parse by headers, depth or tag.

Edit:(removed links)

I've also looked into regex, but I think I need to know exact amount of tables and rows, but like said, sometimes that info is not in the same order or might not be there at all. I'm looking to learn and figure out the actually code myself (that's the best part), so any hints or reading material would be greatly appreciated. Thanks Gary

Replies are listed 'Best First'.
Re: parse a changing html table
by CountZero (Bishop) on Dec 28, 2007 at 21:24 UTC
    Don't listen to the insane ramblings of the Anonymous Monk.

    If you have to work some HTML-file, your best bet is one of the parser modules such as HTML::Parser.

    Looking at the structure of your HTML file, you will see that the info you are looking for is always contained within a a-tag and more specifically within the href-attribute in that tag.

    With HTML::Parser it is trivially easy to extract all a-tags from the HTML file and then use a regex to see if the attribute contains the string 'CIS Report for followed by a date. Extracting the date and the filename is simple. You probably just want to extract the filename for the date of today. That is left as an exercise for you.

    Here follows the basic program:

    use strict; use HTML::Parser (); # Create parser object my $p = HTML::Parser->new( api_version => 3, start_h => [\&a_tag, "tag, attr"], ); $p->parse_file('c:/data/perl/script/volvo.html'); sub a_tag { my $tag = shift; return unless $tag eq 'a'; my $hashref = shift; return unless $hashref->{'href'}=~m|'CIS Report for (\d{2}/\d{2}/\ +d{4})|; my $date = $1; (my $file) = $hashref->{'href'}=~m|PopupInfo\('(.*pdf)|; print "$date: $file\n"; }
    And it prints:
    12/26/2007: CE1B85BDDDE8F58A852573BD003B1FBC/$file/vcmorarorp346601226 +200712262007944204.pdf 12/26/2007: CE1B85BDDDE8F58A852573BD003B1FBC/$file/vcmorarorp346601226 +200712262007944204.pdf 12/25/2007: C0FBD06860B80B7B852573BC003B3738/$file/vcmorarorp346601225 +200712252007939900.pdf 12/25/2007: C0FBD06860B80B7B852573BC003B3738/$file/vcmorarorp346601225 +200712252007939900.pdf 12/24/2007: 31ED64C95AFF1A06852573BB003F55E5/$file/vcmorarorp346601224 +200712242007930592.pdf 12/24/2007: 31ED64C95AFF1A06852573BB003F55E5/$file/vcmorarorp346601224 +200712242007930592.pdf
    Remark: It seems each filename is mentioned twice in the HTML-page.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

      CountZero,
      First, thanks for all your help. I did figure out how to add in the date to help parse the correct info. And later today I will learn more about your code, but I understand most of it already. I do have more questions, but I rather read more and experiment, then I'll ask.

      Anon Monk,
      Don't worry, I intend to learn regex also. From what I read, it seems that I'd be better off learning both ways.

      Thanks Peeps, you both helped.
      Gary
        Ok, I've figured out how to add in today's date.
        Last night I studied your code and HTML::Parser, so I have a better understanding of both. Now I should be able to download the PDF file automatically every morning.
        Again, Thank You to everyone for the help and Thank You for taking the time to help.
        Should I post my code? I will if anyone thinks it will help someone else (and no one laughs). I will play around with regex. Thanks,
        Gary
      I am the insane anon. why html::parser? can not he roll his own?
        If the axle on your car breaks, do you

          mine some iron ore

          smelt it

          cast a new axle (it better not be too short, or too long)

          install it

        Or do you go to your mechanic to have a new one installed?
A reply falls below the community's threshold of quality. You may see it by logging in.