http://qs1969.pair.com?node_id=126420

dataDrone has asked for the wisdom of the Perl Monks concerning the following question:

Hey Folks... Any one else have trouble with using the Headers method of extracting a table from a page? I've narrowed it down to this: If two columns have similar headers TableExtract chokes. My HTML is a follows.
DateDate of Incident Unit cost
11-14-2001
etc.... and when i make an array 
@heads = ("Date","Date of Incident","Unit Cost");
$headers = \@heads ;
$te = HTML::TableExtract->new(headers => $headers) ;
$te->parse($page);

[download]
I get Bubkiss. If i change the HTML and @heads array to "Date", "Rate of Incident" , and "Unit Cost" the Regex pattern match works fine. Also if i change to Date_of_Incident is works. Unfortunately I don't have control of the generation of the HTML. I have gotten around the problem by using brute force and Liberal doses of Ignorance as follows. 
@heads = ("myDate", etc...
$page =~ s/Date/myDate/gs ;
$te->parse($page);

[download]
Any suggestions? Thanks, The Drone.

Edit: footpad, ~Tue Nov 20 15:21:52 2001 (UTC)

Replies are listed 'Best First'.
Re: HTML::TableExtract revisited
by mkmcconn (Chaplain) on Nov 20, 2001 at 05:55 UTC

    That's a very curious problem, dataDrone. Apparently, the module will only succeed if it partly fails. For example, just switching the order of your headers makes the silly thing work.: 

    #!/usr/bin/perl -w
use strict;

my $row;

use HTML::TableExtract;
$/ = "";

my $string = <DATA>;
my $headers = ['Date of Incident','Date','Unit cost'];

my $te = HTML::TableExtract->new(headers => $headers) ;

$te->parse($string);

 # Examine all matching tables
foreach my $ts ($te->table_states) {
  print "Table (", join(',', $ts->coords), "):\n";
  foreach $row ($ts->rows) {
     print join(',', @$row), "\n";
  }
}

# use Data::Dumper ;
# print Dumper($te);

__DATA__
<TABLE>
<tr>
<th>Date</th>
<th>Date of Incident</th>
<th>Unit cost</th></tr>
<tr>
  <td>11-19-2001</td>
  <td>11-14-2001</td>
  <td>.01</td>
</tr>
<tr>
  <td>11-18-2001</td>
  <td>11-14-2001</td>
  <td>.01</td>
</tr>
</TABLE>

    [download]

    The explanation might be found if you uncomment those Data::Dumper lines above, and examine the output, where the pattern to be tested is printed out. A match is tested against this pattern:
    'hpat' => '((Date of Incident)|(Date)|(Unit cost))'
    But I gave it no further thought beyond this. Good luck.
    mkmcconn

[reply]
[d/l]
      Ah Yes.... having the hpat=>((Date of Incident)|(Date)|(Unit cost)) changes the way the pattern gets matched to the $self->htxt string. I think (very much not sure) that this is a Left associative regex problem. any Regex Gurus care to pitch in?
[reply]

Back to Seekers of Perl Wisdom