I spent 30 minutes trying to figure out what the )*(&)&@#! was wrong with my program... it turned out I had sort of a functional mindset about what I was doing that led to problems... I figured it out, but only after I had documented and described my problem completely.

Have fun sleuthing out what appears to be a problem with HTML::Tree but in fact has to do with my own oversight... good luck Sherlock!

SYNOPSIS

My goal is to take the td elements of a tr element and replace any td element with a colspan of 2 with 2 td elements.

Now, the input HTML has 6 td elems, one of which has a colspan of 2.

So, the goal is for the output HTML to have 7 td elems.

For some reason, even though a structural dump of the transformed HTML shows 7 elements (up from the 6 elements) as expected, when I do a look_down on the tr, I only get 6 elements, not 7.

use strict; use warnings;
use Data::Dumper;
use LWP::Simple;
use HTML::TreeBuilder;


my $html = '<table<tr><td><a href="http://files.gokgs.com/games/2009/4
+/1/metaperl-2.sgf">Yes</a></td><td colspan="2"><a href="gameArchives.
+jsp?user=metaperl">metaperl [11k]</a> (<a href="gameArchives.jsp?user
+=shygost">shygost [-]</a> vs. <a href="gameArchives.jsp?user=metaperl
+">metaperl [11k]</a>)</td><td>19×19 </td><td>4/1/09 3:20 AM</td><td>R
+eview</td><td>Unfinished</td></tr></table>';


my $tree = HTML::TreeBuilder->new_from_content($html);

my $table = $tree->look_down('_tag' => 'table');

my @potential_game = $table->look_down('_tag' => 'tr');

warn scalar @potential_game, ' potential games';

my @viewable_game = grep {
  my $elem = $_;

  my $game_link = $elem->look_down(href => qr!files.gokgs.com/games!);

  if ($game_link) {
    my $content = ($game_link->content_list)[0];
    $content eq 'Yes';
  } else {
    ();
  }

} @potential_game;

warn scalar @viewable_game, ' viewable games';


sub fix_colspan {
  my($td)=@_;

  my $colspan = $td->attr('colspan');
  if (defined($colspan) and $colspan == 2) {
    my @td = (HTML::Element->new_from_lol( [td => 'a'] ) );
    push @td, HTML::Element->new_from_lol( [td => 'b'] ) ;
    $td->replace_with(@td)->delete;
  }
}

for my $game (@viewable_game) {
  my @td = $game->look_down('_tag' => 'td');

  warn 'pre_colspan has ', scalar @td, ' td elems' ;
  $game->dump;

  fix_colspan($_) for @td;

  warn 'post_colspan has ', scalar @td, ' td elems' ;
  $game->dump;


}
[download]

program output

Administrator@LIFEBOOK ~/prg/games-go-kgs : perl simple-case.pl 
1 potential games at simple-case.pl line 16.
1 viewable games at simple-case.pl line 32.
pre_colspan has 6 td elems at simple-case.pl line 49.
<tr> @0.1.0.0 (IMPLICIT)
  <td> @0.1.0.0.0
    <a href="http://files.gokgs.com/games/2009/4/1/metaperl-2.sgf"> @0
+.1.0.0.0.0
      "Yes"
  <td colspan="2"> @0.1.0.0.1
    <a href="gameArchives.jsp?user=metaperl"> @0.1.0.0.1.0
      "metaperl [11k]"
    " ("
    <a href="gameArchives.jsp?user=shygost"> @0.1.0.0.1.2
      "shygost [-]"
    " vs. "
    <a href="gameArchives.jsp?user=metaperl"> @0.1.0.0.1.4
      "metaperl [11k]"
    ")"
  <td> @0.1.0.0.2
    "19×19 "
  <td> @0.1.0.0.3
    "4/1/09 3:20 AM"
  <td> @0.1.0.0.4
    "Review"
  <td> @0.1.0.0.5
    "Unfinished"
post_colspan has 6 td elems at simple-case.pl line 54.
<tr> @0.1.0.0 (IMPLICIT)
  <td> @0.1.0.0.0
    <a href="http://files.gokgs.com/games/2009/4/1/metaperl-2.sgf"> @0
+.1.0.0.0.0
      "Yes"
  <td> @0.1.0.0.1
    "a"
  <td> @0.1.0.0.2
    "b"
  <td> @0.1.0.0.3
    "19×19 "
  <td> @0.1.0.0.4
    "4/1/09 3:20 AM"
  <td> @0.1.0.0.5
    "Review"
  <td> @0.1.0.0.6
    "Unfinished"
Administrator@LIFEBOOK ~/prg/games-go-kgs :
[download]