$parser->parse_chunk($summary)
[download]

my $summary = $quake->get("summary");
print $summary;
$parser->parse_chunk($summary);
while ( $summary = read_chunk_of_data() ) {
    $parser->parse_chunk($summary);
}
    
my $data = $parser->last_chunk();
    my $dd  = $data->get("dd");
    print $dd, "\n";
[download]

G'day mcoblentz,

It would appear that's poorly documented and read_chunk_of_data() is either pseudocode or a function you're supposed to write yourself.

I haven't used this module previously; however, something like this seems to be the intent:

$ perl -Mstrict -Mwarnings -e '
    use XML::Rules;
    use Data::Dumper;
    my @xml = qw{<some_tag> some content </some_tag>};
    my $parser = XML::Rules::->new(
        rules => [ _default => sub {$_[0] => $_[1]->{_content}} ]
    );
    for (@xml) {
        $parser->parse_chunk($_);
    }
    my $data = $parser->last_chunk();
    print Dumper $data;
'
$VAR1 = {
          'some_tag' => 'somecontent'
        };
[download]

Someone who's actually used the module before may have a better answer.

Update: Minor text change: s/write it yourself/write yourself/

or a function you're supposed to write it yourself.

XML::Rules does show read_chunk_of_data() in documentation for parse_chunk()

or a function you're supposed to write it yourself.

XML::Rules does show read_chunk_of_data() in documentation for parse_chunk()

I've got no idea why you wrote that. There is no suggestion by anyone that it doesn't appear in the documentation. Perhaps you could elaborate on what you meant.

I have changed "write it yourself" to "write yourself" which is what I originally intended; however, I can't see that either form changes the meaning with respect to what appears in the documentation.

-- Ken

Here is a simplified version of your script which uses XML::FeedPP to get the XML, XML::Rules to extract the data and HTML::TreeBuilder::XPath to extract the times from the summary. I have also included a simple regex to extract the times should you not be able to install the XPath module. Adapt as you require.

#!/usr/bin/perl -w
use strict;
use warnings;
use XML::FeedPP;
use XML::Rules;
use HTML::TreeBuilder::XPath;

# input
my $source = 'http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary
+/4.5_day.atom'; 
my $atom_xml = XML::FeedPP::Atom->new( $source )->to_string();

# output
my $outfile = "quake.txt";
open my $fh,'>',$outfile or die "$!";

# parser
my @rules = (
  _default => 'content',
  title    => \&title,
  entry    => \&report_item,
);
my $parser  = XML::Rules->new(rules => \@rules);

# process
report_header();
$parser->parse( $atom_xml );
close $fh;

sub title {
  my $title = $_[1]->{'_content'};
  'magnitude' => substr($title,2,3), 
  'place'     => substr($title,8); 
}

sub report_item {

  my $summary = $_[1]->{summary};
  
  # extract time from summary using XPath  
  my $tree = HTML::TreeBuilder::XPath->new_from_content($summary);
  my @dd = $tree->findvalues('//dd');
  
  # extract time using regex
  my $t1;
  my $t2;
  if ($summary =~ m!<dt>Time</dt>
  <dd>(.*)\ UTC</dd>
  <dd>(.*)\ at\ epicenter</dd>!x){
    $t1 = $1;
    $t2 = $2;
  }
   
  print $fh <<EOF
  Place     : $_[1]->{place}
  Magnitude : $_[1]->{magnitude}
  Updated   : $_[1]->{updated}
  Location  : $_[1]->{'georss:point'}
  Time Xpath: $dd[0]
              $dd[1]  
  Time regex: $t1 
            : $t2  
  Summary   : $summary 

EOF
}

sub report_header {
  my $cur_time = localtime;
  print $fh <<EOF
  # This Quake file created by quake_parsing_9
  # Matt Coblentz; Perl version unknown
  # For more information, see the USGS website
  # Last Updated: $cur_time
  
EOF
}
[download]

Here is a more simplified script which uses the get method on XML::FeedPP items as an alternative to XML::Rules.

#!/usr/bin/perl -w
use strict;
use warnings;
use XML::FeedPP;
use HTML::TreeBuilder::XPath;

# input
my $source = 'http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary
+/4.5_day.atom'; 

# output
my $outfile = "quake.txt";
open my $fh,">",$outfile or die "$!";

# process
report_header();

my $feed = XML::FeedPP->new( $source );
foreach my $quake( $feed->get_item() ) {   
  
  my $title     = $quake->get('title');
  my $magnitude = substr($title,2,3); 
  my $place     = substr($title,8);
  my $updated = $quake->get('updated');  
  my $locn    = $quake->get('georss:point');
  my $summary = $quake->get('summary');
  my $id      = $quake->get('id');
  
   # extract time from summary using XPath  
  my $tree = HTML::TreeBuilder::XPath->new_from_content($summary);
  my @dd = $tree->findvalues('//dd');
  
  # extract time using regex
  my $t1;
  my $t2;
  if ($summary =~ m!<dt>Time</dt>
  <dd>(.*)\ UTC</dd>
  <dd>(.*)\ at\ epicenter</dd>!x){
    $t1 = $1;
    $t2 = $2;
  }
  
  print $fh <<EOF
  ID        : $id
  Title     : $title
  Place     : $place
  Magnitude : $magnitude
  Updated   : $updated
  Location  : $locn
  Summary   : $summary
  
  Time Xpath: $dd[0]
              $dd[1]  
  Time regex: $t1 
            : $t2  

EOF
}
close $fh;

sub report_header {
  my $cur_time = localtime;
  print $fh <<EOF
  # This Quake file created by quake_parsing_9
  # Matt Coblentz; Perl version unknown
  # For more information, see the USGS website
  # Last Updated: $cur_time
  
EOF
}
[download]

poj

I tried this with your problem area:

    my $summary; # = $quake->get( "summary");
#   print $summary;
      while ($summary = $quake->get("summary")) {
        $parser->parse_chunk($summary);
      }
[download]

So the question is have you read in the whole document, and if so is there another method, say parse that should be used instead of parse_chunk?

UPDATE

I tried it again, this way using parse:

    my $summary = $quake->get( "summary");
    print $summary;
     #while ($summary = $quake->get("summary")) {
        $parser->parse($summary);
     #}
     #my $data = $parser->last_chunk();
     #my $dd = $data->get( "dd");
     #print $dd, "\n";
[download]

C:\Users\JKeys>perl \myperl\quake.pl
# This Quake file created by quake_parsing_9
# Matt Coblentz; Perl version unknown
# For more information, see the USGS website
# Last Updated: 1 17 5 2013, 4:34:55
#

junk after document element at line 1, column 601, byte 601 at C:/stra
+wberry/perl/site/lib/XML/Rules.pm line 745.
<p class="quicksummary"><a href="http://earthquake.usgs.gov/earthquake
+s/eventpage/usc000hsdj#pager" title="PAGER estimated impact alert lev
+el" class="pager-gree
n">PAGER - <strong class="roman">GREEN</strong></a> <a href="http://ea
+rthquake.usgs.gov/earthquakes/eventpage/usc000hsdj#shakemap" title="S
+hakeMap maximum estim
ated intensity" class="mmi-V">ShakeMap - <strong class="roman">V</stro
+ng></a> <a href="http://earthquake.usgs.gov/earthquakes/eventpage/usc
+000hsdj#dyfi" class="
mmi-IV" title="Did You Feel It? maximum reported intensity (5 reports)
+">DYFI? - <strong class="roman">IV</strong></a></p><dl><dt>Time</dt><
+dd>2013-06-16 21:39:0
9 UTC</dd><dd>2013-06-16 23:39:09 +02:00 at epicenter</dd><dt>Location
+</dt><dd>34.491&deg;N 25.087&deg;E</dd><dt>Depth</dt><dd>37.85 km (23
+.52 mi)</dd></dl>
[download]

So I'm still getting the "junk" message, this time from the parse method. Don't know if that's the feed, your code, or my tweaks. But it's sleepy time now

Hi,

I tried originally with 'parse' and several other XML methods. Because I had already read in the XML document and extracted the CDATA material, the "chunk" becomes an incorrectly formed XML document (it does not have a single root element wrapping all the other elements. The CDATA content starts with a paragraph tag, which stops in the middle, then picks up with some table elements). That's where the 'junk' error comes from - Perl is complaining about a poorly formed XML document.

I was a little embarrassed that I thought "read_in_some_data" was a real method - but the conversation has been really helpful - I honestly thought the whole documentation was trying to trick me and just couldn't figure out where my error was (silly me). Jenda had alluded to just picking up the CDATA fragments and re-parsing, which led me down a whole weird dead end. He has the right approach, I just interpreted his comments incorrectly.

This XML stuff is "fussy". I'm hoping to wrap my head around all of this because having scripts like this will make the code overall much easier to maintain. Just find the feed, parse, and go.

I'm just having trouble with the recursive bits of the overall process. The data changes and thus the details have to change.

Thanks to all for pitching in. This has been a useful discussion about feeds, XML, etc.

Matt

It had slipped my mind that the summary was CDATA as I didn't look back at the previous post. And you're right, it's the explanation for the junk message. If Perl is complaining about a poorly formed XML document, it's because we are trying to convince it that $summary is XML.

It isn't, of course, it's HTML. And that's what Jenda meant when he said

If you want to split that into pieces you have to pass that string to another HTML or XML parser.

I was about to suggest parsing $summary with LWP or HTML::Parser when I read poj's post. I like how he has simplified it and shown HTML::TreeBuilder handling $summary.