in reply to Re: Mail::MboxParser pegs the CPU
in thread Mail::MboxParser pegs the CPU

Actually, I've been doing that while waiting for a reply here - but I can't quite figure out what's going on. According to the docs, 'make_index' is supposed to run automatically as soon as I exec a 'get_message' method - but the cache file never gets created, no matter what I do (!). I've been trying to figure that out for the past hour or so; still haven't found anything like an answer.

What I really wish is that I actually understood this process of indexing (I have some hazy conception of saving pointers to message positions within the file, and then reusing those instead of traversing the entire file, but no clue of how to make that work efficiently.) I would have preferred to write that part myself, but had to rely on a module instead.


--
"Language shapes the way we think, and determines what we can think about."
-- B. L. Whorf

Replies are listed 'Best First'.
Re^3: Mail::MboxParser pegs the CPU
by zwon (Abbot) on Jan 15, 2010 at 20:09 UTC

    You can use something like this to create index: 

    use strict;
use warnings;

use Mail::MboxParser;

my $mb = Mail::MboxParser->new( 'mbox', );

my $ind = $mb->make_index;
for ( 0 .. $mb->nmsgs - 1 ) {
    printf "%5.5d => %10.10d => %s\n", 
        $_, $mb->get_pos($_), $mb->get_message($_)->header->{subject};
}

    [download]
[reply]
[d/l]
      > You can use something like this to create index:

      [download]

      I'm sorry to disagree, but - no, you can't. Per the docs: 

          enable_cache
            When set to a true value, caching is used but only if you 
+gave
            *cache_file_name*. There is no default value here!

    cache_file_name
            The file used for caching. This option is mandatory if
            *enable_cache* is true.

      [download]

      Neither of these is set in your code - and setting them does not create the specified file.

      I've tried explicitly using '$mb->make_index' in my code, by the way - in which both of the above are defined (please see the code I originally posted.) The cache file still does not get created (and, yes, I do have write permissions in that directory; an 'open' call in the script creates one without any problem.) At this point, it's beginning to look like a module bug.


      --
      "Language shapes the way we think, and determines what we can think about."
      -- B. L. Whorf
[reply]
[d/l]
[select]
        I'm sorry to disagree, but - no, you can't.

        Have you tried? 

        $ perl mailbox_parser.pl /home/zwon/Mail/FreeBSD-security | head -n 20
00000 => 0000000000 => FreeBSD Security Advisory FreeBSD-SA-04:03.jail
00001 => 0000007411 => [FreeBSD-Announce] FreeBSD Security Advisory Fr
+eeBSD-SA-04:04.tcp
00002 => 0000015717 => FreeBSD Security Advisory FreeBSD-SA-04:04.tcp
00003 => 0000024280 => [FreeBSD-Announce] FreeBSD Security Advisory Fr
+eeBSD-SA-04:05.openssl
00004 => 0000032992 => FreeBSD Security Advisory FreeBSD-SA-04:05.open
+ssl
00005 => 0000041959 => FreeBSD Security Advisory FreeBSD-SA-04:06.ipv6
00006 => 0000049688 => FreeBSD Security Advisory FreeBSD-SA-04:07.cvs
00007 => 0000058113 => FreeBSD Security Advisory FreeBSD-SA-04:08.heim
+dal
00008 => 0000069503 => FreeBSD Security Advisory FreeBSD-SA-04:09.kadm
+ind
00009 => 0000077885 => FreeBSD Security Advisory FreeBSD-SA-04:10.cvs
00010 => 0000086754 => FreeBSD Security Advisory FreeBSD-SA-04:11.msyn
+c
00011 => 0000094967 => FreeBSD Security Advisory FreeBSD-SA-04:12.jail
+route
00012 => 0000102596 => FreeBSD Security Advisory FreeBSD-SA-04:13.linu
+x
00013 => 0000112415 => FreeBSD Security Advisory FreeBSD-SA-04:14.cvs
00014 => 0000123949 => FreeBSD Security Advisory FreeBSD-SA-04:15.sysc
+ons
00015 => 0000131204 => FreeBSD Security Advisory FreeBSD-SA-04:16.fetc
+h
00016 => 0000141362 => FreeBSD Security Advisory FreeBSD-SA-04:17.proc
+fs
00017 => 0000151053 => FreeBSD Security Advisory FreeBSD-SA-05:01.teln
+et
00018 => 0000160935 => FreeBSD Security Advisory FreeBSD-SA-05:02.send
+file
00019 => 0000170195 => FreeBSD Security Advisory FreeBSD-SA-05:03.amd6
+4

        [download]

        I just replaced 'mbox' with $ARGV[0] in my example.

        PS: note, that cache is not the same as the index

        PPS: and you should have Mail::Mbox::MessageParser::Cache installed in order cache to be created, don't know why it is not mentioned in Mail::MboxParser documentation

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom