Esteemed Monks,

I'm trying to insert about ~5000 records into a Postgres (v. 8.0.1) database.

This is a simple program that reads MailDir dirs, opens each mail, extracts some parts from it and saves it into the database. 

#!/usr/bin/perl -Tw

use diagnostics;
use strict;
use IO::All;
use DBI;

my $db = DBI->connect( "dbi:Pg:dbname=db;",
    "user", "password" 
) or die( "error message ..." );
$db->{RaiseError} = 1;
$db->{pg_server_prepare} = 1;
$db->{AutoCommit} = 1;

my @MAILDIRS = (
    '/path/to/maildir/1/',
    '/other/path/to/maildir/2/',
);

my $sql = "
    INSERT INTO tabmail(para,de,assunto,data,corpo,cabecalho)
    VALUES(?,?,?,?,?,?)
";
my $sth = $db->prepare($sql);

my $i = 0;

foreach my $dir (@MAILDIRS ) {

    for my $mail ( io($dir)->all ) {

        print "\n", $i, ": ",io($mail)->filename,"\n";

        my $flag = 0;
        my ($headers,$body);
        my ($from, $to,$date,$subject);

        foreach my $linha ( io($mail)->getlines ) {

            $from = $linha if $linha=~/^From: .*/;
            $to =  $linha if $linha=~/^To: .*/;
            $date = $linha if $linha=~/^Date: .*/;
            $subject = $linha if $linha=~/^Subject: .*/;

            if ( $linha =~/[^\n].*/ && $flag != 1 ) {
                $headers .= $linha,
            } elsif ($linha=~/^\n/ && $flag != 1) {
                $flag =1;
            } else {
                $body .= $linha;
            }

        }

        $sth->execute(
            $to || "-",
            $from || "-",
            $subject || "-",
            $date || "-",
            $body || "-",
            $headers || "-"
        );
        $i++;
    }
}

[download]

All runs fine (records are stored) until 1017 records. Then stops with: 


  +----------------Record number
  |     +----------File name
1017: 1911
Can't locate Carp/Heavy.pm in @INC (@INC contains:
        /usr/lib/perl5/5.8.6/i486-linux /usr/lib/perl5/5.8.6 /usr/lib/
+perl5/site_perl/5.8.6/i486-linux /usr/lib/perl5/site_perl/5.8.6 /usr/
+lib/perl5/site_perl) at /usr/lib/perl5/5.8.6/Carp.pm line 255, <GEN73
+27> line 77 (#1)
    (F) You said to do (or require, or use) a file that couldn't be
    found. Perl looks for the file in all the locations mentioned in @
+INC,
    unless the file name included the full path to the file.  Perhaps 
+you
    need to set the PERL5LIB or PERL5OPT environment variable to say w
+here
    the extra library is, or maybe the script needs to add the library
+ name
    to @INC.  Or maybe you just misspelled the name of the file.  See
    perlfunc/require and lib.

Can't locate Carp/Heavy.pm in @INC (@INC contains: /usr/lib/perl5/5.8.
+6/i486-linux /usr/lib/perl5/5.8.6 /usr/lib/perl5/site_perl/5.8.6/i486
+-linux /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl) at /u
+sr/lib/perl5/5.8.6/Carp.pm line 230, <GEN7327> line 77.

[download]

No matter what email is at that position (1017). Always crashes with the same error message.

What am I missing here?

Thank you,
Miguel

Retitled by BazB from 'Inserting 1000s of records in a database'.

In reply to Problem locating Carp::Heavy by Miguel

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.