krivenburg has asked for the wisdom of the Perl Monks concerning the following question:

I am getting a strange error when I am trying to create a dbm file. My code will run for a long while and then will crash with the following error:

ndbm store returned -1, errno 27, key "SND2002020000018" at ./ResetLinks.pl line 72

The following is a snippet from my code showing how I am creating the file. 
$link_file_name = "$THIS_APP_DIR/d_link/ini/g166Links";
$link_dir_file = "$THIS_APP_DIR/d_link/ini/g166Links.dir";
$link_pag_file = "$THIS_APP_DIR/d_link/ini/g166Links.pag";

# Create new link database files
if ( dbmopen(%INI_records,$INI_file_name,0666) &&
     dbmopen(%link_records, $link_file_name, 0666) ) {
    my @all_files, $loop, $loop2, @loop3;
    my $i=0;

    # get list of files from data directory
    foreach $loop (sort `cd $data_dir; ls -d *`) {
        chop($loop);

        foreach $loop2 (sort `cd $data_dir; ls -d $loop/*`)
        {
            chop($loop2);

            foreach $loop3 (sort `cd $data_dir; 
                            ls -d $loop2/*`) {
                chop($loop3);
                $all_files[$i++]=$loop3;
            }
        }
    }

    foreach my $data_path (sort @all_files) {
        print "processing file $data_path\n";
        $data_path =~ s/[\n\r]//g;
        my ($this_table,$blank,$file) = 
               split(/\//,$data_path);

        my ($key_field_name,$function_name) = split
                   (/::/,$INI_records{$this_table});
        my $key_field = &get_key_field
               ($data_path,$key_field_name,$function_name);
        my $this_file_id = $this_table . "." . $file;

        if ( $key_field gt "" ) {
            if ( ! defined $link_records{$key_field} ) {
                $link_records{$key_field} = $this_file_id;
            }
            else {
????            if (length($link_records{$key_field}) + 
                    length($this_file_id) < 998) {
                    $link_records{$key_field} .= "::" . 
                           $this_file_id;
                }
            }
        }

[download]
I marked the line that is flagged by the error (????). The files get very large so I'm thinking that this is a size issue. Any suggestions would be appreciated.

Thanks, Kevin

Replies are listed 'Best First'.
Re: Error creating database file
by robartes (Priest) on Oct 24, 2002 at 11:57 UTC
    You are right in assuming this is a size issue: errno 27 (returned by ndbm store) is "File too large". What filesize are we talking about here?

    CU
    Robartes-

[reply]
      Robartes,

      Thanks for the reply. The .dir file is 152 MEG and the .pag file is 704 MEG.


      Kevin-

[reply]
        That adds up to 856M, which does not sound like a magic number at all. What filesystem is this, and do you know the filesize limit of said filesystem?

        It probably wouldn't hurt to check diskspace as well, although you would expect a different error in that case.

        CU
        Robartes-

[reply]
Re: Error creating database file
by graff (Chancellor) on Oct 25, 2002 at 02:40 UTC
    OK, you're on a unix box, so using "chop" instead of "chomp" is not a problem (but it is depricated, IIRC). I think you made a mistake copying the code into the post -- there's a problem in the section shown below, which I have commented (but if the code really looks like this, maybe this is part of the problem): 
     foreach $loop (sort `cd $data_dir; ls -d *`) {
     chop($loop);

     foreach $loop2 (sort `cd $data_dir; ls -d $loop/*`)
     {
         chop($loop2);

###      foreach $loop3 (sort `cd $data_dir; ls -d $loop2/*`) {
## should be:
##       foreach $loop3 (sort `cd $data_dir/$loop; ls -d $loop2/*`) {
## or:
         foreach $loop3 (sort `cd $data_dir; ls -d $loop/$loop2/*`) {
            chop($loop3);
## BTW, this:  $all_files[$i++]=$loop3;
## should be:
            push( @all_files, $loop3 );
         }
     }
 }

    [download]
    Apart from that, you're doing  $data_path = s/[\n\r]//g; which seems unnecessary -- you already chopped it in the loop above, and it's just a line of output from "ls".

    Then you're splitting $data_path on slash characters, which would never be present in $data_path given the code as posted, so $blank and $file are always empty strings, I think.

    I didn't see where &get_key_field() is defined; are you confident about what it's return value is, or should you be testing that?

    I'm sorry if this is all peripheral to the real problem, but I think it's worth looking into -- try "perl -d" on the script, set some breakpoints at key lines, and step through it to see whether the values of the variables match your expectations.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom