Hi there, I have a shell script that in turn calls a perl script. The function of the perl script is to read a file that contains thousands of lines (let's call it XMLList.txt), each line is the absolute path of an XML file, so the perl script takes each line as its input, opens each file and parses some nodes via XML::Twig.

The script seems to be working okay when XMLList.txt contains, say 10-15 lines.

Anything more and perl script seems to abruptly stop parsing half-way and the shell script (which calls the perl script) returns a "27000838 Segmentation fault(coredump)" error.

Should I take this to be a memory handling problem with my perl script? If yes, how can i optimize my perl script to better handle memory during the parsing?

One thing to note is that XMLList.txt can contain about 5000 lines and each XML file on that list could be, say about 20 MB each.

Here's the code for the perl script: 
#!/usr/bin/perl
use warnings;
use strict;

use Text::CSV_XS;
use XML::Twig;

my $csv = 'Text::CSV_XS'->new({
    sep_char => '|',
});

sub process_EDI_DC40 {
    my ($twig, $thingy) = @_;

    my @values = map {
        my $ch = $thingy->first_child( $_ );
        $ch ? $ch->text : ""
    } qw( DOCNUM MESTYP SNDPRN RCVPOR RCVPRN );
unshift @values,'XML';
$csv->say (*STDOUT, \@values);
}

my $listfile = shift;
open my $list, '<', $listfile or die $!;

my $twig = 'XML::Twig'->new(
    twig_handlers => {
        EDI_DC40 => \&process_EDI_DC40,
    },
);

while (my $xmlfile = <$list>) {
chomp $xmlfile;
    $twig->parsefile($xmlfile);
}

[download]

In reply to Segmentation coredump issue by Kal87

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.