Dear Monks,

First let me thank you for accepted my registration and given the chance to interact the people like you who all are guiding us to become the Monster.

Thank you once again.

Let me start my first interaction for getting the suggestion instead providing. I have a question for file handling. I have two files named DUMP_ID and DUMP_CARD_NO and both the files contains the records of 37 million (this might be increase/decrease some quarter).

The scenario is - DUMP_ID contains the unique id of the card member and DUMP_CARD_NO contains unique id and the corresponding card account number. The unique id can have more than one card account number depends on the region/locale. Here I have to take a record from DUMP_ID and compare this unique id with DUMP_CARD_NO’s unique id; if it matches then unique id and the corresponding account number has to be write into an another file(remember the single unique id can have more than one card account number).

Currently, I take all the IDs from DUMP_ACCT_NO. and moved into the hashmap. 

while (<DUMP_ACCT_NO>) {
    chomp;
    my ($id, $accno) = split /\|/, $_;
    push @{$hashList{$guid}}, $accno;
}

[download]

Then take the ID from DUMP_ID and do the comparison with this hashmap. 

while(<DUMP_ID>){
    chomp;
    s/\s+//g;
    if ($hashList{$_}) {
        for my $accno (@{$hashList{$_}}) {
            print FINAL_FILE "$_|$accno\n";
        }
    }
}

[download]

Here the problem is, we are getting the out of memory; thus makes unable to complete the execution. Kindly guide me to go with the right approach.

Thank you.

In reply to Help on file comparison by Danu

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.