I have 2 files that need to have certain data merged and other data discarded. Each line of the file is 210 characters long and has no separators. Here is the jist of what I am attempting to do:

1. From file_1 select specific character positions and match this in file_2.
2. From the matched characters in file_2 select the whole line and change certain character positions to some predefined values.
3. Output this to file_3.
4. If you encounter a line in file_1 that starts with the number 2, this is a special line that must be copied to file_3 unchanged and in the order it was encountered in file_1.

In my code I do not want to open file_2 in memory as these files will have a few hundred thousand lines. I have tried to use Tie::File and read it one line at a time but I am encountering the following error:

Can't modify negation (-) in predecrement (--) at ./SS7Merge line 62, near "$output_line ="
Execution of ./SS7Merge aborted due to compilation errors.

Here is my code: 

use File::Copy;
use Tie::File;

my($input_file1) = $ARGV[0];
my($input2) = $ARGV[1];
my($output_file) = $ARGV[2];

if ( !defined($input_file1) || !defined($input2) || !defined($output_f
+ile) ) {
    print "Error: usage: ./SS7Merge input_file1 input_file2 output_fil
+e\n";
}
else {
    # -----Backup the input files in case of error-----
    copy( $input_file1, $input_file1 . ".bak" ) or
        die "Could not backup file 1 $input_file1 to $input_file1.bak:
+ $!\n";
    copy( $input2, $input2 . ".bak" ) or
        die "Could not backup file 2 $input_file2 to $input_file2.bak:
+ $!\n";

    # -----Attempt to open all of the files-----
    open( INFILE1, $input_file1 ) || die( "Could not read input file 1
+ ($input_file1): $!" );
    open( OUTPUT, "> " . $output_file ) || die( "Could not open output
+ file ($output_file): $!" );

# We are going to read file2 into an array. The file will not be loade
+d into memory which will improve processing of large files.

tie @input2, 'Tie::File', \*FH, or die "Problem tying file $input2: $!
+"; 

while (<INFILE1>) {
        my $line = $_;
        chomp($line);

        # -----A line starting with a '2' is a header and is left unch
+anged
        if ( $line !~ m/^2/ ) {

            foreach $line2 (@input2) {
                $date = substr( $line, 6, 6 );
                $number_dialed = substr( $line, 29, 10 );

                if ( index( $line2, $date ) != -1 and index( $line2, $
+number_dialed ) != -1 ) {
                    $record_type = substr( $line, 5, 2 );

                    # -----From File2-----
                    $carrier_info = substr( $line2, 44, 5 );
                    $destination_number = substr( $line2, 122, 10 );

                    $connect_time = substr( $line, 54, 6 );
                    $send_to_OCN = substr( $line, 186, 4 );

                    $record_type = "25";
                    $send_to_OCN = "2604";

                    -----Generate the output string-----
                    $output_line = substr( $line, 0, 4 ) . $record_typ
+e . $date . substr( $line, 12, 17 )
                        . $number_dialed . substr( $line, 39, 5 )
                        . $carrier_info . substr( $line, 49, 5 )
                        . $connect_time . substr( $line, 60, 62 )
                        . $destination_number . substr( $line, 132, 54
+ )
                        . $send_to_OCN . substr( $line, 190, 20 ) . "\
+n";

#                    -----Debug code. Add in if you are experiencing p
+roblems-----
#                    print OUTPUT $output_line;
#                    print STDOUT "Output " . ++$outputcount . "\n";

                    last;
                }
            }
        }
        else {
            print OUTPUT $line . "\n";
        }
    }
# Untie the array before closing the file use

    untie @input2;

# -----Close all of the files-----
    close( INFILE1 );
    close( OUTPUT );
}

[download]

In reply to Merging specific data from 2 files into a third. by sheasbys

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.