in reply to Re: Script not working on large file size
in thread Script not working on large file size

Ok... That is my script
#!/usr/bin/perl -w use Fcntl; use Getopt::Std; use constant DIR => "/d/testing"; use constant MAX => 11; my $prog = $0; $prog =~ s|.*/||; my %opts; getopts('o:', \%opts); my $cmd = $opts{o} || 1; my @date = localtime(time); my $dd = sprintf ("%02d", $date[3]); my $mm = sprintf ("%02d", $date[4] + 1); my $yy = sprintf ("20%02d", $date[5] % 100); my $yymmdd = "$yy$mm$dd"; my @a; my $bnum; my $template = "A12 A4 A20 A20 A2 A2 A2 A2 A12 A4 A24 A24"; unshift (@ARGV, "-") unless @ARGV; for my $mfile (@ARGV) { $file = $mfile; open (IN, "< $file") or die ("Can't open $file: $!\n"); while (<IN>) { @normalDb = (); @errorDb = (); s/\s//g; $dd = $_; my @a = unpack($template, $dd); $sdate = $a[8]; $state = $a[4]; $err = $a[7]; $bnum = substr($a[3], skip_zero($a[3])); substr($dd, 37, 0) = '00' if ( $state eq '02' ); substr($dd, 62, 3) = '01' if ( $sdate !~ /^0/ && $err !~ /^0/ + ); substr($dd, 62, 3) = '01' if (length($dd) == 131); #The 2 last character of lines must be number if ($dd !~ /\d\d$/ || $dd =~ /<u$/ ) { $dd =~ s/..$/00/g; } #insert X for the rest if ( length($dd) < 130 ) { $dd = insert0($dd); } else { $dd = substr($dd, 0, 130); } if ( length($dd) == 130 ) { push (@normalDb, $dd); } else { push (@errorDb, $dd); } if ( @normalDb > 0 ) { my $path = "$file\.ok"; open (NORMAL, "+>> $path") or die $!; for (@normalDb) { print (NORMAL "$_\n") }; close (NORMAL); } if (@errorDb > 0 ) { my $path = "$file\.error"; open(ERROR, "+>> $path") or die $!; for (@errorDb) { print (ERROR "$_\n") }; close (ERROR); } } close (IN); } sub skip_zero { my ($a) = @_; my $j = 0; foreach my $i (split //, $a) { $j++; return $j if ($i > 0); } } sub insert0 { my ($tmp) = @_; my @data; push(@data, $tmp); for (my $i = length($tmp); $i < 130; $i++) { push(@data, "X"); } return sprintf ("%s", join('', @data)); } die;
As your info, the the file size is about 507156496 bytes....

Replies are listed 'Best First'.
Re: Re: Script not working on large file size
by BrowserUk (Patriarch) on Apr 22, 2004 at 12:25 UTC
    ...its working but not on larger files ?

    Nothing stands out as particulalry wrong with your code from a quick browse--except maybe that it is probably quite slow. However, you don't say in what way it is not working on large files? Does it produce error messages? Give the wrong output? Never finish?

    Are you simply not waiting long enough for it to complete?

    Without some hints as to how it's failing, working out why is hard.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
      Thanks,
      I spend about 2/3 hour to monitor the script and its done whitout any error message. But, some of the data are not changes as per requirements. As example, the 2 last character of lines must be integer, if not then changes to default values ("00")
      #from my script ... if ($dd !~ /\d\d$/ || $dd =~ /<u$/ ) { $dd =~ s/..$/00/g; } ...
      But, the script skips on that case and once I test again on small files, it done..

        Sorry, but I thought you were serious.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail