I think the mistake in the code is that you're reading the input file multiple times without opening and closing it, but beyond that, the program itself is inefficient because it's attempting to read the data many times.

I'd rewrite the program to read the data once into a hash and then write many times, something like this:

#! /usr/bin/perl use warnings; use strict; my %hash; while(<DATA>) { chomp; my @col = split " ", $_; next unless exists $col[3]; next unless $col[3] =~ /^S\d{5}GM\d{3}$/; my $key = substr($col[3],1,5); push @{$hash{$key}} , [ @col ]; } for ( sort keys %hash ) { my $i = $_; $i =~ s/^0+//g; my $file = "output_$i.txt"; # open FILE, ">", "$file" or # die("Cannot open file $file\n"); print "FILE: $file\n"; for my $col ( @{$hash{$_}} ) { print join (" ", @$col), "\n"; # print FILE join (" ", @$col), "\n"; } # close (FILE); } __DATA__ 11880 13417 - S00010GM001 sml_056 sp|YV02233 desc 13804 14685 - S00010GM002 sml_045 sp|YV02643 desc 15525 18026 - S00001GM001 sml_032 sp|V023334 desc 32763 34239 + S00002GM001 sml_028 sp|YV02376 desc 67929 68933 - S00003GM001 sml_025 sp|YV02346 desc 90562 91368 + S00012GM001 sml_025 sp|YV02376 desc 10209 10433 - S00012GM002 sml_046 sp|YV02355 desc 12522 12576 + S00013GM001 sml_027 sp|0235777 desc 13247 13349 - S00013GM002 sml_088 sp|YV02375 desc
Please note the commented out open, close, and print statements have not been tested.

The results are:

C:\Perl>perl onfour.pl FILE: output_1.txt 15525 18026 - S00001GM001 sml_032 sp|V023334 desc FILE: output_2.txt 32763 34239 + S00002GM001 sml_028 sp|YV02376 desc FILE: output_3.txt 67929 68933 - S00003GM001 sml_025 sp|YV02346 desc FILE: output_10.txt 11880 13417 - S00010GM001 sml_056 sp|YV02233 desc 13804 14685 - S00010GM002 sml_045 sp|YV02643 desc FILE: output_12.txt 90562 91368 + S00012GM001 sml_025 sp|YV02376 desc 10209 10433 - S00012GM002 sml_046 sp|YV02355 desc FILE: output_13.txt 12522 12576 + S00013GM001 sml_027 sp|0235777 desc 13247 13349 - S00013GM002 sml_088 sp|YV02375 desc

In reply to Re: segregating data from one file to many files by dwm042
in thread segregating data from one file to many files by patric

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.