in reply to Re: Private Utilities
in thread Private Utilities

From what I understand, these files can be large. I also see that you're essentially reading the whole file into memory. The combination of those two could be bad. Before I propose a better solution, is it okay to put adjacent lines in different files? That is, let's say that I have a file that contains the following: 
AAA
BBB
CCC
DDD
EEE
FFF

[download]
Is it okay to split into two files like so: 
AAA
CCC
EEE
---
BBB
DDD
FFF

[download]
If so, I may have a really slick solution floating in my head...

thor

The only easy day was yesterday

Replies are listed 'Best First'.
Re^3: Private Utilities
by l3v3l (Monk) on Dec 01, 2005 at 22:36 UTC
    no problem!! round robin is fine - here is a way I have done it in a liner without having to read the entire file into mem (just an example) : 
    # Choose first N FASTA records
perl -ne 'BEGIN {$/=">";$o=0}{chomp;$o<N?(/^$/?next:print">$_"):last;$
+o++}' EXAMPLE.fa

    [download]
    example usage: 
     perl -ne 'BEGIN{$/=">";$o=0}{chomp;$o<22?(/^$/?next:print">$_"):last;
+$o++}' b4x_est100.fa # print the first 22 records

    [download]
[reply]
[d/l]
[select]
      Alright...here's a simple multiplexer: 
      use strict;
use warnings;

use Getopt::Long;

my ($number, $format) = (2, "output");
GetOptions( 
        "number=i" => \$number,
        "format=s" => \$format,
);

my @output_file;
foreach my $num (1..$number) {
        my $file = "$format.$num";
        open( $output_file[$num-1], ">", $file ) or die "Couldn't open
+ '$file' for write: $!\n";
}

my $file_num = 0;
while(<>) {
        print {$output_file[$file_num]} $_;
        $file_num += 1;
        $file_num %= $number;
}

      [download]
      Save it in a file of your choosing, and call it like: 
      perl script.pl --format out --number 7 input1 input2 ...

      [download]

      thor

      The only easy day was yesterday

[reply]
[d/l]
[select]
        Right on! thank you for the code thor. If you are interested or have any ideas on how to improve - here are some other (bioinformatics) liners that I have been working with and have put on the site l3v3l's scratchpad
[reply]

In Section Meditations