Just storing your number in binary rather than ascii with delimiters would cut your memory requirements to about 1/3 on average. Assuming that your example is representative and your numbers will fit in 16-bits. You mentioned the figures of 50,000 sets and assuming 4 per set as shown.

Instead of approx 1 MB, your down to 390k, ignoring the cost of the arrays which should be constant That's a start.

Projecting the timing of the code below gives around 3 seconds to pack the 50k x 4 x 16 bits and 0.002 secs to unpack 100 lines, which comfortably fits your spec, and my machine is hardly the quickest around.

#! perl -slw
use strict;
use Benchmark::Timer;
my $t= Benchmark::Timer->new();


my (@ary1, @ary2);

#! 1000 lines of 4 random numbers (0-65536) in ascii with commas.
@ary1 = map{ join ',', map{ int rand(2**16) } 1..4 } 1 .. 1000;
print $ary1[0];

$t->start('packing');
#! The same numbers stored as binary strings
@ary2 = map{ pack 'S4', split',', $_ } @ary1;
$t->stop('packing');
print length $ary2[0];

print '1000 in ascii:  ', scalar @ary1, ' : ', length "@ary1";
print '1000 in binary: ', scalar @ary2, ' : ', length "@ary2";

$t->start('unpacking');
for (@ary2) {
    my @line_of_nums = unpack 'S4', $_;
#    print "@line_of_nums";
}
$t->stop('unpacking');

$t->report();
__END__
C:\test>229792
52318,4206,9238,48758
8
1000 in ascii:  1000 : 22310
1000 in binary: 1000 : 8000
1 trial of packing (50ms total)

1 trial of unpacking (20ms total)
[download]