process($data_ref,\@set_up,$template,*OUT);
# Process
sub process {
    my @data = @{ shift()};
    my @set_up = @{ shift()};
    my $template = shift; #For unpack
    local *FH = shift;

    foreach my $record (@data) {
        --do things--}
}
[download]

The @data array was very big, and the subroutine above actually makes a copy of it. I didn't appreciate this when I first wrote it, but it was slow, and was consuming silly amounts of memory (even by Perl standards). Changing the relevant lines as follows makes it run faster, and it now uses less memory - about half as much.

    my $data_ref = shift;     #Big array

    foreach my $record (@{$data_ref}) {
        -- do things more efficiently --
                }
[download]

I am now a happy perl loving bunny, and I can read in my very big files ;-)

--
Anthony Staines

PS - What's happening here, for monks whose Perl skills are closer to my own, is that I passed the array as a reference ($data_ref) to the subroutine. There, partly from force of habit I was turning it back into an array. This, as was obvious with hindsight, doubled the memory requirements of the program. Now @data was big, 10 MB to 50MB or so, and the program was not working well. Accessing the array, through the reference only, fixed the problem - hence the @{$data_ref} bit.