I wrote a piece of code to help you understand how Tie::File works internally:

test.pl:

use Tie::File;
use Data::Dumper;

my $self = tie @array, "Tie::File", "test.pl";
print Dumper($self);
print $array[2];
print Dumper($self);
[download]

If you run it, the result could be (the sample data is from win 98):

$VAR1 = bless( {
                 'autochomp' => 1,
                 'mode' => 258,
                 'deferred_s' => 0,
                 'autodefer_threshhold' => 3,
                 'sawlastrec' => undef,
                 'defer' => 0,
                 'dw_size' => 2097152,
                 'offsets' => [
                                0
                              ],
                 'deferred_max' => -1,
                 'autodeferring' => 0,
                 'recsep' => '

',
                 'rdonly' => '',
                 'memory' => 2097152,
                 'filename' => 'test.pl',
                 'fh' => \*Tie::File::FH,
                 'ad_history' => [],
                 'autodefer' => 1,
                 'deferred' => {},
                 'autodefer_filelen_threshhold' => 65536,
                 'recseplen' => 2,
                 'cache' => bless( [
                                     bless( [
                                              [
                                                0,
                                                $VAR1->{'cache'},
                                                0
                                              ]
                                            ], 'Tie::File::Heap' ),
                                     {},
                                     2097152,
                                     0
                                   ], 'Tie::File::Cache' )
               }, 'Tie::File' );
$VAR1 = bless( {
                 'autochomp' => 1,
                 'mode' => 258,
                 'deferred_s' => 0,
                 'autodefer_threshhold' => 3,
                 'sawlastrec' => undef,
                 'defer' => 0,
                 'dw_size' => 2097152,
                 'offsets' => [
                                0,
                                '16',
                                '35'
                              ],
                 'deferred_max' => -1,
                 'autodeferring' => 0,
                 'recsep' => '

',
                 'rdonly' => '',
                 'memory' => 2097152,
                 'filename' => 'test.pl',
                 'fh' => \*Tie::File::FH,
                 'ad_history' => [],
                 'autodefer' => 1,
                 'deferred' => {},
                 'autodefer_filelen_threshhold' => 65536,
                 'recseplen' => 2,
                 'cache' => bless( [
                                     bless( [
                                              [
                                                1,
                                                $VAR1->{'cache'},
                                                1
                                              ],
                                              [
                                                0,
                                                2,
                                                '

'
                                              ]
                                            ], 'Tie::File::Heap' ),
                                     {
                                       '2' => 1
                                     },
                                     2097152,
                                     2
                                   ], 'Tie::File::Cache' )
               }, 'Tie::File' );
[download]

The size of the read catch and the deferred write buffer might be big, but their upper limit is under control, and you even can set the size by setting the memory option. This is not our major concern.

Now let's look at that hash element called 'offsets'. That's where Tie::File stores, by default, the offsets of file lines, if you don't modify the value of recsep.

Actually the author of this package used memory quite carefully. His code does not populate the offsets array with the offsets of all lines, instead it only populates the offsets of lines up to the last line you accessed.

In our example, at the beginning, the offsets array only contains one element, after we accessed the third line, it only contains three elements.

But still this offsets array is growing, especially in your case, as you are using it as a queue, which means the last line you accessed is the last line of the file, so the offsets of all lines are stored in memory.