The most efficient way I know of is to use a ramfile and seek (Note: the space allocated is uninitialised).

The nice thing about this is that if you need to extend it, you can, and any existing contents will be preserved even if Perl has to re-allocate a larger chunk of ram.

#! perl -slw
use strict;

my $bigstring;
open my $temp, '>', \$bigstring or die $!;
print $temp 'My oh so unique signature';
seek $temp, 2**19, 0;
print $temp chr(0);
close $temp;

print length $bigstring;

open $temp, '+<', \$bigstring or die $!;
seek $temp, 2**20, 0;
print $temp chr(0);
close $temp;

print length $bigstring;
print substr $bigstring, 0, 25;


__END__
c:\test>junk6
524290
1048578
My oh so unique signature
[download]

But then I've never found a way to truncate that (so that I can append to it), without risking having the space returned to the pool. So, I've used substr as an lvalue with a pointer to manage the space myself:

my $p = 0;
substr( $bigstring, $p, 4 ) = pack 'J', ..
$p += 4;
[download]

Two caveats are: 1) truncate doesn't seem to work on ramfiles; 2) if you over allocate, the unused space remains as a part of the string until you do something about it. Eg.

substr( $bigstring, $p+1 ) = '';
[download]

The somewhat sucky way is:

my $string= ' ' x 2**19;
$string= '';
[download]

It'd be cool if the following worked but it doesn't:

length( $string )= 2**19; # Doesn't work
[download]

Or you can get creative:

open NUL, "<", File::Spec->devnull();
sysread( NUL, $string, 2**19-length($string), length($string) );
close NUL;
[download]

Testing in my environment shows that you can even get away with:

sysread( STDOUT, $string, 2**19 );
[download]

which leaves $string's current value unchanged while extending the storage allocated to it. Of course, there is a risk of running into a system where STDOUT is open for both read and write access. I wonder if there is another Perl built-in that reads into a buffer that can be used more conveniently.

Update: Testing also shows the following works well:

sub grow
{
    sysread( DATA, $_[0], $_[1] );
}
__DATA__
[download]

my $string= ' ' x 2**19;
$string= '';
[download]

I dimly remember having seen this being suggested somewhere else... However it doesn't actually seem to result in speeding up things (if that's the idea behind avoiding reallocations). On my machine and version of Perl it's in fact marginally slower.

use Benchmark "cmpthese";

sub pre {
    my $str = " " x 2**19;
    $str = "";
    $str .= "aaaa" for 1..2**17;
}

sub std {
    my $str = "";
    $str .= "aaaa" for 1..2**17;
}

cmpthese( -1, {
    'pre' => 'pre()',
    'std' => 'std()'
});
[download]

Results:

      Rate pre std
pre 43.6/s  -- -7%
std 46.7/s  7%  --
[download]

Yes, what is somewhat sucky about it is that it allocates the space twice, initializes one copy then copies it (then frees one); though I haven't checked this assumption of mine. Which means iterating over the requested size twice. Pre-sizing avoids reallocating. Reallocating is done exponentially (doubling the size each time) and so averages out to copying the maximum size about twice (unless you get lucky and reallocate in-place, rather unlikely in my experience). So it isn't surprising that the performance difference is minimal between those two choices.

But note that pre-allocating will likely reduce a bit of memory fragmentation by avoiding leaving those power-of-two-sized buffers of now-free space in its wake.

- tye        

For new SVs, macro NEWSV and function newSV.
For existing SVs, SvGROW and function sv_grow.
See perlapi.

Update: I can't find an existing interface to them. You could use Inline::C or write a simple XS module.

In an effort to answer your question (which I haven't) I learned that:

    keys(%users) = 1000;        # allocate 1024 buckets
[download]

Will preallocate for hashes. I know - useless for you.

Just thought I'd share anyway. :)

Kurt

my $str = "";
vec($str, 2**19-1, 8)=0;
$str = "";
[download]