in reply to 2GB limit to vecs

Until a better version of vec is available, you might try something like this: 

sub myvec(\$$$) :lvalue  {
    use constant TWO_GB => 2**31;
    my( $ref, $offset, $bits ) = @_;
    if( $offset > TWO_GB - 1 ) {
        $offset -= TWO_GB;
        $ref = \substr $$ref, ( TWO_GB * $bits ) / 8;
    }
    CORE::vec( $$ref, $offset, $bits );
}

[download]

Which should be reasonably efficient as it avoids copying the huge string. If you wanted to get fancy in anticipation of the fixed version, you could stick it in a module and export it as CORE::GLOBAL::vec.

The above is untested at the transition limit as I don't have enough memory to create strings that big. You might want to look closely at that TWO_GB - 1...

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^2: 2GB limit to vecs
by bop (Acolyte) on Jun 24, 2008 at 01:34 UTC
    Dear Monks,
    thank you for all your help! I used the same workaround (working with different chunks), albeit nowhere near as elegant as that - I will use this from now on. (I am on a 64 bit machine - I just had thought that when using a 64 bit version of perl everything would be running on 64 bits.)
    thank you,
    bop
[reply]
      I am on a 64 bit machine

      If there is any chance that your strings will get bigger than 4GB, I think you just need to change the if for a while and the rest would take care of itself: 

      sub myvec(\$$$) :lvalue  {
    use constant TWO_GB => 2**31;
    my( $ref, $offset, $bits ) = @_;
    while( $offset > TWO_GB - 1 ) {
        $offset -= TWO_GB;
        $ref = \substr $$ref, ( TWO_GB * $bits ) / 8;
    }
    CORE::vec( $$ref, $offset, $bits );
}

      [download]

      but again that's untested, so check it and convince yourself.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
        thank you again, will do!
[reply]

In Section Seekers of Perl Wisdom