ForeverLearning has asked for the wisdom of the Perl Monks concerning the following question:

I have an odd thing happening in my script where AnyEvent::HTTP is not releasing variables and just adds data onto the end of a file read.

My main code body calls anyevent

$condvar = AnyEvent->condvar; for (1..$multiLimit) { doProcess(); } $condvar->recv;

doProcess then does:

$condvar->begin; say "Active ${aval}, Offset ${oval}"; ### This was originally another sub, but I brought it inline f +or debug my $content; my $fh = FileHandle->new; if ($debug) { say "Reading ${localfile} offset ${offset} for l +ength ${contentLength}";} $fh->open("< $localfile"); binmode($fh); read($fh,$content,$contentLength,$offset); if ($debug) { say "readData read ".byteSize($content);} $fh->close; ### http_request "PUT" => $url, headers=>$headerVals, body=>$content, sub { my ($body, $hdr) = @_; say "received, Size: ", length $body; say $body; $activeCount--; $condvar->end; doProcess(); }; return 1

And this is the odd behaviour I'm seeing

Active:1, Offset:0 Reading test offset 0 for length 4194303 readData read 4194303 Active:2, Offset:4194304 Reading test offset 4194304 for length 4194303 readData read 8388607 Active:3, Offset:8388608 Reading test offset 8388608 for length 4194303 readData read 12582911

see how readData is being appended and not reset ? Weird huh ! (or, more likely, am I just doing something stoopid ? ;-)

Replies are listed 'Best First'.
Re: AnyEvent::HTTP Not releasing variables
by Anonymous Monk on Mar 21, 2016 at 00:30 UTC
    see how readData is being appended and not reset

    Isn't that what you're telling read to do by giving it an offset?

    Also, your code will not compile so others cannot easily run and test it to reproduce the problem - please see How do I post a question effectively? and http://sscce.org/

      Here is some test code !
      #!/usr/bin/perl use 5.014; use strict; use warnings; use autodie; use AnyEvent; use AnyEvent::HTTP; use FileHandle; use File::stat; my $multiChunkSize=4194304; # Maximum size of a single block (in bytes +) my $multiLimit=6; # Maximum number of parallel HTTP requests my ($wholeChunks,$chunkRemainder,$runID,$condvar,@offsets,@blocklist); my $activeCount=0; my $debug=1; my $localfile="test"; # Byte size sub byteSize { use bytes; my ($inval)=@_; return length($inval); } # Quotient remainder calculator sub qrem { use integer; my( $dividend, $divisor ) = @_; my $quotient = $dividend / $divisor; my $remainder = $dividend % $divisor; my @result = ( $quotient, $remainder ); return @result; } # Get file size sub fileSz { my($file) = @_; my $stat = stat($file); return $stat->size; } # Get data sub readData { my ($file,$length,$offset)=@_; my $fh = FileHandle->new; my ($data); if ($debug) { say "Reading ${file} offset ${offset} for length + ${length}";} #open ($fh,"<",$file); $fh->open("< $file"); binmode($fh); read($fh,$data,$length,$offset); if ($debug) { say "readData read ".byteSize($data);} #close($fh); $fh->close; return $data; } # Process sub doProcess { return if $activeCount >= $multiLimit; my $offset = shift @offsets; return if !defined $offset; $activeCount++; if ($debug) { say "Active:${activeCount}, Offset:${offset}";} $condvar->begin; my $contentLength=$multiChunkSize-1; my $content = readData($localfile,$contentLength,$offset); } # Populate offsets sub populateOffsets { my ($count,$offsetSize)=@_; if (!defined $count || !defined $offsetSize) {exit 1;} my $offset=0; my @offsets; for my $i (1..$count) { push @offsets,$offset; $offset = $offset + $offsetSize; } return @offsets; } ######### MAIN # Calculate chunk quantity my @chunks = qrem(fileSz($localfile),$multiChunkSize); $wholeChunks=$chunks[0]; $chunkRemainder=$chunks[1]; @offsets=populateOffsets(${wholeChunks},${multiChunkSize}); $condvar = AnyEvent->condvar; # DO IT for (1..$multiLimit) { doProcess(); } $condvar->recv;

        You are using the four-argument version of read:

        read FILEHANDLE,SCALAR,LENGTH,OFFSET

        An OFFSET may be specified to place the read data at some place in the string other than the beginning. A negative OFFSET specifies placement at that many characters counting backwards from the end of the string. A positive OFFSET greater than the length of SCALAR results in the string being padded to the required size with "\0" bytes before the result of the read is appended.

        This would to me explain the behaviour you're seeing, as Anonymous Monk already diagnosed as well.

      "Isn't that what you're telling read to do by giving it an offset?"

      (a) The read routine is in a loop so the variables should be reset, no ?

      (b) Isn't the point of an offset to dictate the start location where the read should start ? Nothing to do with appending ???