Hello, Monks!
I am a newbie in Perl threading model, so I've got some problem while using threads and shared access to data among them.
My aim was to make a queue of data to work with, which is being read from a file in blocks. Block size is equal to threads count, so, first we are filling up an array with new block of data. Then threads shifts the data lines from an array, work with it, and check if an array with data queue is empty. If if is so, it initializes filling up this array with fresh data. Also I dont want to use ready-to-use classes like Thread::Queue, I need to do it only using threads and threads::shared. Here is my code(I used binary access to be able to operate very large files):
use threads;
use threads::shared;
my @block : shared; # block of data
my $pos : shared; # offset in data-file here
my $threadscnt=30; # count of threads
open INFH, '<', $path.'list.txt' or die("Cannot open list.txt : $!\n")
+; # file with input data
$listsize = -s INFH;
binmode(INFH);
$pos=0;
for(0..abs($threadscnt-1)){ $trarr[$_]=threads->create(\&smeacts, $_);
+ }
for(@trarr){$_->join; }
sub smeacts($)
{ my $num = shift(@_);
until(($pos>=$listsize)and((scalar @block)==0)){ # until we reached th
+e end of datafile
if (@block){
{ lock(@block); $sline = shift(@block); } # get line of data and lock
+block from other threads to prevent errors
print "from $num: $sline($pos,$listsize)\n"; sleep(rand(5)); } # emula
+te some work with data
if ((scalar @block)==0){ async{ getnewblock(); } } # get new block if
+necessary
}
}
print 'end.';
sub getnewsline()
{ my $oneline=''; my $tmp='';
while((read(INFH, $tmp, 1))and(not(($tmp eq "\n")or($tmp eq "\r")))){
+# binary reading line of data
$oneline .= $tmp;
$pos++; }
$pos++ if ($tmp eq "\n"); # for unix new-line format
if ($tmp eq "\r"){ seek(INFH, 1, 1); $pos += 2; } # for win new-line f
+ormat
return $oneline; }
sub getnewblock()
{ my $i=0; my $tmp = getnewsline();
while(($i<=abs($threadscnt-1))and($tmp)){
push(@block, $tmp);
$i++;
$tmp = getnewsline(); }
return; }
But when I try to use it, sometimes I get an infinite output, or strange output in console, like:
from 1: foo(45,999)
from 2: bar(48,999)
from 3: foo(45,999)
...
from 8: a(1050,999)
Why it happens? Using async and lock didnt fixed the problem, as I saw. I think that the problem is in shared access, or access to line in one time... But sometimes this program gives me the perfect output, just as I expected, and sometimes some 'madness'. For info, I am using ActivePerl 5.10.1. Remember, that sometimes it gives the CORRECT output, but if you make more tests - you will see what I said! Now, what can you correct or suggest to fix this unstability?
Sorry for my mistakes in English, it is not my native language.
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.