BrowserUk,
I found the problem(s). I believe I had 3 things wrong and they were all part of the code I posted!
I did this by "reverse engineering" the problem. After showing you how the test case and module were put together, I decided to take them apart and load the module with "use".
I then decided to use '%RCache' hash to point to the data contained in an array. Now I could re-initialize '%RCache' but the array couldn't be re-initialized. So I thought it's something wrong with the data I'm producing in the index file. And that enabled me to see the actual problem.
In the code, I copied the temp buffer to the reference passed from the caller:
substr($$buffer,0,$reclen+4,$tmpbuf); #*#
The first thing I did wrong, was that I initialized '$buffer' only once. So when the above was executed and '$$buffer' was larger than '$tmpbuf', it would grow. The 'sub GetSubBuffer' gets called 24,000,000 in this test case, and the data in '%RCache' just kept getting bigger. When I replaced it with:
$$buffer = substr($tmpbuf,0,$reclen+4);
the problem with re-initialize just went away and the test throughput improved from 15m37.516s to 11m16.684s (approximately 30+%).
I still don't know why I couldn't re-initialize '%RCache', but it works now. Maybe the failing code created a self-reference. You would know better than I. The third problem was scoping in nature, and had to be fixed just to get the test case to compile after splitting the code.
If you notice the '#*#' after the code, that should have told me to look at that line first, since I add that to code that I think should have additional testing. It worked for the program, since '$reclen' is the size of the actual data, so it's always using the beginning (front) of '$buffer' and when it goes to disk, I'm using 'syswrite' with a count, so the problem was only for the cached data that was growing.
Thank you for your help.
"Well done is better than well said." - Benjamin Franklin
|