Given that it is not repeatable on any other OS or even in a test case the two obvious fixes are to kludge it by stashing the value of tell() or to drop the use of IO::File. I have never been able to understand why people use a module that seems to me to be a totally useless use of OO to provide syntactic sugar. For the interested the offset drift is unusual in that it varies wildly and is nothing like common buffer sizes +/- a line.

11530000    11477968    = -52032
23064612    22977360    = -87252
34602683    34593093    = -9590
46134989    46040223    = -94766
57668448    57664020    = -4428
69205366    69145548    = -59818
80739978    80667412    = -72566
92271131    92205201    = -65930
103808049    103807699    = -350
115343814    115293110    = -50704
126879579    126807910    = -71669
[download]

I think graffs idea that you may be simultaneously be mixing calls that use/bypass stdio could be at the heart of the problem.

Thanks for posting the diffs. I calculated them, but just ended up rubbing my eyes, cursing, and getting another cup of coffee instead of doing anything useful with the info.

RE: using a wrapper module, this code was written and in production with 5.00503 for a while, and back then I was using FileHandle to stash FH's as object members. It's been low in my refactoring list, up until this particular deployment.

So, the only thing happening between the "pausing" tell call and the "continuing" tell call is a few fork calls? I remember folks telling me recently that forked processes will share memory with the parent, but what you're seeing still should not happen.

If you conclude that forking is somehow triggering this behavior (and you can't convince the bankers to switch to Linux ;), then IMHO it would not be viewed as "klugey" to provide commentary in your code that mentions the apparent instability of IO::File offset pointers when used in combination with forking, call "tell" before the forking is done and even close the file; then reopen and seek after forking.

I can understand why you don't post a sample of your code in this case, but something to consider is to create a test-case script that you think might isolate the problem -- remove all "irrelevant" detail, and limit it to open file; while (whatever) {read 10K records; tell; fork...; tell }

If the most minimal script does not reproduce the problem, start adding in details from the target app. At some point, you'll find the thing in your code that you thought wasn't there or wasn't relevant, etc. (At least, one can hope...)

(update: the only other issue I could imagine bein relevant is to make sure you aren't doing anything that involves improper mixing of i/o styles -- e.g. if you're using getline and tell, you should not also be using any i/o function that starts with "sys". Of course, if you were, then I'd expect it to break under Linux as well.)

So, the only thing happening between the "pausing" tell call and the "continuing" tell call is a few fork calls? I remember folks telling me recently that forked processes will share memory with the parent, but what you're seeing still should not happen.

Actually, there's a ton of code between the pause and resume calls, but nothing that should be relevant to this filehandle. Trivial test scripts run fine, and this doesn't happen on identical or comparable hardware in other shops. The only changing variable in this scenario seems to be the client-specific configuration. That would influence things like the number of open file descriptors, number of open database handles, etc.

I do use sysread/write in the same code suite for some unrelated socket interaction, but no other process or piece of code has any awareness of this object's filehandle. The object itself does gets shared across a bunch processes during the forks. Still, assuming that the forked process get CoW'd shared memory and each child gets a dupe of the open file table for that FD, and nobody moves the offset pointer, this shouldn't happening. AArGh.

Anyway, I'll check for mixing I/O access methods just for sanity's sake. That's definitely a Good Thing to know about even if it doesn't solve this particular issue. Thanks for your time!

Cheers,
Ezra

Dave.

Hi Dave,

Why would the posix _exit() be unsatisfactory?

thanks :-)

Jason L. Froebe

No one has seen what you have seen, and until that happens, we're all going to think that you're nuts. - Jack O'Neil, Stargate SG-1

Why would the posix _exit() be unsatisfactory?

Because if for any reason whatsoever, a child process somehow accidently exits in any other way, the parent is screwed. So it's a bit of a fragile mechanism.

Dave.

Why would the posix _exit() be unsatisfactory?

One word. Kludge.

Fixing the underlying OS bug would be satisfactory, everything else is a workaround and therefore not very satisfactory. That said something that works is by definition beter than something that does not.

cheers

tachyon

Hey Dave, just out of curiousity, do you have any documentation on this Solaris 'feature'? Also, do you know offhand if calling POSIX::_exit() will honor DESTROY hints like DBH's InactiveDestroy? I'll test all this later today when I get to work, but any heads-up on specifics would be great. Thanks...

Ezra

do you have any documentation on this Solaris 'feature'?

Well, I know about it because of a Perl bug report from a few weeks ago. I'm not sure whether we got as far as resolving whether it's Perl's or Solaris's fault, and I haven't had time yet to go back and look at it further. Solaris is trying to backout of a stdio buffered read before exiting.

do you know offhand if calling POSIX::_exit() will honor DESTROY

_exit() does an immediate process exit without doing any cleanup of any kind, either at the Perl or C level.

Dave.

Very interesting, and near what I was expecting. I need further testing with this. Thanks...