in reply to Multithreaded script keeps files locked when reading output from multiple processes on Windows

If you run this modified version of your test code, you'll see that the main thread can delete file 1 whilst thread 2 is still running; but only after the process thread 2 spawns, terminates.

<Reveal this spoiler or all in this thread>

The output:

C:\test\junk>..\junk41
26: Cant remove file1 because Permission denied/The process cannot acc
+ess the file because it is being used by another process
29: Cant remove file2 because Permission denied/The process cannot acc
+ess the file because it is being used by another process
[1] eof pipe
[1] pipe closed
[1] thread ending
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
32: Still cant remove file1 because Permission denied/The process cann
+ot access the file because it is being used by another process
[2] eof pipe
[2] pipe closed
[2] thread ending
36: This isnt printed No such file or directory/The system cannot find
+ the file specified

[download]
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: Multithreaded script keeps files locked when reading output from multiple processes on Windows
  • Select or Download Code

Replies are listed 'Best First'.
Re^2: Multithreaded script keeps files locked when reading output from multiple processes on Windows
by rmahin (Scribe) on Jan 13, 2014 at 23:25 UTC

    Thanks for your explanation. Was able to work around this by opening files with the FILE_SHARE_DELETE permission as you using Win32API::File and it does seem to do the trick. Replaced the thread code with: 

        my $tid = threads->tid;
    my $fn = "file" . $tid;

    my $winFileHandle = createFile( $fn, "w", "d" ) or die "$!/$^E";
    OsFHandleOpen(*OUTPUT, $winFileHandle, "w") or die "$!/$^E";
    
    my $pid = open(my $pipe, "-|", "perl test.pl");
    
    print OUTPUT $_ while(<$pipe>);

    close(OUTPUT);
    close($pipe);
    CloseHandle($winFileHandle);

    [download]

    I do however have another question, well more of a clarification. So in my actual usage, I am using a threadpool and a Thread::Queue and not starting/joining threads after one another. But, I'm just making sure I understand correctly, this does not matter because if I have for example:

    1. thread 1 open file1
    2. thread 2 start process A while file1 is still open
    3. thread 1 closes file1
    process A will still have a filehandle for file1 even though thread2 did absolutely nothing with the file because the spawned process inherits ALL of the the file handles currently in use by the main perl process at the time of opening the pipe? Sorry if this comes out more as just paraphrasing what you said, just wanting to make sure I understand fully :)

    Thanks for the help!

[reply]
[d/l]
      process A will still have a filehandle for file1 even though thread2 did absolutely nothing with the file because the spawned process inherits ALL of the the file handles currently in use by the main perl process at the time of opening the pipe?

      That is correct, but overstates the conditions.

      Forget which thread does what. Indeed, forget threads entirely. (Even forget the file bit.)

      When one process spawns another process, the child process will inherit every open handle that has the bInheritHandle flag set in the Security Attributes structure supplied when that handle was created in the parent.

      At the Win32 API level, this can be controlled on a per (file) handle basis; but as POSIX doesn't have that concept, Perl doesn't provide access to it and thus uses a generic security descriptor that means everything gets inherited.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        That was all very illuminating! Thanks a bunch for your time and explanation. :)

[reply]

      Ignore this...created on accident

[reply]

In Section Seekers of Perl Wisdom