I think you're misinterpreting the output from your program.

Opening 1 of 4 Thread 1 starts processing file 1
Opening 2 of 4 thread 2 starts processing file 2
Closing 1 of 4 Thread 1 finished processing file 1 and ends
Opening 3 of 4 Thread 3 starts processing file 3;             thread 2
+ is still processing file 2
Closing 3 of 4 thread 3 finishes processing file 3 and ends;  thread 2
+ is still processing file 2
Opening 4 of 4 Thread 4 starts processing file 4;             thread 2
+ is still processing file 2
Closing 2 of 4 Thread 2 finishes processing file 2 and ends.
Closing 4 of 4 Thread 4 finishes processing file 4 and ends.
[download]

That all looks pretty much as might be expected to me. What makes you think something is wrong?

The only problem I see is that you have no way to exit the loop when all files have been processed.