Hello hello,

I have a perl script that I need to run a large number of times on different input data. I use Grid Cluster to send all the processes to our compute cluster.

All the processes are supposed to write to the same output files, of which there are four. However, in my first attempts the data was jumbled. I would get things like:

line1 data data data
line2 data data dataline3 data data data
line4 data data data

line5 data data data
line6 dataline7 data data data data

line8 data data data data
[download]

I took this to mean that sometimes one process didn't quite finish printing when another process tried writing to the file, so I figured I was going to have to lock the output file before writing to it.

I read several tutorials, among which File locking, but it still doesn't work. Below are simplified versions of my scripts. Can someone help me?

use strict;
use warnings; 
use GRID::Cluster;

my $script = "script.plx";

my @processes = (...);
my @machines = (...);
my %max_num_processes = (...);

my $cluster = GRID::Cluster->new(
      host_names => \@machines, max_num_np => \%max_num_processes,
);

$cluster->qx(@processes);
[download]

use strict;
use warnings;
use Fcntl qw(:flock);
my @letters = qw(a b c d);

foreach my $letter (@letters) {

    my $output_file = "output_" . $letter . ".txt";

    # This didn't solve my problem
    # I also tried +< instead of >>, but nothing happened at all
    # when I did that
#     open my $filehandle, ">>$output_file";
#     flock($filehandle, LOCK_EX);

    # So instead I tried the version in the comments on the 
    # tutorial page although I don't really understand it :-(
    open my $semaphore, ">$output_file.lock";
    flock($semaphore, LOCK_EX);
    open my $filehandle, ">>$output_file";

    print $filehandle "line$number data data data\t";

    close $filehandle;
    close $semaphore;
}
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.