Delaying NFS writes

ZlR has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it. I tried different things and ended with the following code :

$| = 1 ; 
my $fh  ;
    for my $master (1..100) {
        for my $fast (1..50000) {
                unless ( print $fh "PRINT OK $master $fast\n" ) {
                        my $now = localtime ;
                        print "Failed $it $master $fast at ", $now,  "
+\n" ;
                        close $fh ;
                        sleep 1 ;
                        my $ok = 0 ;
                        while ($ok == 0)  {
                                if (open $fh, ">>", "/mnt/fs_1/ttt.$it
+.txt") {
                                        print $fh "PRINT OK $master $f
+ast\n" ;
                                        $ok = 1 ;
                                } else {
                                        my $now = localtime ;
                                        print "Failed delayed write fo
+r $it $master $fast ", $now,  "\n" ;
                                        sleep 1 ;
                                }
                        }
                }
   }}
[download]

I also tried without closing the filehandle, just repeating the if (print "...) until it's ok, but in both versions i still end up losing some data ! That is, it seems there's a delay between the moment the nfs folder becomes unavailable and the moment the OS (and my script) become aware of it.

Is there a specific way to handle writes to nfs folders ?

Thanks for your advice !

Comment on Delaying NFS writes Select or Download Code

Replies are listed 'Best First'.
Re: Delaying NFS writes by Generoso (Prior) on Dec 10, 2010 at 15:02 UTC
Just an idea, why don’t you write to a local file in temp and at the end copy it over?	[reply]
Re^2: Delaying NFS writes by ZlR (Chaplain) on Dec 10, 2010 at 15:57 UTC
That would report the problem on the way "copy" deals with stale nfs ! I'm actually trying to understand how i can cope with this myself ;)	[reply]
Re: Delaying NFS writes by JavaFan (Canon) on Dec 10, 2010 at 16:03 UTC
Have you tried flushing the filehandle after each write?	[reply]
Re^2: Delaying NFS writes by ZlR (Chaplain) on Dec 10, 2010 at 16:26 UTC
I don't know how to do that. I thought $\| = 1 would force flushing.	[reply]
Re: Delaying NFS writes by afoken (Chancellor) on Dec 11, 2010 at 12:22 UTC
I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it. Why does the NFS server go down? How can you avoid that? Is it a planned downtime? If so, can you arrange to be notified automatically before and after the downtime? I think you should not try to work around such problems in your application. Think about different solutions. rsync could be useful: You write to a local file, and tell rsync to sync it to the storage server. If the connection breaks, you simply restart rsync until the entire file is transfered. Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm. Consider using a simple ssh connection (hoping that that won't go down as often as NFS): `#!/usr/bin/perl use strict; use warnings; open my $out,'\| ssh server cat ">>" /tmp/log.txt' or die "open failed: + $!"; print $out "hello world\n" for 1..10; close $out;` [download] Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l]
Re^2: Delaying NFS writes by ZlR (Chaplain) on Dec 11, 2010 at 16:22 UTC
Well, thanks for these answsers, but the real setup is one of an NFS service failing over to another let's say cluster node, where it will be started over the same IP... It's expected behaviour that it happens at the worst possilbe time ;) Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm. I'll have to check what options I have, but i'd expected default mount options to behave like that, ie stall but not lose data :/	[reply]