Odd behaviour from print statement

periapt has asked for the wisdom of the Perl Monks concerning the following question:

I have been experiencing some odd behaviour that I've tracked down to a print statement (I think). Essentially, the print statement (I/O buffer?) doesn't output specific lines. Every line after the first line in fact although it always prints the first line. When run against different data sets, the loop will not drop lines consistently from set to set but will consistently drop data from the same data set on multiple runs.

The data is output to file if a filename is specified on the command line and to STDOUT if no filename is given.

The problem seems to occur only when I specify the file name on the command line. It does not occur if when printing to STDOUT. I can also correctly redirect the output from STDOUT into a file.

I've tried ...

setting $| ($AUTOFLUSH) to true.
hard coding file names and all the other variables
checking for Ctrl-Z or other non-printable chars in data
Output to both STDOUT and file. The loop will output all the expected records to STDOUT while, at the same time, only printing the first record to file.
running the program directly on the server (the output file is on a shared drive)
running the program under a different OS on different machines

Additional info & suggestions from various monks ...

The debugger shows everything is OK. Results from debugger are, however, same as from cmd line
The loop will retrieve and process all of the records it is supposed to
$obstr is set to each record and appears correctly formatted after the join
the return value for print is TRUE for all records
use strict; use warnings; use diagnostics; are set
using syswrite with the same results as print
corrected buffer AUTOFLUSH to actually flush the file buffer vice STDOUT

System specs ...

The data is standard ASCII (not utf8) data. A check on the raw data does not turn up any untoward characters.
The database connection is Win32::ODBC
The system is Activestate Perl 5.6.1 build 635 on WinXP and Win 2000

Relevent code

if($db01->FetchRow()){
    open my $fh,">",$cfg->{outfile} if(defined($cfg->{outfile}));
    unless($fh){
        warn "Unable to open output file $cfg->{outfile}\n$!\nUsing ST
+DOUT\n" if(defined($cfg->{outfile}));
        $usestdout = TRUE;
        $fh = \*STDOUT;
    }
    do{
        ++$recnum;
        printf("\b\b\b\b\b\b\b\b\b%-9s",$recnum) if($cfg->{displaycoun
+t});
        @obrec = $db01->Data();
        $obrec[$_] = sprintf("%-*s",$oblen03[$_],($obrec[$_] || '')) f
+oreach (0..$#obrec);
        $obstr = join("\t",@obrec);

# some troubleshooting attempts
#        print STDOUT "$obstr\n" if($obstr =~ /[[:cntrl:]]+/); #checki
+ng for control chars in rec
#        print STDOUT "$obstr\n" if($obstr =~ /[[:^print:]]+/); #check
+ing for unprintable chars in rec
#        print STDOUT "$obstr\n";

         print $fh "$obstr\n";
    }while($db01->FetchRow());
    close($fh) unless($usestdout);
}
[download]

At this point, I'm stumped. Any suggestions will be most appreciated.

Update01: added some additional info and the results of some monk suggestions.

Update02: I solved it. Read operator error :o( Under certain circumstances, the program makes a a second pass through the loop. The open statement then clobbers the file and only inputs a portion of the remaining records. The open mode should be append ">>" not create ">". Thanks for all the great suggestions though. I learned a few things.

PJ
use strict; use warnings; use diagnostics;

Comment on Odd behaviour from print statement Download Code

Replies are listed 'Best First'.
Re: Odd behaviour from print statement by JediWizard (Deacon) on Jan 18, 2005 at 21:07 UTC
You could try checking the return value of `print`, as it will return true on success and false on failure. May the Force be with you	[reply] [d/l]
Re: Odd behaviour from print statement by ikegami (Patriarch) on Jan 18, 2005 at 23:38 UTC
It really does sound like you're output is buffered. Did you `select` the filehandle before changing `$\|`? If not, you're setting autoflushing for the currently selected handle (presumably STDOUT). `open(FILE, ... { my $old_selected = select(*FILE); $\| = 1; select($old_selected); } print FILE ...` [download]	[reply] [d/l] [select]
Re^2: Odd behaviour from print statement by periapt (Hermit) on Jan 19, 2005 at 13:53 UTC
Good idea :o) I was setting AUTOFLUSH on STDOUT vice the file handle. However, after setting it on the file, no luck. I was optimistic for a few minutes though. PJ use strict; use warnings; use diagnostics;	[reply]
Re: Odd behaviour from print statement by holli (Abbot) on Jan 18, 2005 at 21:12 UTC
Where does $fh come from? Is it an instance of FileHandle.pm or from a statement like `open $fh, ">file";`? What if you try a "normal" handle like OUT as in `open OUT, ">file";`? holli, regexed monk	[reply] [d/l] [select]
Re^2: Odd behaviour from print statement by Anonymous Monk on Jan 19, 2005 at 14:08 UTC
References to filehandles have worked since, oh, eons. (5.001 or so). Autovivifying references to filehandles has worked since 5.6.0. That's two months short of five years. Not something to be really surprised about.	[reply]
Re: Odd behaviour from print statement by talexb (Chancellor) on Jan 18, 2005 at 20:58 UTC
Looks like a short script. What does it do in the debugger? Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply]
Re^2: Odd behaviour from print statement by periapt (Hermit) on Jan 18, 2005 at 21:11 UTC
It is a pretty short script. The rest of the program is housekeeping, parsing the command line and configuration files, preparing the SQL statement, that sort of thing. The debugger shows everything is OK. When stepping through the loop, it will retrieve and process all the records it is supposed to even though it only prints the first one to file. The variable $obtstr contains each record and is correctly formatted after the join without unexpected characters. PJ use strict; use warnings; use diagnostics;	[reply]
Re^3: Odd behaviour from print statement by talexb (Chancellor) on Jan 18, 2005 at 21:31 UTC
So it runs correctly in the debugger, but not from the command line? Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply]
Re^4: Odd behaviour from print statement by periapt (Hermit) on Jan 19, 2005 at 13:49 UTC