Win98 strangeness

Odud has asked for the wisdom of the Perl Monks concerning the following question:

I've found that under Windows 98 there seems to be a limit on the number of opens from a pipe that can be done. What I had was

use warnings;

$runcount = 0;

while(1) {

    # see if there is an internet connection running

    open(IPCONFIG,"ipconfig|") || fatal("Can't fork ipconfig");
    $runcount++;
    print "Good opens so far $runcount\n";
    while(<IPCONFIG>) {
        if (/IP Address/) {
            ($ipaddr) = /(\d+.\d+.\d+.\d+)/;
            last;
        }
    }
    close(IPCONFIG);
    
    sleep 1;
}

sub fatal {
    my ($rpt) = @_;
    print STDERR "Fatal Error Report\n\n";
    print STDERR "Reason $rpt\n";
    print STDERR "Error $^E\n";
    print STDERR "Good opens $runcount\n";
    die;
}
[download]

This fails after 64 opens (a mystical number! but very small for these inflationary times). It was suggested to me that W98 fakes pipes by writing to a temporary file and then making the temporary file available for input. So I though the way round it was to make the command output available through backticks thusly

    $res = `IPCONFIG`;
    fatal("Call failed") if (!defined $res);    
    $runcount++;
    ($ipaddr) = ($res =~ /(\d+.\d+.\d+.\d+)/);
[download]

but this also fails on the 65th attempt. It looks as though backticks are faked also.

Two questions
1) does anyone know of a way around this and
2) can anyone confirm that W2000 doesn't suffer from this problem (I'm pretty sure that NT 5 doesn't but I'll have to wait until I'm back in work to try it)

Thanks in anticipation of another interesting thread.

Odud

Comment on Win98 strangeness Select or Download Code

Replies are listed 'Best First'.
RE: Win98 strangeness by lhoward (Vicar) on Jun 21, 2000 at 23:04 UTC
All OSes have limits on the number of filehandles that a single process can have open at one time. Sometimes this can be increased by setting a kernel parameter or using a command like ulimit (on UNIX). I don't know about win98 personally, so I can't say for sure. There is a module FileCache that comes with perl. It allows you to have more files open that then OS permits by closing and re-opening files (as needed) in excess of your OSes limit. I'm not sure if this works with win98 or not but it is worth a shot.	[reply]
RE: RE: Win98 strangeness by Odud (Pilgrim) on Jun 21, 2000 at 23:11 UTC
Good point and I will look at the module - but as I'm closing the file and (in the case of backticks) never opening anything I don't hold out much hope. The odd thing about open from the pipe output is that if you hit EOF then it looks like Windows closes the temporary file before the Perl close gets a chance and strange warnings are sometimes produced. It sometimes worries me that the beta of ActiveState 5.6 was only available for W2000 and NT not W9x.	[reply]
(jcwren) Re: Win98 strangeness by jcwren (Prior) on Jun 21, 2000 at 23:16 UTC
I went hunting through the MSDN and couldn't find any express limits on the number of pipes. I too find it odd that such a magic number would occur. According to MSDN, the low level 'C' I/O routines can handle 2048 open files, and the upper level 'stdio' stuff 512. Pipes are implemented in memory, and not as temporary disk files. I don't know what exact conditions will cause a block-on-write. NT may spill to the swap file (W98 will probably just BSODs on you...) What disturbs me is that other files are already open, such as stdin, stdout, and stderr. I wouldn't be surprised that you might hit a process limit (such as W98 supports processes, totally unlike NT), but the number of files already open should only allow you to open 60 or 61 pipes. Which leads me to believe there's something else going on I can't find. Sorry I couldn't locate any more detail. You could try some selective searches at msdn.microsoft.com. Oh, and it might be worth checking the source out, to see if it's some sort of artificial limit of being a Windows port of Perl. --Chris	[reply]
RE: Re: Win98 strangeness by Odud (Pilgrim) on Jun 21, 2000 at 23:29 UTC
I had a quick look through MS Technet and couldn't find anything either. Shendal has tried it on NT with no problem so I suspect it is something funny with 98 rather than a Windows port issue. For all it pretends otherwise its DOS roots show sometimes!	[reply]
RE: Win98 strangeness (a kludgy workaround) by Odud (Pilgrim) on Jun 22, 2000 at 00:15 UTC
Doing `system("ipconfig > test.txt") && fatal("System call failed"); open(IPCONFIG,"test.txt") \|\| fatal("Ordinary open failed");` [download] doesn't seem to suffer from the problem (at the time of writing it has opened without failing more than 512 times) It's ugly but it works!	[reply] [d/l]
Re: Win98 strangeness by Shendal (Hermit) on Jun 21, 2000 at 22:58 UTC
I just tried this under Windows NT 4.0, and it works fine (well, I stopped the script at around 100 successful tries).	[reply]
Win2k Process count by Rydor (Scribe) on Jun 22, 2000 at 04:29 UTC
I have it going under Win2k and it is currently at 1515 processes. I'll add an update when it kills itself @:::::::((==========Rydor====> Update: just broke 3000 Upto 4300. I'll keep it running overnight and post the result tomorrow 6000(btw, this isn't tomorrow yet) Now it is tomorrow, and i'm up to 24800 and going Just broke 30,000. I didn't know it could open so many. 41500. about noon IT'S DONE! 56694. No error message though. just stopped. hmmm	[reply]
Re: Win98 strangeness by cbraga (Pilgrim) on Jun 22, 2000 at 05:07 UTC
Maybe that's thanks to another arcane leftover from dos. Try adding a line `FILES=100` to your `config.sys` and see if it helps. Old DOS had a maximum limit on the number of simultaneuosly open files in order to save memory, and I don't know for sure if win9x inherits that limitation.	[reply]
RE: Re: Win98 strangeness by t0mas (Priest) on Jun 22, 2000 at 13:20 UTC
Sweet? memories of tons of hours spent tuning my MS-DOS system comes to me... FYI, BUFFERS is 30 by default, FCBS is 4, and FILES is 8 in Win98 if nothing else is specified in config.sys /brother t0mas	[reply]
RE: Re: Win98 strangeness by Odud (Pilgrim) on Jun 22, 2000 at 18:09 UTC
Tried setting FILES=100 in config.sys but it still fails at the same point (and yes I did remember to reboot). My kludgy workaround (sending the output to a file from system and then opening the file) is working fine so I think I may have to stay with it until I move to W2000.	[reply]
RE: Win98 strangeness by buzzcutbuddha (Chaplain) on Jun 22, 2000 at 00:48 UTC
On W2K I got 202 pipes before I killed it. Dunno.	[reply]
Re: Win98 strangeness by Anonymous Monk on Jun 22, 2000 at 11:12 UTC
Editing config.sys seems like the Answer. Please report back if it works!!!	[reply]
(jcwren) RE: Re: Win98 strangeness by jcwren (Prior) on Jun 22, 2000 at 16:56 UTC
I can tell you with absolute certainty that FILES= and several of the other CONFIG.SYS commands only affect processes running within the 16 bit DOS emulator. IPCONFIG.EXE is a 32-bit console app, and as such, is not subject to the restrictions of the 16-bit emulation environment. This problem has been kind of intruiging. I've been poring over the MSDN docs, and have found no mentioned limits, other than exhausted system resources, that would limit the number of pipes that may be open. As I mentioned before, pipes are implemented in memory, except in the case of named pipes over a network. What's curious, of course, is that the pipe is being explicitly closed, yet the program fails. It would be useful to know if the ActiveState perl is using _popen/_pclose pipe functions, or the Win32 API interfaces, CreatePipe, CreateNamedPipe, etc. --Chris	[reply]
RE: RE: Re: Win98 strangeness by t0mas (Priest) on Jun 22, 2000 at 17:03 UTC
Just scanned the AP613 source. Didn't find any ocurrence of "CreatePipe" but 25 files contained "_popen". I think it's using either _popen or _pipe after a brief study of win32.c ( but I'm no C expert :). /brother t0mas	[reply]


Problems? Is your data what you think it is?
	PerlMonks