Perl interpreter stops when writing logfile

Ratazong has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks!

I have a nested hash data-structure with about 80000 entries, which I want to dump to a logfile for debugging reasons. I use the following line:

print( Data::Dumper->Dumpxs( [ \ %WPs ], [ qw{ *WPs } ] ));
[download]

The output is directed to the logfile by the Windows-7 redirection

D:\tools\perl\bin\perl -Id:/tools/perl/lib -w Tool5.pl report.xlsx > l
+og.txt
[download]

Unfortunately, I get a pop-up-error-window stating that the perl-interpreter is no longer working and needs to be stopped. Looking at the logfile, this error occurs somewhere "in the middle" of the line above - the hash has not been fully dumped. I am using ActiveState Perl 5.8.2 under Windows 7.

Do you have any idea why this could happen and how to prevent it? Fortunately, it is only a minor annoyance, as the script finished processing, and the error occurs afterwards - nevertheless it would be fine if there would be an easy explanation (is there a limit of characters when redirecting script-output?) and a workaround.

Thanks in advance! Rata

Comment on Perl interpreter stops when writing logfile - why? Select or Download Code

Replies are listed 'Best First'.
Re: Perl interpreter stops when writing logfile - why? by Corion (Patriarch) on Dec 09, 2014 at 13:44 UTC
Constructing the string for the dump of a nested hash structure with about 80_000 entries might be too large for your (presumably) 32-bit Perl. In theory, Perl should quit with `Out of memory`, but maybe it doesn't here. Have you considered using Storable or something else to get the data out? Alternatively, don't dump the nested hash structure as one big thing but dump it in loops: `my %struct= (80_000 other hashes); for my $key (sort keys %struct) { print {LOGFILE} Dumper $struct{ $key }; };` [download]	[reply] [d/l] [select]
Re: Perl interpreter stops when writing logfile - why? by LanX (Saint) on Dec 09, 2014 at 13:37 UTC
I can only guess this might be an OS related "feature" to check if an app is still "responsive" by sending signals or monitoring resources. Perl (to be precise the dumper XS code) might be too busy writing and is not replying. Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :)}	[reply]
Re^2: Perl interpreter stops when writing logfile - why? by Ratazong (Monsignor) on Dec 09, 2014 at 14:37 UTC
Dear LanX, Corion and Anonymous Monk! Thanks for your suggestions! Here are some results of my tests: It is depending on the perl-version ... perl 5.14.2 works without problems It really seems to depend on the size of the dump - the result is the same independently if I write directly to the logfile or if I redirect Even with dumping the data in a loop (with redirection), the same error occured (there might be a problem that the perl-script writes the data faster than it could be processed ...) Summary: I "solved" the problem by reducing the amount of debugging-info, and by using a newer perl-version when all data is really needed. Your comments helped me reaching this solution :-) . Thanks a lot! Rata	[reply]
Re^3: Perl interpreter stops when writing logfile - why? by LanX (Saint) on Dec 09, 2014 at 15:32 UTC
> Even with dumping the data in a loop Maybe try to `sleep` or usleep between iterations ... just to nail down the original problem. :) Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :)}	[reply]
Re: Perl interpreter stops when writing logfile - why? by Anonymous Monk on Dec 09, 2014 at 13:42 UTC
If LanX is right, then a workaround might be to write the file from the Perl script instead of using redirection. Also, that's a pretty old version of Perl, why not update?	[reply]
Re^2: Perl interpreter stops when writing logfile - why? by Anonymous Monk on Dec 09, 2014 at 13:52 UTC
On second thought, the suggested workaround actually doesn't relate to LanX's post - it's a workaround for the possibility of something strange going on with redirection, not dumping. Adding an option to your script to write to an output file instead of STDOUT might still be worth it, since then you have more control over it and don't have to rely on the shell for that.	[reply]