|
Re: Hard to Debug windows memory error
by fishmonger (Chaplain) on Dec 31, 2015 at 20:11 UTC
|
This may or may not help, but you could try using Devel::Trace or Devel::DumpTrace and send its output to a log file to see what line(s) of code were being executed when it fails.
| [reply] |
|
|
Hi Jandrew :)
1) I see StrawberryPerl64, have you tried with StrawberryPerl 32 bits?
I say this because I installed version 32 bits on Windows Server 2012 64 bits for threads problem or stability.
2) Did you profile your code?
(if there is memory leaks, over usage of RAM...).
With Devel::NYTProf in console:
https://metacpan.org/pod/Devel::NYTProf
perl -d:nytprof foo.pl<br>
./nytprofhmtl --open #to generate the html output and open the index f
+ile)
Peace | [reply]
[d/l] |
|
|
hotchiwawa Thank you for your suggestions. At this point I would rather try and get StrawberryPerl64 to work or at least get some kind of coherent/small failing test case rather than/before I use a 32 bit StrawberryPerl From a general memory standpoint I am not really budging the needle (~72M of RAM before dying) but I'm doing my checking in the task manager so there may or may not be a different result in perl. However based on my research this error code can occur (among other reasons) in windows when an attempt to access a memory location is made after the contents have been deleted. Currently I'm following up on fishmongers suggestions as well as looking into Devel::cst and Devel::SizeMe.
Update: it looks like Devel::cst and Devel::SizeMe are both Linux specific modules requiring execinfo.h Devel::cst in this case seemed very promising.
| [reply] |
|
|
|
|
fishmonger++ perl -d:Trace 01-libxml.t errored out after the line from the primary module
$self->_DESTROY which I believe is a Moose addon. This seems like the biggest smoking gun. I got no joy from Devel::Dumptrace. The first time I ran it in verbose mode it overran the memory on my 16G Ram machine. The second time I ran it in quiet mode and I couldn't tell if it locked up or was just grinding away with no output but after about a half an hour I quit.
Based on the last executed line from Devl::Trace as $self->_DESTROY I'm suspecting some trickiness that I tried where two instances are holding an attribute with a reference to each other. In fairness the simple case of this doesn't fail but I suspect that there probably is a really large "don't go there!" sign somewhere in the Moose documentation that I missed. I think I have some possible workarounds to try but I have reached the limit of root causing this based on available tools from the Monastery so I'm headed back to the architecture drawing board to poke at alternatives. It would have been nice if the tricky cross referencing had worked though.
| [reply]
[d/l] |
|
|
| [reply] |
Re: Hard to Debug windows memory error
by GotToBTru (Prior) on Dec 31, 2015 at 19:58 UTC
|
Can you use $SIG{__DIE__} to add diagnostic code? At least to identify where the program is failing.
But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Romans 5:8 (NASB)
| [reply] |
|
|
I should have mentioned that I tried that too and the script errors out before the $SIG{__DIE__} handler can print. Update: Windows opens a popup prior to the final error saying Perl interpreter stopped working The actual error message comes up in a pop-up window not as part of the script.
| [reply]
[d/l] |
Re: Hard to Debug windows memory error
by ysth (Canon) on Dec 31, 2015 at 19:25 UTC
|
Do you know whether your script is actually finishing? Or even fully getting started? Try cutting out bits of the script to see whether that has any effect and to see if you can come up with a minimal complete example that still fails that you can show us.
--
A math joke: r = | |csc(θ)|+|sec(θ)| |-| |csc(θ)|-|sec(θ)| |
| [reply] |
|
|
The very last line of my script is a print statement that does print under several tested environments. I suspect the problem is in the perl garbage collection. The difficulty is the module I'm writing using the script is is complex and multi-layered. I was hoping to get some debugging advice/techniques that I could use so that I could narrow it down to a simple script that reliably fails.
| [reply] |
|
|
| [reply] |
|
|
|
|
|
Re: Hard to Debug windows memory error
by stevieb (Canon) on Dec 31, 2015 at 20:25 UTC
|
Can you make your code publicly available? Perhaps put it on github so those of us who have access to multiple platforms can test it out...
| [reply] |
|
|
| [reply] |
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Jan 06, 2016 at 19:08 UTC
|
Status Update 1: I have tested the fail on Strawberry perl 5.18, 5.20, and 5.22 on the failing windows machine and they all failed. Weirdly this is the only perl script failing in this way on that machine. I tried the failing script on a different windows machine and it passes. After working on reduction of complexity in the background modules the failure still occurs but has migrated to an earlier point in the script. (Although still failing at the $self->_DESTROY command when traced in the debugger) I suspect some weird corner case interaction between Strawberry perl and windows but since another windows box passes I'm now pursuing this as a pure windows problem. I just wish I was able to narrow it down enough to write a bug case for submission somewhere.
| [reply] |
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 05, 2016 at 20:06 UTC
|
Status Update 3: In a potentially unrelated issue, while working on the code base that originally prompted the question I ran into another garbage collection issue on a different windows machine. I'm still in the process of getting the test suit to fully pass after the fix but I thought it useful to post what I found since it is relevant to at least one thread in this overall node. For those in the know it turns out that perl does two-stage garbage collection and for objects that have circular references (which mine do in spades) the object destruction is deferred to the exit phase not just when the variable where the object is stored goes out of scope. There is a warning about this in perldoc for perl 5.8 but the perldocs for 5.22 Destruction are silent on this issue luckily my google foo was strong when I found this. Additionally, The very underrated but handy Perl Cookbook has a specific recipe (13.13) to fix this as well! I have included two sets of example code that represent information gleaned from both sources. The first download is an object module that is self referential but still cleans up as soon as it goes out of scope (Ring.pm) and the second is a script which uses the module for demonstration and then shows some de-referencing magic for simple recursively referenced objects. I have chosen to use the Cookbook style solution in my case.
Update: removed link to the pirated copy of the Perl Cookbook with apologies
| [reply]
[d/l]
[select] |
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 08, 2016 at 20:25 UTC
|
Status Update 4: I have filed a ticket for Strawberry Perl based on the failing results of depends.exe however these same fails are present in a machine that passes this test so I'm now not sure that this is the root cause of my originally posted problem. I suspect that the issue lies in core perl itself at this point. I remain stuck on how to boil this issue down small enough to submit a case that will resolve the problem.
Update: to clarify the failure appears to be the way core perl does garbage collection on libxml2 objects. (murkier) I was unable to find obvious open bugs in libxml2 or perl on this issue.
| [reply] |
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Jan 25, 2016 at 03:10 UTC
|
Status Update 2: For anybody wandering by this thread. After a full hardware test on the laptop and a complete re-image of the OS I still get the failure. Yes it continues to be hardware dependent but I don't have any way to writing a test case to submit either for bug fixing in Strawberry perl or to the laptop vendor.
| [reply] |
|
|
| [reply] |
|
|
Thank you for the suggestions. It got me further down the road than before. The googleverse seems to be mixed on the missing dynamic dll's and the concrete suggestion of reinstalling visual studios 2013 - 2015 didn't fix the problem. I also can't seem to find any useful suggestions on resolving the kernelbase.dll not returning 1. I keep poking at this when I have time but clearly my attention has wandered.
Update: I stripped the log down since it exceeds perlmonks post size. If something is relevant that was removed let me know and I can send it to you.
| [reply]
[d/l] |
|
|
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 15, 2016 at 05:52 UTC
|
Status Update 5: At this point I have tried everything I can think. I have found some interesting things but none of my discoveries have been conclusive and to make matters worse I have started getting some similar fails for my Travis-CI testing in linux. In the end I think I may need to persue a non XML::LibXML solution. I hate to roll a pure perl alternative but I'm not even beginning to excersize libxml2 to it's capabilites and I can't seem to get a clean test.
| [reply] |
Re: Hard to Debug windows memory error
by jandrew (Chaplain) on May 11, 2016 at 00:19 UTC
|
Status Update: 6 The package without XML::LibXML has been released as Spreadsheet::Reader::ExcelXML and is available for use. It does appear to work where the old one did not including a confirmed pass for two prior failing machines.
| [reply] |
|
|
Hi. I've briefly reviewed this entire thread, but I did not expand every readme, nor follow every link;
Did you ever post (minimal) code that demonstrates this issue?
I ask, because vaguely remember reading your OP from 5 months ago and thinking that when you got around to posting code that I could run, that demonstrated the problem, this would probably be quite simple to tie down to a cause. From my review, you never have?
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
| [reply] |
|
|
|
|
|