Trouble profiling database script

cyberconte has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

Looking through the past posts, i noted that in order to optimize efficently (which i'm trying to do), i should use the DBIx::Profile and/or Devel::DProf module.

This, however, causes a problem. Whenever i ue DBIx::Profile, i get a slew of "DBI Handle has uncleared implementor data" errors - and with the number of inserts i'm doing, it quickly washes out any good results from that module.

Also, when i try to use the Devel::DProf module, it starts with "Subroutine DB redefined at /ur/lib/perl/5.6.1/Devel/DProf.pm line 184", after which it runs, but dies with SEGVs in the children at (seemingly) random places - at least, not in the same place twice in a row (its always in the children, since the parent completes successfully every time). The script runs fine under normal conditions.

(Its probably also worth noting that i fork a lot in the script, for concurrency)

Has anyone else had these problems? Are there other profilers that people reccomend that i can try?

Comment on Trouble profiling database script

Replies are listed 'Best First'.
Re: Trouble profiling database script by tachyon (Chancellor) on Apr 11, 2002 at 12:16 UTC
The Devel::DProf segfault problem is known: =head1 BUGS Builtin functions cannot be measured by Devel::DProf. With a newer Perl DProf relies on the fact that the numeric slot of $DB::sub contains an address of a subroutine. Excessive manipulation of this variable may overwrite this slot, as in $DB::sub = 'current_sub'; ... $addr = $DB::sub + 0; will set this numeric slot to numeric value of the string C<current_sub>, i.e., to C<0>. This will cause a segfault on the exit from this subroutine. Note that the first assignment above does not change the numeric slot (it will I<mark> it as invalid, but will not write over it). [download] cheers tachyon s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print	[reply] [d/l]
Re: Re: Trouble profiling database script by cyberconte (Scribe) on Apr 11, 2002 at 12:57 UTC
i'm almost surpised i havn't found a script that just take another script and adds a couple global variables, inserts Benchmarks at the beginning and end of all subs, and prints out the results of that. It would be rude and crude, but would seem fairly practical, and do profiling w/o relying on anything low-level. Of course i could be wrong, in that either it does exist, or it wouldn't be practical....	[reply]
Re: Trouble profiling database script by perrin (Chancellor) on Apr 11, 2002 at 14:28 UTC
I've used DBIx::Profile a lot with Oracle and never seen this error. Maybe you should report it to the author of the module. It should be easy enough to ignore those lines with something like `grep -v implementor` for now. As for Devel::DProf, it doesn't really work in scripts that fork unless you do some tweaking (like the mod_perl-specific Apache::DProf does). You could look at the technique on this node for using the debugger with forking scripts and see if that works for you. DProf uses the same hooks as the debugger.	[reply] [d/l]
Re: Trouble profiling database script by gregorovius (Friar) on Apr 11, 2002 at 15:55 UTC
I've used Devel::SmallProf and Apache::SmallProf (mod_perl profiler) with good success. These profilers will tell how much time was spent on each line executed, as opposed to Devel::DProf, which I understand outputs time spent on each subroutine. BTW, I used Apache::SmallProf to profile a very large mod_perl application and it was giving me a segmentation fault when logging subroutine calls. I had to "fix" this by commenting out the "sub sub" method in the profiler source code, so it no longer logged subroutine calls. I only saw this problem when profiling the large application. I couldn't spend time on finiding what was it that made it crash. If someone knows about this problem I'd be glad to learn from him/her.	[reply]