What is consuming the memory? Is it data you are loading? Or data you are generating? Or?

I guess that's the first question. I did an strace on it, and the first thing that I see is that the mmap() calls total up to 12,512,290. (But this includes 7,340,032 that did not succeed...)

“Ahem... let me try that again... awk, if you please... thank you. Now, where was I?”

The mmap calls for the program as it runs to normal completion total 8,329,178 bytes. Which, frankly, doesn't seem like a lot.

I really don't know what else might be found in an strace output that would point the way to more memory-allocations. I do know a lot about the program itself.

This is a Moose / Mojolicious program which uses DBIx::Class for database-access and Template::Toolkit. It doesn't get fancy with a lot of memory allocation... at least, not in my code. All in all, I think it's a fairly “run of the mill” application that does use a lot of CPAN material.

Since the program does run, after a fashion, even with these constraints, about the only logical culprit I can think of is DBIx::Class. That is to say, “that's what tips it over the edge of the cliff,” whereupon it bounces and goes on to successfully produce the template and so-on.

But what I am hoping to find is something “quick 'n dirty” that will substantially reduce the footprint, and do so quickly without demanding major logic-changes. (Y’know, like “rewriting it in PHP...” ;-) )

Edit:   There are also brk() calls, the first one setting the limit to 0x814df20 and the last to 0xa267000, a span of 34mb... which does not look good. :~}

Learning... it's not for sissies...

You might be able to gather more information on what is using the memory using Devel::Size v0.72.

If you run:

use Devel::Size qw[ total_size ];
...

printf "%s :: %.f\n", $_, total_size( $::{ $_ } ) for keys %::;;
[download]

At some point before the crash occurs (experiment), then it will produce a list something like:

/ :: 382
stderr :: 289
SIG :: 3705
, :: 399
utf8:: :: 4359
" :: 347
_<c:/Perl/site/lib/auto/Devel/Size/Size.dll :: 479
DynaLoader:: :: 55772
Devel:: :: 22977
strict:: :: 6435
stdout :: 289
AllocMemory :: 2263
&#8597; :: 262
| :: 383
_<c:/Perl/lib/auto/Time/HiRes/HiRes.dll :: 463
Mac:: :: 1706
CleanString :: 15703
Regexp:: :: 948
_code :: 443
UNIVERSAL:: :: 1872
overload:: :: 39770
$ :: 287
time :: 829
NewString :: 13443
size :: 820
Data:: :: 111665
- :: 679
_<..\universal.c :: 363
_<HiRes.c :: 343
BEGIN :: 287
! :: 380
IO:: :: 943
&#9788; :: 399
total_size :: 844
&#8593; :: 345
pp :: 12454
_ :: 345
_<c:/Perl/site/lib/auto/Win32/API/API.dll :: 471
+ :: 679
Exporter:: :: 81873
Internals:: :: 3436
STDIN :: 287
Config:: :: 91676
warnings:: :: 59111
DB:: :: 850
Time:: :: 33192
_<.\win32.c :: 343
&#9644; :: 345
_<perllib.c :: 343
2 :: 388
_<API.c :: 335
cmpthese :: 27607
1 :: 406
&#8616;ARNING_BITS :: 400
CORE:: :: 930
_<Size.c :: 339
attributes:: :: 962
stdin :: 287
ARGV :: 405
INC :: 2857
Scalar:: :: 1789
ENV :: 6689
? :: 395
vars:: :: 9565
subs:: :: 3764
_<..\perlio.c :: 351
main:: :: 720485
AutoLoader:: :: 24976
Carp:: :: 31355
VMS:: :: 1229
Win32:: :: 256452
PerlIO:: :: 2378
0 :: 431
 :: 562
_<..\xsutils.c :: 355
@ :: 950
ApiLink :: 10523
Benchmark:: :: 131181
STDOUT :: 289
] :: 355
3 :: 386
&#8616; :: 383
MIME:: :: 1215
STDERR :: 289
_<dl_win32.c :: 330
<none>:: :: 460
sleep :: 839
[download]

Which is a crude assessment of the memory being used by each of the packages you have loaded (plus other stuff). That may allow you to zero in on one particular package that is being profligate and then look at that in more detail by examining its symbol table more closely:

[0] Perl> printf "%s :: %.f\n", $_, total_size( $::{ $_ } ) for keys %
+{ Benchmark:: };;
timesum :: 112
__ANON__ :: 112
cpu_a :: 112
a :: 112
n_to_for :: 112
_doeval :: 112
iters :: 112
_Usage :: 112
Min_CPU :: 112
init :: 112
cmpthese :: 27607
debug :: 112
export :: 112
new :: 112
timestr :: 112
EXPORT_TAGS :: 112
timethis :: 112
countit :: 112
EXPORT_OK :: 112
confess :: 112
EXPORT_FAIL :: 112
timediff :: 112
cpu_c :: 112
b :: 112
ISA :: 112
export_to_level :: 112
Default_Style :: 112
timethese :: 112
time :: 829
Cache :: 112
cpu_p :: 112
Debug :: 112
clearcache :: 112
BEGIN :: 287
Do_Cache :: 112
runloop :: 112
timedebug :: 112
real :: 112
usage :: 112
clearallcache :: 112
timeit :: 112
EXPORT :: 112
croak :: 112
import :: 112
disablecache :: 112
enablecache :: 112
Min_Count :: 112
carp :: 112
Default_Format :: 112
VERSION :: 112
mytime :: 112
[download]

Make sure that you get my unofficial v0.72, as previous versions use a substantial amount of memory for internal tracking that skews the results and would probably push you over the limits. My version doesn't suffer that defect.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

This is a Moose / Mojolicious program which uses DBIx::Class

Well, that sounds like your problem right there. You also mention in the original post that it is a CGI program, which Moose have been known to attack and bite ;)

First, if it really is a vanilla CGI program, try switching to a persistent environment this will use less memory over time since a lot of Moose and DBIx::Class only grab memory once at compile time and are they pretty reasonable thereafter.

If that doesn't work, or if you already are in a persistent environment. Try swithing Moose to Mouse (or possibly Any::Moose) as that will at least reduce that part of the memory footprint.

-stvn

Skim the fat. First thing I'd do is run it with -Dolm (needs perl compiled with -DDEBUGGING) to see where it sucks. Then maybe inspection via Devel::Leak::Object could be interesting.

Today.

How much is left of your day? - get a cruft of hackers... ;-)

When I run Devel::Leak::Object, I am presented with a modest list of objects, mostly in Class::MOP and Moose::Meta, none of which (to me...) appear “particularly remarkable.” In fact, all of them appear to be associated with CPAN modules of quite-reasonable provenance and reputation.

I am rather pressed to believe, at this point, that this application is simply “a little bit too-big for the space that it's got to fit into.” And, perhaps, through no real fault of its own. (Which is not necessarily the verdict that I would have preferred to hear, of course.)

Those objects shown as "leaks" are your Moose metaclass instances which need to live for the entire life of your program, so they really aren't leaks. You might want to give Devel::Events a look, it is a much more fine grained leak checker. And lastly, yeah sounds like your program is just to big for the shared host environment.

-stvn

Re-write your classes using hash-based objects rather than Moose. You'll have to generate your own method subs, but none of the program logic will need to change much.

They'll take 1/10th the space and run 3 to 5 times more quickly.

Avoid globals (you are using strict, I hope).

You are aware that strict doesn't prevent you from having globals, aren't you? Declaring all your variables at the beginning of the program strict doesn't object to in the least. All it cares about is whether there's a 'my' (or a fully qualified path). It doesn't care about narrow scope. strict is not the silver bullet many people think it is.

I do. But yours is an important point.

use warnings; use strict; ... throughout, of course.

I can now conclusively see that it fails when loading DBD::mysql (specifically when loading mysql.so under conditions known to be correct):   it successfully loads, but then fails to initialize that module. This produces the message:

Undefined subroutine &DBD::mysql::db::_login called ...

... because the _login subroutine is implemented in the XS-extension .so that didn't get loaded. (Other messages, such as unexpectedly increased this-or-that, can occur for the same fundamental reason.)

But the program keeps going, by design. In fact it produces a credible “an error occurred” output of its own making. (In my local tests, it has about a 50% chance of doing that; at other times it bombs with Perl's sudden-death “Out of Memory!”)

As the final coffin-nail to the diagnosis, the program is known to work properly both in my test-rig and in the hosting company's test rig. And I can induce it to fail ... 50% or so of the time “in the same way” ... through the use of ulimit as described in other recent threads by me.

So the fundamental nature of the problem, by now, is conclusively known ... copious thanks to the Esteemed Monks!

This doesn't sound at all like running out of memory. It sounds like failing to access files or having a broken DBD::mysql. The only time you have memory problems is when you cause them with ulimit.

Pursuing the suggestion to use Devel::Size, I find a memory-profile that ends like this (sorted in ascending order):

   112475 :: Mojolicious::
   178335 :: HTML::
   186117 :: Data::
   241205 :: B::
   246604 :: DBD::
   248277 :: MojoX::
   253124 :: File::
   323990 :: DBI::
   410491 :: Template::
   491445 :: MooseX::
   505433 :: DB::
   626215 :: DateTime::
   645231 :: Mojo::
   847504 :: DBIx::
  1072300 :: Moose::
  1557717 :: SPE::
  2066977 :: Class::
  7846344 :: main::
[download]

Now, I surmise that main:: is the all-encompassing name-space, and of course SPE is the application name-space. I also know that B represents Perl itself. Everything else is, well, “what’s left.”

I pared-down the application namespace (by nearly 3 megabytes...) by eliminating Schema objects that I didn't strictly need. The application already does things like using require to demand-load a page only when it's the page that has been requested.

Does anyone have any ideas what I can do to squeeze this thing down just a little bit more?

Well-l-l... the hosting-company's ever-helpful “technical support” (sic) team offered the helpful suggestion that I could get more room by switching from PHP-4 to PHP-5...   :-D ...

Urk!

If Devel::Cycle prints a report of 89 memory-cycles at the end of my program, am I correct to surmise that “I might be on to something (big and really significant...) here?”

A typical cycle looks like this, and all of the cycles found, look like this:

Cycle (72):
        $Mojo::Server::CGI::A->{'app'} => \%SPE::B
                 $SPE::B->{'renderer'} => \%Mojolicious::Renderer::C
        $Mojolicious::Renderer::C->{'handler'} => \%D
                            $D->{'tt'} => \&E
                     $E variable $self => \$F
                                   $$F => \%MojoX::Renderer::TT::G
        $MojoX::Renderer::TT::G->{'tt'} => \%Template::H
             $Template::H->{'SERVICE'} => \%Template::Service::I
        $Template::Service::I->{'CONTEXT'} => \%Template::Context::J
        $Template::Context::J->{'LOAD_TEMPLATES'} => \@K
                               $K->[0] => \%Template::Provider::L
        $Template::Provider::L->{'LOOKUP'} => \%V
        $V->{'/var/www/spedemo-dev/public/cgi-bin/../../templates/pre_
+process.tt'} => \@T
                               $T->[0] => \@S
                               $S->[4] => \@T
[download]

The more I look at this, the more I think it's a red-herring... All but one of the cycles appear to be identical at least as far as the line which reads LOAD_TEMPLATES. I think I'm just looking at a curiosity of the template-engine's implementation ... i.e. a circular queue.

I surmise, upon reading Template/Provider.pm, that this most-definitely is a non-issue. The DESTROY method in this module makes explicit reference to the circular structure, and its effect upon Perl's memory-management ... and it contains the necessary code to clean all of that up in its DESTROY method, which quite-obviously has not run yet.

Okay... red herring confirmed. Mea culpa. There are no memory-cycles other than these, which is, in and of itself, a good-mark for my program. I conclude that I have eliminated this potential issue, in spite of the “89 cycles.” My program doesn't appear to contain any bona fide leaks of this nature.

I am now calling this thread [SOLVED] because after reasonable investigation (using Devel::Cycle and Test::Memory::Cycle and other tools suggested), I conclude that the program is ... pure and simply ... too big for the narrow confines of a “shared hosting” deployment, and through no fault or error of its own. The site will be deployed from the outset using Managed or Dedicated Hosting.

From the start, my test-suites for the so-called Context object structure, which is what everything else related to a request “hangs from,” already check that there are no circular references (that are not properly “weakened”). So, I was pretty confident of that.

So, how can I place this program on a quick memory weight-loss diet, without recoding the thing? (I'm going to assume that a 32meg limit is probably fairly typical among shared-hosting companies, and “shared hosting” is where it is right now...) I don't have to reduce it by some huge amount – just enough to make it run... Today.

I see London , I see France, I see a huge leak in some underpants