in reply to Possible Server Resources problem

Perl has essentially no issues with data structure size. A 'typical' perl process does use quite a lot of memory. 10-40MB would not be unusual and you can go a lot larger.

If you are not logging any errors you need to get some debugging happening, otherwise you are working in the dark. The simplest method is to add $DEBUG && warn "Got to here!\n" type statments all though your code and set $DEBUG=1 at the top of the script to make it active. At that point you can accurately see where things grind to a halt.

Your mention of 5/50MB of memory looks like a quote from a shared server brochure. Shared servers limit use of resources in many ways but can certainly throttle your processes. If you have shell access fire up top and watch it as you make a request.

cheers

tachyon

Replies are listed 'Best First'.
Re2: Possible Server Resources problem
by meetn2veg (Scribe) on Sep 10, 2004 at 02:15 UTC
    tachyon - Thanks for your prompt reply - I'm currently still working on the problem (4.10am here)!!!

    I doubt that I'm even approaching 10MB of memory - if one can go by the filesize * nº files 'required' etc... at least as a stating point?!? But thanks for the figures - at least now I know what ballpark we're talking about re: memory usage! :-)

    Re: debuging - I have included a &my_err() sub to sort-of 'trap' errors, but only for opening files and requiring files, nothing else. However, it certainly does the job.

    But the prob I'm having doesn't even get that far! It's nothing to do with headers not being sent either. The whole thing just freezes! Almost as if the program were in some sort of loop - but I can't see where a loop might be! And all I'm trying to do is appent html output to a scalar variable for s///g;ing later in the script.

    The "Gets to here" bit I do in my sleep! (I've learnt the hard way many a time b4! ;-)

    Shared server would be correct, although it was not a quote! I have no idea how many other sites are using the same server, and the host (Powweb) say that they have not changed their server config, neither do they allow their clients shell access. And, understandably, 'cos this is a script problem, neither am I expecting them to offer support (or classes in Perl programming in this case!)

    So, apart from using a debug sub and following the 'flow' of data by putting "Ooh look! I managed to get this far" in the script, is there anything else I might be able to try or am I destined to go thru 1,000's of lines of code, doing CTRL-V and getting my script to go "Boo!" on-screen?!?

    My 'just as important' problem is to find out what's happening so I can temporarily plug the whole, in order to start selling this app. which'll give me time to rewrite completely in a far more 'professional' way. At the moment, I just can't afford another rewrite (which'll probably take > 2-3 months!!!

    Any further suggestions are more than welcome! Please!!!

    kind regards and big thanks in anticipation...

    Richard.

      I doubt that I'm even approaching 10MB of memory - if one can go by the filesize * nº files 'required' etc...at least as a stating point?!?

      You can't make that assumption at all. Hello World in Perl takes over 1MB although it is obviously only one line of Perl. I can use *all* your systems resources with just one 12 char line of perl. Perl is a memory pig in many ways. This reflects the optimisation strategy of spending memory to gain speed.

      Re: debuging - I have included a &my_err() sub to sort-of 'trap' errors, but only for opening files and requiring files, nothing else. However, it certainly does the job.

      You are totally missing the point. You need to see where your code gets to, so you add the debugging as suggested to do that. BTW you dont need the & if you call function().

      $DEBUG=1; $DEBUG && warn "Doing foo!\n"; foo(); $DEBUG && warn "Done foo!\n"; #etc

      The whole thing just freezes!

      It most certainly does not. You just don't know where it stops which is somewhere between starting and completing :-) That is why you need the debugging flow messages. The messages appear in the error log. See CGI Help Guide

      So, apart from using a debug sub and following the 'flow' of data by putting "Ooh look! I managed to get this far" in the script, is there anything else I might be able to try or am I destined to go thru 1,000's of lines of code, doing CTRL-V and getting my script to go "Boo!" on-screen?!?

      You don't seem to get it. You *need* to add the debug code. If the $DEBUG is set it writes messages to the error log for you to review. If $DEBUG=0 it does nothing. The code runs as usual. Thus you can put the code in and leave it. You don't even know where your code gets to, so you have no idea what the problem is. I have not seen your code, nor do I want to. All I can offer is the logic used by millions of programmers across the world. Don't speculate. Isolate the problem first, then fix it.

      But there is a shortcut of sorts. If you don't mind going through reams of output then install Devel::Trace. In your cgi activate it as shown. Every *line* that is executed will then generate a logging message in the error logs.

      #!/usr/bin/perl -d:Trace Devel::Trace::trace('on'); # Enable print "Hello World\n"; foo(); Devel::Trace::trace('off'); # Disable foo(); sub foo{ print "Foo\n" }

      Make sure this sample runs first, then just add -d:Trace to the shebang +/- Disable/Enable it in sections. Here is what happens when I run that and redirect stderr to a file.

      C:\>test.pl 2>out Hello World Foo Foo

      So there is all the expected output, no more no less. Situation normal....but if we look in file 'out' (this will automatically be error_log with a CGI) we have a complete stack trace:

      C:\>more out >> C:\test.pl:3: Devel::Trace::trace('on'); # Enable >> D:/Perl/site/lib/Devel/Trace.pm:31: my $arg = shift; >> D:/Perl/site/lib/Devel/Trace.pm:32: $arg = $tracearg{$arg} while +exists $tracearg{$arg}; >> D:/Perl/site/lib/Devel/Trace.pm:33: $TRACE = $arg; >> C:\test.pl:5: print "Hello World\n"; >> C:\test.pl:7: foo(); >> C:\test.pl:13: sub foo{ print "Foo\n" } >> C:\test.pl:9: Devel::Trace::trace('off'); # Disable >> D:/Perl/site/lib/Devel/Trace.pm:31: my $arg = shift; >> D:/Perl/site/lib/Devel/Trace.pm:32: $arg = $tracearg{$arg} while +exists $tracearg{$arg}; >> D:/Perl/site/lib/Devel/Trace.pm:33: $TRACE = $arg; C:\>

      cheers

      tachyon

        Thanks again for your advice and help - we all have to start somewhere to gain experience.

        "Hello World" @ 1mb - oh! Would never have guessed!
        Debugging - I do/did understand. What I didn't know how to do was your suggested $Debug plan. I will of course be using it - as well as reading the CGI Help Guide.
        If you don't want to see my code, skip this bit ;-)

        my $list; foreach $a(0..$#my_array){ $list .= qq¿<TR>\n¿; foreach $b(0,1,8,16,14){ if($b == 0){ $list .= qq¿\t<TD >${'flag_'.$languages[$my_array[$a][20]] +[1]} ¿; $list .= &build_link("?act=subscriptions_view&what=$my_arr +ay[$a][0]&m0=$m0&m1=1",$target,$my_array[$a][0]); $list .= qq¿\t</TD>¿; } elsif($b == 1){ [Note: $email is null: Included - script fails. Excluded - script com +pletes. ?!? DEBUG should find out why.] $list .= qq¿\t<TD >$email $my_array[$a][2] $my_array[$ +a][1] </TD>¿; } elsif($b == 14){ $list .= qq¿\t<TD ALIGN="right" >$cat[$my_array[$a][14 +]][0]</TD>¿; } elsif($b == 16){ $list .= qq¿\t<TD ALIGN="right" >$diff[$my_array[$a][1 +6]][0]</TD>¿; } else{ $list .= qq¿\t<TD ALIGN="right" >$my_array[$a][$b]</TD +>¿; } } $list .= qq¿</TR>\n¿; } $list .= qq¿</TABLE>\n $files_found <P> \$email = $email ¿; return $list;

        I'm sure the above code is fairly rudementary and undoubtely 100's of ways exist to achieve the same result, but this is what I've got! I have already spent more than several fruitless hours searching and trying to find out why and what 'breaks' just by including an empty variable '$email' in the above code. Again, DEBUG might hold the answer.

        I will of course be following up on all your suggestions.

        Thanks again for your time and help.

        R.