Odd eval Problem

nontrivial has asked for the wisdom of the Perl Monks concerning the following question:

Greetings,

I am putting together a (in my opinion :-) very cool Perl application that stores website metadata in a database, and then dynamically generates HTML. Just the initial version has lots of useful features, and I plan on making it Open Source as soon as it is somewhat functional, but that isn't why I'm posting this. :-)

My problem is that I am pulling Perl subroutines out of the database, and after I eval about 20k of code eval seems to get full, and I get errors as though the code has been truncated. I know the code isn't truncated, so I'm pretty sure it is some wierd problem with eval. Here is the example code:

eval "sub " . $SubName . " {" . $SubBody . "}";

After checking here, I came accross http://www.perlmonks.org/index.pl?node_id=5430&lastnode_id=864 and tried the following code for first subroutine to be loaded that goes beyond this mysterious limit:

      my $SubRef;
      my $Hack = "sub " . " {" . $Subroutine->[2] . "}";
      eval "\$SubRef = $Hack";
      undef &{ *VWP_GenQueryHTML{CODE} };
      *VWP_GenQueryHTML = $SubRef;
[download]

Unfortunately this appears to suffer from the same problem.

Can somebody please help? It's frustrating to be so close to a useful project and get stymied like this.

Thanks,
James

Edit by mirod: added code tags

Comment on Odd eval Problem Select or Download Code

Replies are listed 'Best First'.
Re: Odd eval Problem by chromatic (Archbishop) on Jun 10, 2001 at 21:20 UTC
As the perpetrator of that node, you could make the code shorter: `my $Hack = 'sub {' . $Subroutine->[2] . '}'; my $sub = eval "$Hack"; undef &{ VWP_GenQueryHTML{CODE} }; VWP_GenQueryHTML = $SubRef;` [download] We've used a similar technique in Everything to speed up common page components, and haven't run into any trouble with eval "filling up". There are reports that certain versions of Perl have a leak in eval-string constructs, but it's more likely you'll run out of memory first. There's no error checking to see if the compilation fails, and, if you're not extremely careful on the code from the database, it's easy to create some code that'll fail right off the bat For example, if you have a comment as the last line of the code in the database, the interpreter won't see the closing brace of the subroutine declaration. I'd definitely check $@ to see if the eval failed and I'd make sure $sub and &$sub are both defined before assigning to the glob.	[reply] [d/l]
Re: Re: Odd eval Problem by nontrivial (Novice) on Jun 11, 2001 at 05:41 UTC
chromatic, First off, thanks for the reply. Yes, I know I could make the code shorter, but somebody has to balance out brother Russ in a cosmic sense. :-) I believe I have been extremely careful moving the code to the database. For development I have been keeping all the code in one text file. The reason I think eval is "filling up" or whatever is that I have spent time moving various subroutines from the text file to the database. The only constant appears to be the size of the code in the database. Could I be running out of memory with no warning to indicate that? That might be a problem, as I have been using my somewhat underpowered laptop as the development server. And I will definately check $@. James	[reply]
Re: Re: Odd eval Problem by nontrivial (Novice) on Jun 11, 2001 at 20:01 UTC
chromatic, It appears you put your finger on the problem. 6 of the 40+ subroutines are perfectly happy living in a text file, but for whatever reason when stuffed into a database and then pulled out and eval'd they don't like it very much. I suppose it is a bit degrading. :-) I should have been checking $@, and I was in other places, but oh well. It turns out that the 6 lame subroutines also happen to be some of the larger ones, which is why it seemed like I was "filling up" eval when I was experimenting. sigh Now all I have to do is figure out how to make eval happy. I'm already stumped on the first couple problem subroutines, but now that I know what the problem is it's only a matter of time. And this kind of blows in that user code (as opposed to my system code) will have to worry about this problem as well. Anyway brother chromatic, thank you very much for your wisdom and insight. :-) James	[reply]
Re: Odd eval Problem by Abigail (Deacon) on Jun 10, 2001 at 22:11 UTC
It's hard to say what goes wrong. You talk about 20k lines of code being evalled, but you only show a few. There are known problems with `eval`, which especially arise if there are so called "failed" evals (that don't compile) and you have many evals. The causes are not well understood, but a failed eval could set perl into such a state that hundreds, if not thousands, of evals later there will be failures, including core dumps, infite loops, and possible other problems. While evalling 20k lines should of course work, are you really sure you need to eval that much? It smells very fishy. -- Abigail	[reply]
Re: Re: Odd eval Problem by nontrivial (Novice) on Jun 11, 2001 at 05:46 UTC
Abigail, I only included the code I was having trouble with. :-) As I said in another reply, if all the code is in a text file it works fine. If I randomly move subroutines from the text file to the database, it seems, after pulling out about 20k of code eval seems to break. It boggles my neophyte Perl skills. I will explain in another reply why I need that many evals. James	[reply]
(Vynce) Re3: Odd eval Problem by Vynce (Friar) on Jun 11, 2001 at 06:34 UTC
is it possible that the database is not actually storing the whole pile of code? most databases have a size limit for each individual field. try retrieving it from the database and printing it to a file, then compare the new, DB-filtered file to your original file using diff. see if they are really still the same. that's my first guess, anyway.	[reply]
Re: Odd eval Problem by princepawn (Parson) on Jun 11, 2001 at 05:11 UTC
My problem is that I am pulling Perl subroutines out of the database, Why are you storing subroutines in a database instead of in a Perl module ?	[reply]
Re: Re: Odd eval Problem by nontrivial (Novice) on Jun 11, 2001 at 05:57 UTC
princepawn, I am storing my Perl in a database because I am storing EVERYTHING in the database. I have broken all HTML formatting aspects down to where I can store in a table. I store arbitrary SQL queries as entries in a database. I also provide session and value latencies. The reason I'm doing this is that I am tired of reinventing the wheel (well, swiping my old code anyway) everytime I build a web application. It also bothers me to mix HTML code with Perl when I come back a few months later and I'm trying to figure out what's going on. When I am done I will be able to whip up a complex web page in a few minutes, draw it with a standard API, and then handle the logic (and only logic) with Perl. You probably don't see the advantage of something like this, and I'm not going to try to convert you. But I have used a system like this before (TCL/Sybase instead of Perl/MySQL) and I know from experience that it is a powerful and efficient way to create websites. James	[reply]