I came up with a nifty speed trick and I'm wondering if this is a Good Thing (TM) or just a kludge. I have a lot of data spread across about 15 tables in MySQL. One hit to a page in some parts of my application can generate as many as 15 hits to my MySQL database in the background (using DBI).

Some of my data is pretty static, changing only once every few weeks. Some of the searches performed against the data, while complex, will yield the same data over several iterations. Let me cut to the chase here, I have been using XML to store the results of my database calls. The next time I call that search, I check for the existence of the appropriate XML representation, if I have it, I load THAT into a nice reference, if the XML version doesn't exist, run the MySQL code, return the reference, and save a copy of it to XML for the next iteration.

I also have code embedded in my maintenance scripts to wipe the appropriate XML files whenever the underlying data changes and scripts to run through my data and generate the most commonly needed XML representations. Generating the XML in the background means that when someone hits the web site there will most likely be an XML version of the data they are looking for already in the system.

I am not using this for all cases, just where it makes sense to do so. The speed increase in using this is incredible. I have gone from some complex web pages taking anywhere from 1 to 4 seconds to load, to under half a second (this is an estimate, I haven't benchmarked the XML queries). As a bonus, I now have a nice data-export tool that dumps queries into an XML format that I can share across system boundaries.

#### Cut, subroutine from module

 foreach my $tid (@completed){ 
      # does an XML interpretation already exist?
      my $cache_file = "/xmldumps/$tid.xml";
      my $fhh;
      my $flag;

      if(-e ($cache_file)){
          open($fhh, "<$cache_file") || die "Boom $!";
          $results{$tid} = XMLin($fhh);
      close($fhh);
           }else{
     $results{$tid} =  $self->Complex_Crap($tid);
     $self->Save_XML($tid,$results{$tid});
           }
  }

  $self->Results(\%results);
  
  return($self);

}

sub Save_XML {
my ($self,$tid) = @_;

# Save XML representations of data to file system
my $cache_file = "/xmldumps/$tid.xml";
my $file_handle;
open($file_handle,"> $cache_file");
my $data_representation = XMLout($object);
print $file_handle $data_representation;
close($file_handle);
}
[download]

This the the 'magic' part. Read the XML, or perform the complex database search, and save an XML version of the results for future reference.

Obviously, this wouldn't work for binary data, or data where I need to preserve order (making XML::Simple save order is a pain in the ass), but in this case, I don't need binary or order :) My question is, "have you tried anything similar to this" and "is this a good solution"?
-oakbox

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.