Fellow Monks,

One of my systems is a large multi-user Linux setup with several hundred users frequently accessing a selection of utility scripts I've written in Perl. These aren't big or complicated scripts but at peak times the result can be several calls per second to several scripts.

Monitoring the resources of my machine I can see this is absolutely killing the disk. CPU and RAM seem to be ok. I've also tested running this from a Solaris platform (in case this was a Linux OS specific issue) but the same is observed there.

Perl and all modules are installed on a RAID1 pair of 15K 6G SAS disks. This is as far as I can go with the money available for hardware (no SSD I'm afraid!). I've also tested a single 15K disk (to remove any RAID overhead) but no joy.

We've had a similar problems before with standard command line tools and in those cases I statically compiled them and placed the resulting binaries in a RAM disk. The OS seemed to understand what I was after and the disk was hit a lot less. Of course, they were written in C and the same isn't really possible with Perl. I've tried "pp" but it doesn't seem to work out...

I've messed around with the idea of moving my utilities to a TCP socket based "service" (to save the massive number of short lived processes being started) and having users call into that. Problem there is that my users would have to rewrite a lot of their calling code and some basic performance testing didn't really blow me away.

I'm looking for some new suggestions to investigate. Has anyone met a similar issue? I know a good OS (and I'm counting Linux in that mix) should be caching regularly used things in memory anyway but I think I've simply got too many different little things being called for that to work. Any ideas as to how to cache Perl scripts into memory and leave my poor disk alone?........

fx, Infinity is Colourless

In reply to Techniques to cache perl scripts into memory ? by fx

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.