Hey,

I'm running a photo gallery solution (ImageFolio) coded in Perl running as plain CGI. Everytime a request is received, the software reads all data (category descriptions, etc) from text files (some over 600KB) as well as the photo IPTC information. So having many concurrent users brings the system to a SERIOUS crawl. I'm aware that the software (ImageFolio) is completely non-scalable in the way it has been developed. And I'm working on replacing the whole photo gallery solution for something new and efficient.

However, in the meantime, I'm considering a hardware upgrade for the server. Is there any hardware combination that could possibly speed up the current system (ie: SCSI drives, more RAM, etc)? So that Perl works faster and processes files faster with many concurrent users.

Also, would changing the way I read files in Perl make much of a difference? Right now it uses a while loop to keep RAM usage at a minimum, rather than loading the whole file into an array. I've also considered having some caching system that will reduce the processing load from each request.

Thanks,
Ralph

In reply to Speeding up large file processing by ralphch

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.