In the example I mentioned I was around to witness it _was_ a bank and it was the online banking application for business accounts. They spent somewhere around 2 million on the software and planned on spending about $250,000 on the hardware. Turns out they had to spend nearly 6 million on hardware just to get the thing to support the current customer base. Meanwhile we (the internal developers) had the system supporting the personal banking app working on half the hardware, supporting 20x the sessions... Yes, the bank had the money to purchase the additional hardware. Yes they sued the development company for delivering faulty software. Yes, probably far less than 1 million spent on efficiency tuning would have let them save over 5 million on hardware. Just because a company has the money to throw at a problem doesn't mean they should spend it. In this case they had to go that route because of a hard launch date tied to national advertising campaigns. (God I hated those, they'd sometimes tell us of a campaign to start in a few days and then request about 3 weeks worth of work to support the campaign.)

The systems i've dealt with have hundreds or thousands of tables in the backend with tens of thousands of columns and millions to billions of rows. One of them automated the companies knowledge and processes to the point where we could hire $12/hour operations staff instead of $50/hour professionals as the knowledge was manged by the system not the people.

Another company i've seen with efficiency problems is one of the larger ERP application providers recently purchased by a fortune 500 database company. It is massively complex software and I recall hearing the company's tech snickering, leaving a requirements session saying things like "that screen they want to draw will take 20 minutes to generate" (a screen to be used regularly by people in bank branches to service customers).

Does the under 25kLOC count the code from CPAN modules used? The one huge efficiency bugaboo I encountered in a CPAN module " DBIx::Class" (actually it was in redhat's version of perl5.8.8 was the culprit, but it was DBIx::Class that was exercising the slow section of perl). With the problem which was a bugfix of only a handful of lines perl/DBIx::Class got an order of magnitude slower for certain operations. Operations that would definitely affect site performance. If memory serves that bug also affected things other than DBIx::Class but i can't recall what. https://bugzilla.redhat.com/show_bug.cgi?id=196836

I should mention again that I don't value efficiency over flexbility and clean code. But time and time again i've seen the value of having an efficiency mindset. Personally I prefer to start flexible and optimize where necessary.

Consider omething simple like this: 

open(FILE, "<billionlinelogfile.log");
my @file = <FILE>;
close(FILE);
foreach my $line (@file) {
  do stuff...
}

[download]
vs. 
open(FILE, "<billionlinelogfile.log");
while(<FILE>) {
  do stuff...
}
close(FILE);

[download]
Now admittedly for most software professionals, this isn't even conscious thought and was a very basic example.

dratsab

In reply to Re^3: CPU cycles DO NOT MATTER! by bastard
in thread CPU cycles DO NOT MATTER! by dragonchild

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.