Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Warnings and Strict in Production/Performance

by deep submerge (Scribe)
on Oct 17, 2006 at 16:51 UTC ( [id://578835] : perlquestion . print w/replies, xml ) Need Help??

deep submerge has asked for the wisdom of the Perl Monks concerning the following question:

I'm really hopped up at the office right now, in a losing holy war about using warnings in production... Probably not the best state of mind to compose a post, but I hope it spurs a fruitful discussion nonetheless.

Brother Monks, I ask you... Should warnings be enabled on production, yay or nay?

Due to the way our system is architected, it likes to crap out if a script generates a warning, but it will do so silently. There's no way for us to know if a script dived out because of a warning or successfully sent its output to the next system.

So my coworker is on a cleansing mission of sorts, to remove all warning switches from our scripts. I'm adamantly opposed to the idea, as use strict and use warnings are my best friends (as they should be yours).

Coworker #2 notes that I can have the best of both worlds, developing in warnings and sending out to production without warnings (see ENVIRONMENT or PERL5OPT in perlrun).

Our leading programmer agrees with both coworkers #1 and #2 that warnings cause more problems than they are worth. I maintain that really good code won't be generating warnings anyway.

Coworker #2 adds enabling warnings comes at a performance cost. Is this even true? I counter via absurdity that why don't we develop without use strict? Then our code would be lightning fast.

I'm adamantly opposed to running production code without warnings. I think programmers should take more responsibility on the warnings they helped generate. And I think everyone in the office is wrong. Except me.

Monks, I bid you good day. May your code be speedy and true.

  • Comment on Warnings and Strict in Production/Performance

Replies are listed 'Best First'.
Re: Warnings and Strict in Production/Performance
by ikegami (Patriarch) on Oct 17, 2006 at 17:09 UTC

    Warnings are strong indicators of errors. Do you know the cause of the warnings you have received? If the warnings you received reveal code that needs to be fixed, then you can show their worth.

    Are your coworkers planning on altering core and 3rd party modules as well? Wouldn't it be simpler to make warnings die noisily? Something as simple as $SIG{__WARN__} = sub { die ... }; might satisfy them.

    The performance hit of warnings is negligible, IIRC. Feel free to Benchmark them.

    Update: The following demonstrates that 100,000 warnings checks did not slow down Perl at all.

    use strict; use warnings; use Benchmark qw( cmpthese ); my $passes = 100_000; my $tests = -10; sub with { use warnings; my $i = 0; $i = $i + 1 # Undefined check for 0..$passes; 1; } sub without { no warnings; my $i = 0; $i = $i + 1 # No undefined check for 0..$passes; 1; } cmpthese($tests, { with => \&with, without => \&without, });

    outputs

    Benchmark: running with, without, each for at least 10 CPU seconds... with: 10 wallclock secs (10.24 usr + 0.00 sys = 10.24 CPU) @ 20 +.40/s (n=209) without: 11 wallclock secs (10.40 usr + 0.01 sys = 10.41 CPU) @ 20 +.47/s (n=213) Rate with without with 20.4/s -- -0% without 20.5/s 0% --
Re: Warnings and Strict in Production/Performance
by samtregar (Abbot) on Oct 17, 2006 at 17:01 UTC
    Due to the way our system is architected, it likes to crap out if a script generates a warning, but it will do so silently. There's no way for us to know if a script dived out because of a warning or successfully sent its output to the next system.

    This is your real problem. Fix this and the rest will go away. Warnings shouldn't be fatal - that's the whole idea! And of course you'll need a way to view them, throwing them away is suicide.

    -sam

Re: Warnings and Strict in Production/Performance
by chromatic (Archbishop) on Oct 17, 2006 at 17:29 UTC

    In my last paid web development job, I had responsibility for a system that grew over time into a tangled mess. Almost every page view spewed dozens of warnings to the error logs. No one ever looked at them.

    In the course of my refactorings (and writing tests), I silenced all of those warnings. Thus if we ever saw a warning in the error logs, we knew something unexpected had happened -- such as a customer doing something unanticipated.

    This allowed us to refine our tests based on what our customers actually did, not what we thought they might do.

    I would not trade that away for some perceived benefit of several milliseconds -- especially in an application where the greatest performance drain came from network latency.

Re: Warnings and Strict in Production/Performance
by Ovid (Cardinal) on Oct 17, 2006 at 17:03 UTC

    If they claim there are performance problems, ask them for the benchmarks. If they don't have them, they can't use that argument. If they feel that warnings shouldn't be in production code, they've made a serious mistake. Why do you think they're called warnings?

    Cheers,
    Ovid

    New address of my CGI Course.

Re: Warnings and Strict in Production/Performance
by jdtoronto (Prior) on Oct 17, 2006 at 17:33 UTC
    Funny how we all seem to be agreeing on this one! Both pragmae are included in my production distributions - stricture won't return anything - but warnings are logged to a file on disk and then if the user wants to initiate a support issue it is done from within the software and the log is sent to our issue server ( we use Eventum ) and the support tech has the log to work with.

    I don't know about you but most of the warnings I see are 'undefined value' warnings which are taken care of by initialising variables to something when they are created. Config::Simple is particularly bad in my case at throwing warnings. Because my software is always 'calling home' to a SOAP server for data, we have a message that asks the software to send in a log file next time it connects - each user is identified by a unique ID so I can even request a specific user installation. So even in production, warnings are my friend!

    But back to your own situation, if the production code is generating warnings, then you really do want to know why - as Ovid said, pretending you are an ostrich is dangerous - because your butt is very exposed when your head is in the sand.

    jdtoronto

      Both pragmae are included...

      <pedantic>Pragma is a Greek, not a Latin word. The 'learned' plural is therefore pragmata, not pragmae.

      However, like the authors of the Camel Book, I prefer to use the 'English' plural, pragmas</pedantic>

      :-)

      jdtoronto..your concluding statement belongs in Bartlett's Quotations or something..

      "Pretending you are an ostrich is dangerous - because your butt is very exposed when your head is in the sand."

      :) ++
Re: Warnings and Strict in Production/Performance
by brian_d_foy (Abbot) on Oct 17, 2006 at 19:40 UTC

    Update: I'm only talking about warnings coming out of use warnings. This stuff doesn't necessarily apply to warnings the programmer affirmatively generate (e.g. with warn, etc) and intend to go into a log file (or somewhere).


    There are some concerns with warnings in production, and you should decide for yourself how these impact your situation. Like bracing style and other religious wars, arguing with developers over this means you aren't getting work done. Let the project manager decide then get on with life. :)

    I'm not going to vote yay or nay. I'm just going to say to think about the consequences of either course of action and decide which one works for you.


    0: People have to see the warnings. If no one is going to monitor the logs with all the warnings, you don't need the warnings. Also, if there are so many warnings that people will stop looking at the logs, then warnings can be bad. If no one is going to fix the warnings, then what's the point? This might sound stupid, but I've seen plenty of places that don't care about fixing warnings. Catching all warnings sounds good, but in reality people start to filter out the things that show up over and over again, or that annoy them. An ivory-towerish, dogmatic policy usually has unwanted real-life behavior adjustments.

    1: You want to notice important messages. If the script outputs a lot of useless warnings, or the same warning repeated over and over, that's a lot of text to sort through to get to the warnings which might be more important. It's the old needle-in-a-haystack. That's the same for a script that outputs a couple of warnings but executes hundreds or thousands of times.

    2: Warnings such as "Use of uninitialized..." aren't very helpful in log files. If you want to clean those up, you can turn on warnings for a bit, get the warnings, then turn them off again. You might even want to be able to turn on warnings externally (environment variables, etc) so you can decide to have them on or off without editing any files.

    3: A script might start off warnings clean, but then through an upgrade to a third-party module or even perl itself, it all of a sudden isn't warnings clean. I've seen a couple of clients go down because their disk filled up in the middle of the night since warnings that weren't there before showed up all of a sudden. Despite any failings in deployment policy and so on, it can happen.

    5: Turning off warnings to get a performance boost is probably just optimizing in the wrong place. If that's the main argument for turning off warnings, don't bother with the extra work. You don't want to edit files just to turn off warnings. Any change requires attention and monitoring.

    6: Don't just turn things on or off without monitoring the script. No matter how harmless the change I just made, I usually find it's not so harmless. :)


    Now, the goal is always to have warning-clean code, and, as you say, programmers should take responsibility for the warnings their code generates. At the same time, however, a really good test suite with good coverage should catch most of that. Beyond that, a way to replay a particular run (same user input, environment, etc) with warnings turned on should generate the warnings you would have seen for a particular problem.

    I usually recommend that people leave existing scripts as they are and start any new policies with new work. If you have time, slowly fix old scripts. Don't try to fix them all at once lest they all break at the same time because you did something odd while editing them. Avoid spending too much time thinking about the past until you have to (that is, don't mess with what's working). Once you have to fix an old script, give it a general warnings clean up. You have to test the script for your new fix, so you have a chance to what warnings pop out and fix them at the same time.

    --
    brian d foy <brian@stonehenge.com>
    Subscribe to The Perl Review
      1: You want to notice important messages....

      This, for me, is the most important point (and the most important message in your post). I also firmly believe that turning off warnings is inherently wrong; thus putting a script that warns into production and then turning off warnings is a common example of "2 wrongs don't make a right".

      There are always edge cases, but this sounds more like the foundation of the company's coding practices and its investment in IT.

      -=( Graq )=-

Re: Warnings and Strict in Production/Performance
by chargrill (Parson) on Oct 17, 2006 at 17:14 UTC

    I'd argue for leaving warnings turned on in production and making sure they get logged somewhere (not sure what kind of system you're running, but any 'ole logging module ought to work one way or another). For every warning you see in production, you know you'll have to increase your test coverage, because warnings would show up well ahead of the time the code hits production.

    You do test your code, don't you?



    --chargrill
    s**lil*; $*=join'',sort split q**; s;.*;grr; &&s+(.(.)).+$2$1+; $; = qq-$_-;s,.*,ahc,;$,.=chop for split q,,,reverse;print for($,,$;,$*,$/)
Re: Warnings and Strict in Production/Performance
by lyklev (Pilgrim) on Oct 17, 2006 at 17:29 UTC
    There are different types of warnings: Perl-warnings and other warnings. I think you should never see Perl-warnings in production for multiple reasons:

    • variables used without initialisation or declaration, accessing non-existent array cells; this will undoubtedly cause unreliable output
    • security concerns: warnings might reveal more than you want in case security is an issue

    Of course, there are other, non-Perl errors, like "Disk is almost full". These messages should be logged to a file and regularly inspected. I would not use "warn" for this.

    Back to your question: I use strict in production, and depending on the type of application and its audience (general audience: no, expert audience: yes) warnings.

Re: Warnings and Strict in Production/Performance
by neilwatson (Priest) on Oct 17, 2006 at 18:55 UTC
    It seems to me that certain developers are using a supposed performance issue to hide potential errors. By removing the warnings module they are sweeping things under the rug. I suggest that the orignal poster should protect themself by documenting all concerns regarding this practice.

    Neil Watson
    watson-wilson.ca

Re: Warnings and Strict in Production/Performance
by andyford (Curate) on Oct 17, 2006 at 19:09 UTC
    This is scarier than Kim Jong Il:
    Due to the way our system is architected, it likes to crap out if a script generates a warning, but it will do so silently. There's no way for us to know if a script dived out because of a warning or successfully sent its output to the next system.
    Keep the warnings, fix the system. Add some logging to find your problem.

    andyford
    or non-Perl: Andy Ford

Re: Warnings and Strict in Production/Performance
by Dervish (Friar) on Oct 18, 2006 at 03:09 UTC

    If it helps at all, I develop software that runs on embedded systems like printers and factory robots. We develop in C, rather than Perl, but we not only have all build-time warnings turned up to maximum, but we won't accept any production code in which any warning occurs, for the same reasons as are mentioned above. We leave those on at all times, although the ASSERT macro does get commented out for release systems, but only because some of the tests take appreciable amounts of time.

    For our systems as well as for Perl, having a log of user activity -- especially of any warnings generated -- is essential when bugs or problems occur. It's almost impossible to fix a bug if you can't reproduce it, and a good set of warning messages will let you do so. You may also want to dump your errors or warnings to a file rather than to the display, if a 'flawless' look is required for the user interface. On newer systems, we sometimes make the log file a rotating log -- it only retains the last 10 minutes' warnings (or whatever), but that's just to make it easier to manage a large volume of data. You shouldn't need that feature in a working system.

      If it helps at all, I develop software that runs on embedded systems like printers and factory robots. We develop in C, rather than Perl, but we not only have all build-time warnings turned up to maximum, but we won't accept any production code in which any warning occurs, for the same reasons as are mentioned above.
      <humour>
      And also because I imagine that if it is the factory robot the one to issue the warning... it's gonna be a "warning you can't ignore"!!
      </humour>
Question: Warnings and Strict in Production/Performance
by brig (Scribe) on Oct 18, 2006 at 01:30 UTC

    My First Post ;)

    There is no question about using -w and use strict; that much is obvious to anyone with enough Q to code.

    My reply is a question: Where is the deadline?

    I wonder if the true story doesn't have something to do with laziness and irresponsibility? One way to improve your debugging turn around is turning off warnings. See! Look no more errors.

    Reconsider what you are hearing and seeing from that perspective. Many people I've worked with in my life really didn't care that much about what they did (and present tense as well). They punch-in, do enough to get paid but not so much they stand out, they punch out. I see that alot, and the defenses used by those that do it run the gamut of anecdotal to verbal combat. I've never found much logic in the middle.

    Love, --Brig

Re: Warnings and Strict in Production/Performance
by blazar (Canon) on Oct 18, 2006 at 08:50 UTC
    Coworker #2 notes that I can have the best of both worlds, developing in warnings and sending out to production without warnings (see ENVIRONMENT or PERL5OPT in perlrun).

    Oh, c'mon! This comes out quite so often... it seems like (some) people thinks that code in production is not code that should be further maintained any more, possibly by other people. How is code commited out into production? Do they expect one to comment the use warnings; line just before claiming his code to be production ready and to uncomment it on next development cycle? And what if the next developer to take over the code does not know about this "tradition"? (Let's hope he'll be of the school of thought it must be uncommented forever...) Much better to leave it as it is and just like quite everybody here has been suggestiong, (make it warnings free and) adopt a logging system for the wanted warnings. That's how I've seen things done at my last job, and more serious warnings and errors triggered more "invasive" means to draw attention, from emails to groups of people to sms'es to their personal phones.

Re: Warnings and Strict in Production/Performance
by maspalio (Scribe) on Oct 18, 2006 at 06:24 UTC
    Hi,

    Warnings are important and should be kept. This is actually the first question I ask whenever a customer is confronted to ill CAD tool behavior (whether I actually coded it or not): "did you read (and hopefully understood) all warnings?"

    Problem is, your issue seems to be of social engineering kind (i.e., fellow programmers are dead set against warnings) so I would first debunk their cons one by one (benchmarking is IMHO a very good idea) then quiet most or every warning. Of course, you can end up locally inserting 'no warnings' pragma into any dodgy code block you would not have time to properly analyze or refactor.

    My 2 cents anyway...

    Cheers,

    Xavier
Re: Warnings and Strict in Production/Performance
by Bro. Doug (Monk) on Oct 18, 2006 at 00:19 UTC
    I'm of the opinion that all warnings should be handled elegantly, in a log file somewhere. The logs themselves need to be regularly recycled in a production environment, but the warnings must remain in the software or I'll get a massive migraine just thinking about troubleshooting.

    It's useful to write software with a DEBUG switch that offers more warnings. This can be pretty easily incorporated: just use a logger with log-levels and be diligent about it.

    Warnings may offer a performance hit at the machine level, but these days the programmer's performance costs more than processor's.

    Bro. Doug :wq
Re: Warnings and Strict in Production/Performance
by Velaki (Chaplain) on Oct 18, 2006 at 10:39 UTC

    To me, this is tant amount to pilots turning off the various alerts in the cockpit, because they dislike the noise and flashing lights. Shouldn't they be looking at those lights and paying attention to those alerts?

    Wouldn't you agree that flying a passenger plane full of people is a "production" level task? Perhaps they believe the distraction is offset by the perceived increase in concentration, and speed at which they roll back from the gate?

    In a nutshell:

    1. Fix your systems.
    2. Ensure that your code runs without generating warnings.
    3. Monitor the runlogs.

    Leave the warnings turned on. And if your co-workers claim performance issues, either point out to them ikegami's post, or simply remind them of the pilots who turn off their alerts next time they fly.

    use strict; use warnings;
    It's the right thing to do,
    -v.

    "Perl. There is no substitute."
Re: Warnings and Strict in Production/Performance
by Anonymous Monk on Oct 18, 2006 at 09:44 UTC
    Should warnings be enabled on production, yay or nay?

    IMO, it depends. I usually (almost always) turn warnings on, but there might be reasons to turn them off. The question you have to ask yourself is, where are the warnings going? And what action is taken by the people seeing the warnings. Take for instance a CGI program. If things are setup that whatever it writes to STDERR is send to the browser (not that I recommend it to set up your server that way - but you aren't always in control of that), generating warnings may do more damage than good. If the warnings are going to be burried in a log file that gets a few million lines a day, it doesn't really matter whether you have warnings on or not, although with warnings, you still have something to dig for. If you have a program that's run from cron, and cron dutifully mails STDERR to a knowledgable person, by all means, keep warnings on. However, suppose you have a program that runs some support, and generates an excel file. It's run by people who have no idea what "Use of uninitialized value in print" means, and have no idea who is in charge of the program. There's no added value in having warnings turned on.

    So, my recommendation is "keep warnings turned on, unless it does more harm than good".

    Due to the way our system is architected, it likes to crap out if a script generates a warning, but it will do so silently. There's no way for us to know if a script dived out because of a warning or successfully sent its output to the next system.

    I suggest you fix this. Since some warnings can't be turned off and the program can write to STDERR for other reasons as well, you'd benefit from fixing your architecture, regardless of the "warnings on/off" question.

    Coworker #2 adds enabling warnings comes at a performance cost. Is this even true?

    Of course it's true. *Everything* comes at a performance cost. Using a loop instead of repeating lines of code comes at a performance cost. Factoring out code and putting them into subroutines or modules comes at a performance cost. Using objects comes at a performance cost. Using Perl instead of C comes at a huge performance cost. Using general purpose Apache instead of a webserver specially designed and tuned to handle your application comes at a performance cost. Using warnings comes with a performance cost. It's miniscule and typically only used as a strawman argument (which isn't uncommon in the Perl world. Non-measurable performance is used as an argument both by newbies and veterans alike).

    I counter via absurdity that why don't we develop without use strict?

    Not a valid counterpoint actually. Most of the cost of 'strict' is payed at compile time, while most of the cost of 'warnings' is payed at run time.

    There's one point that hasn't been mentioned. Perl isn't always right about its warnings messages. Sometimes it generates a warning because it thinks the programmer made a mistake (usually compile time warnings), when the programmer didn't. Instead of cluttering the code with 'no warning' blocks, one might turn off warnings all together. But then you would do it from the start, not just in production.

Re: Warnings and Strict in Production/Performance
by swampyankee (Parson) on Oct 17, 2006 at 22:32 UTC

    I would guess that use warnings; has some performance cost, if for no other reason than checking for cases like "Use of uninitialized value in print at..." costs something.

    On the other hand, having a program terminating silently is a bad thing: how will anybody know whether it died, let alone how? Sometimes performance is given excess value. After all, nobody should care how rapidly they're given the answer "4" to the question "what is the sine of 4500 radians?" Similarly, the program should do a bit more than silently vanish away when a question like "what is the sin-1(1.5)?" is asked.

    emc

    At that time [1909] the chief engineer was almost always the chief test pilot as well. That had the fortunate result of eliminating poor engineering early in aviation.

    —Igor Sikorsky, reported in AOPA Pilot magazine February 2003.
Re: Warnings and Strict in Production/Performance
by Melly (Hermit) on Oct 18, 2006 at 11:53 UTC

    Hmm, I'm going to play devil's advocaat here (just for fun, you understand, and given the issue that warnings are not being captured in any helpful way)

    First, with regard to strict, let's assume that you have a good development environment, where code is never altered directly in production, then the use of strict in the production environment is redundent - a script doesn't suddenly fail strict just because of input (well, not unless you're doing something very silly). If it passes strict in development, it will pass strict in production (which begs the question, why do your coworkers want it removed).

    Warnings are more problematic, but, almost invariably, the only warning code of a useable quality ever generates is of the undeclared variable variety - and in almost all cases this will be caught in whatever handling you have for false - in other words, the warning is redundant and the issue handled correctly.

    This is not my opinion, humble or otherwise, but it is perhaps a suggestion that the issue is not as critical as all that...

    Tom Melly, tom@tomandlu.co.uk