In practice, on small data sets, the penalty is a factor of 10 or so. Which is similar to the penalty from using Perl. If you can get a good constant, the log(n) might be paid for. (Real example. Wavelet algorithms tend to be O(n) while the FFT is O(n*log(n)). But the FFT has a better constant.) At some point worrying about how the computer spends a millisecond of time itself becomes silly.

Me, I tend to treat logarithmic factors as noise unless I'm trying to squeeze performance. The difference between linear and quadratic I care about. O(n) vs O(n*log(n)) I don't. I know them, I just don't care much.