Re^4: How to optimise " csv into mysql " using TEXT:CSV

I upvoted your response, because I can see where you're coming from and do agree that profiling tools have their place.

I disagree however, with the idea that profiling is never a bad idea. Someone not understanding what the code is doing will get nothing more than a very general idea of their script spending most of its time doing y. They might then spend an inordinate amount of time 'fixing' y to run much more quickly, without ever understanding that changes in other parts of the code means y is called less often (or even completely factored out), potentially resulting in much better performance gains than fixing y might ever yield.

So yes, I agree that profiling is a very useful tool, however it can be a double edged sword and should be used carefully when the developer understands what their code is doing.

Comment on Re^4: How to optimise " csv into mysql " using TEXT:CSV_XS and DBD::Mysql

Replies are listed 'Best First'.
Re^5: How to optimise " csv into mysql " using TEXT:CSV_XS and DBD::Mysql by Laurent_R (Canon) on Aug 07, 2015 at 13:18 UTC
Hi SimonPratt, although I have expressed some disagreement, I also upvoted both your posts, because I understand your point. I disagree however, with the idea that profiling is never a bad idea. That's not what I said. Well, not quite. ;-) I said "very rarely (well almost never) a bad idea". But I fully agree with you that, in order to improve code performance, you need a deep understanding of your program, as well as of the programming language you are using and the external resources your are relying on (files, database, network, etc.). But if I see that, say, my program is spending a lot of time in subroutine Y, then I will certainly not only think about how can I make Y faster, but also if I can call Y less often (perhaps by caching some data), or replace Y altogether with something better. Or even possibly change the whole program architecture and/or algorithm, and re-factor the whole shebang, but that of course comes with a higher cost (non regression testing, etc.), but I have done it a few times. But before stating to do that, I need to ascertain that it is really Y that is guilty. Maybe the problem is somewhere else.	[reply]

Replies are listed 'Best First'.

Re^5: How to optimise " csv into mysql " using TEXT:CSV_XS and DBD::Mysql
by Laurent_R (Canon) on Aug 07, 2015 at 13:18 UTC

SimonPratt

although I have expressed some disagreement, I also upvoted both your posts, because I understand your point.

I disagree however, with the idea that profiling is never a bad idea.

very rarely (well almost never)

But I fully agree with you that, in order to improve code performance, you need a deep understanding of your program, as well as of the programming language you are using and the external resources your are relying on (files, database, network, etc.). But if I see that, say, my program is spending a lot of time in subroutine Y, then I will certainly not only think about how can I make Y faster, but also if I can call Y less often (perhaps by caching some data), or replace Y altogether with something better. Or even possibly change the whole program architecture and/or algorithm, and re-factor the whole shebang, but that of course comes with a higher cost (non regression testing, etc.), but I have done it a few times.

But before stating to do that, I need to ascertain that it is really Y that is guilty. Maybe the problem is somewhere else.

[reply]