in reply to define analyser - performance problem

gcc (which you could use even if the c++ source was written for a different compiler) called with -E will preprocess all your macros and write the resulting source code to standard output. And maybe the compiler you use has a similar parameter

UPDATE: About your profiling problem: You might try to insert a print "$name\n" statement into the loop and watch the output to see if the time is wasted in every loop or if there are specific macros that waste a lot of time. My (wild, unsubstantiated) guess would be the loop regex because the profiler might have problems pinpointing the time wasted in loop expressions. Also regexes can waste a lot of time if they need to backtrack a lot. I can see that your loop regex has to backtrack when it gets to the last line of a regex, but that shouldn't be enough for the delay you are seeing

Replies are listed 'Best First'.
Re^2: define analyser - performance problem
by Marshall (Canon) on Jun 29, 2009 at 13:01 UTC
    I like the idea of using the compiler itself! I use this -E option when working with macros and gcc. I think this is a standard feature of most C compilers (the option may of course be called something else on yours).

    Another thought...sounds like you are into a maintenance problem. Not sure what kind of application could generate that many macros!!! But anyway you might consider converting a bunch of these things into "inline" subroutines. That yields performance of a macro but with type checking etc on the args. Lots of compilers (including gcc) support this. I presume you are driven into writing this code because its hard to figure what these macros are really doing! Maybe a bigger project that "fixes" the source code is in order? I have worked on projects before where Perl actually writes code and .h files as part of pre-compile step - this is a weird idea, but in right app, it can work. 13,000 macros is a mind boggling number - even on a big ASM project!

    I have found that using some of the special variables like $+ can slow things down a lot with regex - some of these things can introduce extra overhead - sorry can't find a code example of last case where I found this.

    The most likely suspect is the regex related code. I would use a subset of test data to try to find out: a)if exe time scales linearly or exponentially, b)maybe by hacking around, you can find some types of macros that take WAY longer than others. Sorry that I can't be of more help right now.

[reply]
      Yeah, the code is a nightmare. It was Win32 COM application, and at some point people decided to port it to linux. Instead of rewriting code, they have written some classes and macros to imitate COM behaviour (hiding unix functions inside). It is now really a mess and company I am working in, takes it over subsystem after subsystem and tries to clean it up...
[reply]
        I wish you well! Sorry to sound flippant, but this does sound like a mess! The good news is that Perl can probably help! But the job is gonna be "nasty".

        Somewhere along my foreign language training, I heard that VOA (Voice of America) transmits with a total vocabulary of about 8,000 words. That is actually an astonishingly small number for English (you probably have a passive vocabulary of way more than 20K words). I would say that 13,000 is an astonishingly large number of macros! This is so many that we are talking about a foreign language!

[reply]
Re^2: define analyser - performance problem
by grizzley (Chaplain) on Jun 30, 2009 at 06:46 UTC

    OMG! That option is really something! It makes my script useless... I wish I could ++ more than one time.

    Regarding regexp: I suspected it too, I tried to //o, but the time was the same. Then I tried to change it to /#define\s+(\w+)(.)(.)/ but the results were still the same. Don't really know why is that. Maybe some buffering of input data? No idea.

[reply]
[d/l]

In Section Seekers of Perl Wisdom