in reply to Re^15: Problem in Inter Process Communication
in thread Problem in Inter Process Communication

Hi BrowserUk,

Please find the answers inline

What type of application servers are involved?

1. These servers are an i/f to the db so instead of firing query to a specifc table all we have

What type of queries are you "firing off"?

2. The queries are simple select queries though can have 100 colummns and can retrun 10K rows as well.

What kind of network bandwidth will the server have?

3. Not Sure

What kind of network latency is involved?

4. Not Sure

What type of "comparison"s are you doing? (Simple string compare or something more complicated.)

5. The o/p returned from prod and test are first string compared . if there is some error then we go into lower levels of comparison.The app server have a charecteristics of messing up with the order of the results returned . so they are sorted first then row by row comparison takes place.There are various clauses that one needs to take care one is exclude certain columns from comparison, compare only certain keys ( these are just a few) in all the comparison is not simple

Why are you redeveloping?

6.1 The QA team would not have to rush to the devlopers each time there is some problem or some enhacement.They know perl better.

6.2 Perl is widely known two be very good is string comparison.

6.3 I have been told by my Lead)(i have to make it and make it good else i would be kicked out :-))

What is wrong with the existing forking solution?

7.1 It is not scalable/extendable

7.2 Comparison logic takes a lot of time.

What language was employed in thae previous solution?

8. C++

Will the 12-cpu box be dedicated to just this task?

9. That i what i have been told but it can go down to 4 cpu also

What are the project requirements?

10. Rewrite/Redesign the C++ code in Perl 5 so that it takes lesser time and is maintaineable.

Which of those is the existing code not meeting? And by how much is it missing it or them?

11. The Suite takes 3 weeks to run i mean the entire cycle i have to bring it down

  • Comment on Re^16: Problem in Inter Process Communication

Replies are listed 'Best First'.
Re^17: Problem in Inter Process Communication
by BrowserUk (Patriarch) on Aug 22, 2008 at 14:54 UTC
      Well I have been told by most all is fine except the comparison that is where majority of the time is spend and this is where perl can excel.

        Well. If my job was on the line, the first thing I would do is ensure that Perl can perform the comparison more quickly than the existing C++ code.

        Get hold of the most complex set of data for comparison and stick it (them? whatever you are comparing it against) in a couple of files and build a C++ exe that calls the existing comparison function to get a base timing. Then write your perl equivalent and see how it compares.

        Unless you can achieve some significant savings in the most expensive area, there is little chance of you achieving anything worthwhile by just porting the application from C++ and forks to Perl and threads.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re^17: Problem in Inter Process Communication
by LesleyB (Friar) on Aug 26, 2008 at 11:40 UTC

    6.1 The QA team would not have to rush to the devlopers each time there is some problem or some enhacement.They know perl better.

    6.2 Perl is widely known two be very good is string comparison.

    6.3 I have been told by my Lead)(i have to make it and make it good else i would be kicked out :-))

    Hmmm ....

    I suddenly have that magic bullet feeling

    From what I've read of your problem, there may be a number of approaches.

    My limited experience of C++ is that it can lead to a lot of bloat. However such code can be trimmed down to be near C-like in performance and size. I think, but I can't be sure, that this is due to the way the code is written and the way the code is built.

    So has the C++ group bothered to review and trim that code?

    The concept that you have been given this task just because there's a rumour that Perl is good at string comparison - without any analysis of how Perl might help being commissioned first, indicates a lack of planning somewhere in your group structure.

    A manager threatening someone with loss of employment on the basis of both poor planning and exploring whether a rumour is true?

    Well, these days, if I were in permanent employment, having verified what planning or assessment of the problem had taken place - and seen the supporting documentation and code to validate chasing this rumour - which in this case I'm guessing doesn't exist - my response to this threat would be to discuss the term 'constructive dismissal' with the individual concerned.

    While one might elect to pursue that matter further, or might not, it may be an effective warning shot to prevent that individual putting you way out on a limb and then sawing it off at the trunk end again.