Re^3: Time efficiency of object creation in Perl (or the "bless" function)

To clarify a little: I have tab-delimited files that usually end up being around half a million lines long each. Originally, I had set my code up so that it would create a new object for each line, where its "fields" were the attributes. This took "half a day", or to be less colloquial about 4 hours. Now, I'm just parsing each line as a hash of attributes into a set of hashes that stores all the lines have their type attributes in common. This is now taking 16 seconds. I don't think my system is thrashing. I just haven't quite learnt how to use Perl effectively!

Comment on Re^3: Time efficiency of object creation in Perl (or the "bless" function)

Replies are listed 'Best First'.
Re^4: Time efficiency of object creation in Perl (or the "bless" function) by GrandFather (Saint) on Jun 03, 2011 at 23:52 UTC
I think there must have been some major (perhaps hidden) work going on in the OO version that you didn't notice. All else being equal, I doubt very much that adding OO wrappers in the fashion you describe would even add a few percent to the run time of your script - the I/O time alone would bury any OO overhead. If you still have the OO version of the code (in your revision control system for example - what? no revision control? I'll wait while you install Mercurial of git) can you post a sanitised version of the code the exhibits extreme slowness along with a trivial amount of data? I'm interested to see if I can spot the time sink. Were you using an OO library like Moose perchance? Aside from that it sounds like your current approach is appropriate. Again, as a learning exercise, you may like to post a sanitised version of your code for comment. True laziness is hard work	[reply]

Replies are listed 'Best First'.

Re^4: Time efficiency of object creation in Perl (or the "bless" function)
by GrandFather (Saint) on Jun 03, 2011 at 23:52 UTC

I think there must have been some major (perhaps hidden) work going on in the OO version that you didn't notice. All else being equal, I doubt very much that adding OO wrappers in the fashion you describe would even add a few percent to the run time of your script - the I/O time alone would bury any OO overhead.

If you still have the OO version of the code (in your revision control system for example - what? no revision control? I'll wait while you install Mercurial of git) can you post a sanitised version of the code the exhibits extreme slowness along with a trivial amount of data? I'm interested to see if I can spot the time sink. Were you using an OO library like Moose perchance?

Aside from that it sounds like your current approach is appropriate. Again, as a learning exercise, you may like to post a sanitised version of your code for comment.

True laziness is hard work

[reply]