Re: Caching process sets

It all depends on what you're doing. An obvious solution would be to use a hash (or HoHoHo..oH) that would keep track of what you've already worked on. You would parse the line, check the cache, and do the actions only if the line wasn't in the cache.

As the other poster said, this is only useful if your actions per line are very expensive. "some simple math" doesn't sound like it would be expensive enough. However, 55% does sound like a potentially signficant savings.

Without Benchmarking, it's impossible to know for certain, but the parsing is often the most expensive part of working with logfiles, not the actions one takes at each point.

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Comment on Re: Caching process sets

Replies are listed 'Best First'.
Re: Re: Caching process sets by billyak (Friar) on Feb 19, 2003 at 20:56 UTC
Sorry, I guess I was not clear. I want to cache the set of actions. If line "aaa moo didley" appears thirty times, I want to establish a set of actions based on the first parsing, and follow through with this set of actions for each subsequent occurance of this line. The idea is to avoid the extra parsing by first looking up the event in a hash to see if there has already been a set of actions determined for it. -billyak	[reply]
Re3: Caching process sets by dragonchild (Archbishop) on Feb 19, 2003 at 22:28 UTC
You want closures. How you want closures ... that's going to be based on what you're doing. If you want more help, you're going to need to give a few examples of data and the actions you'd want to take on them. Then, one of us might be able to point you in the right direction. ------ We are the carpenters and bricklayers of the Information Age. Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.	[reply]