Re^2: Algorithm to search and replace on data file, based on a table file?

Replies are listed 'Best First'.
Re^3: Algorithm to search and replace on data file, based on a table file? by dirving (Friar) on Sep 25, 2009 at 02:19 UTC
The technique will still work on Perl 5.8, the only difference is the performance. In Perl 5.8 if you have a regular expression like `foo\|bar\|baz` it tests for each string one a time. In Perl 5.10 there is an optimization that builds a data structure called a trie, which lets it match against all the strings at the same time. So in Perl 5.8 the time taken is proportional to the number of search strings in your table -- in Perl 5.10 the time taken doesn't depend on the size of the table. As I mentioned in my post, you can use Regexp::Assemble to get performance comparable to Perl 5.10's optimization in Perl 5.8 -- it takes care of re-writing the regexp in a more efficient way. Even if you don't use Regexp::Assemble, even on Perl 5.8 this technique will probably be faster than what you have (and almost definitely no slower). You just won't see the order-of-magnitude speedups that you could probably get by using either 5.10 or Regexp::Assemble. -- David Irving	[reply] [d/l]

Replies are listed 'Best First'.

Re^3: Algorithm to search and replace on data file, based on a table file?
by dirving (Friar) on Sep 25, 2009 at 02:19 UTC

The technique will still work on Perl 5.8, the only difference is the performance. In Perl 5.8 if you have a regular expression like foo|bar|baz it tests for each string one a time. In Perl 5.10 there is an optimization that builds a data structure called a trie, which lets it match against all the strings at the same time. So in Perl 5.8 the time taken is proportional to the number of search strings in your table -- in Perl 5.10 the time taken doesn't depend on the size of the table.

As I mentioned in my post, you can use Regexp::Assemble to get performance comparable to Perl 5.10's optimization in Perl 5.8 -- it takes care of re-writing the regexp in a more efficient way. Even if you don't use Regexp::Assemble, even on Perl 5.8 this technique will probably be faster than what you have (and almost definitely no slower). You just won't see the order-of-magnitude speedups that you could probably get by using either 5.10 or Regexp::Assemble.

-- David Irving

[reply]
[d/l]