in reply to Reading from a fast logfile and storing on Oracle

while ( defined($tail_line = $tail_file->read) ) { foreach my $key ( keys %pattern ) {
That's rather inefficient. If the patterns are just strings (not regexes), you can try something along these lines:
my $regex = join '|', map quotemeta, keys %patterns; while ( defined($tail_line = $tail_file->read) ) { if ($tail_line =~ m/($regex)/) { print "logging $pattern{$1} for $tail_line"; } }

If you can find a possibility to pipe the log directly into your program's STDIN, you can get rid of File::Tail, for which I don't know how efficient it is.

I'd first try to open a connection with DBI, prepare an insert statement, and execute it each time a match is found. If that turns out to be too slow, you can still search for more elaborate ways.

Replies are listed 'Best First'.
Re^2: Reading from a fast logfile and storing on Oracle
by longjohnsilver (Acolyte) on Dec 19, 2008 at 09:43 UTC
    Thanks Moritz,

    Your answer already helped me to get new insights. I'd also wanted to say to everyone that i started to study a great book entitled "Perl Best Practices (O'Reilly)", i got a lot of good ideas out of it;

    F