in reply to Re^7: How to optimize a regex on a large file read line by line ?
in thread How to optimize a regex on a large file read line by line ?

It took 7 mins with your file. It seems to be related to the line ending not being normal for windows (they are LF only). After I 'processed' your file with this code it took less than 1 minute to scan.

#!perl use strict; my $t0 = time; open FH, '<', "dict.txt" or die "$!"; open OUT,'>','dict1.txt' or die "$!"; while (<FH>) { print OUT $_; } close FH; print time-$t0;
Original
Num. Line : 185866729 - Occ : 14900
421 secs

Converted
Num. Line : 185866729 - Occ : 14900
33 sec

Replies are listed 'Best First'.
Re^9: How to optimize a regex on a large file read line by line ?
by John FENDER (Acolyte) on Apr 16, 2016 at 22:42 UTC
    1mn02, far away better. Thanks !