in reply to Re^10: compare two text file line by line, how to optimise
in thread compare two text file line by line, how to optimise

Looks like your code had 2 loops, each counting to +6 million

foreach my $che (@b){
  @aa=split(/\s/,$che);

  foreach my $kh (@a){
    @bb=split(/\s/,$kh);

    for ($l=0;$l<=$#bb;$l++){
      for ($m=0;$m<=$#aa;$m++){

      ## this code executes 6 million x 6 million times
      if(($bb[$l] eq $aa[$m]) ){
      ..

      }
    }
  }
}

[download]

But within the 6 million words, there are only few thousand different ones so your loops were checking the same word thousand of times more than required. By holding the unique words from file1 in a hash you don't have to loop through 6 million words every time to find a match with a word from file2

poj

Replies are listed 'Best First'.
Re^12: compare two text file line by line, how to optimise
by thespirit (Novice) on Feb 29, 2016 at 22:45 UTC

    hi i don't understand this part of code

     my @match = grep $uniq1{$_}, @words; what i understand : here we search for $uniq1{$_} in @words

    we know that @words contain the current line, but $uniq1{$_} what does it contain? and if $uniq1{$_} contain the line of the file N°1 how to browse(iterate) it and how to change from a value to another

[reply]
[d/l]
      while (<FICC>) {
  my @words = split /\s+/,lc $_;
  ++$uniq1{$_} for @words;
}

      [download]

      $uniq1{$_} contains all the words from file1 like a dictionary. $uniq1{'anyword'} will be undef or 0 if 'anyword' was not in file1.
      Try a simple example

      #!perl
my %uniq1= (
  cow => 1,
  dog => 1,
  fox => 1,
);

my @words = ('ant','bat','cat','dog','eel','fox');
my @match = grep $uniq1{$_}, @words;
print "@match\n"

      [download]
      poj
[reply]
[d/l]
[select]

        Now i understand

        But my probelm is to compare line by line, and not only find the word in the $uniq1,so i search by combination, may be i must use a hash of array to stock the word of each line

[reply]

In Section Seekers of Perl Wisdom