in reply to Re^8: compare two text file line by line, how to optimise
in thread compare two text file line by line, how to optimise

Add a print when at least 1 of the words match if that's the output you want

while (my $line = <FIC>) {
  my @words = split /\s+/,lc $line;
  ++$uniq2{$_} for @words;
  $words2 += @words;
  ++$count2;
  my @match = grep $uniq1{$_}, @words;
  print $line if @match >= 1; # <-- add here
}

[download]
poj

Replies are listed 'Best First'.
Re^10: compare two text file line by line, how to optimise
by thespirit (Novice) on Feb 28, 2016 at 16:31 UTC

    Thank you very much, this is really fast.

    Can you please explain what are the major change that did you apply and the error that i must no do, to speed up my code like you?

    Hope i can modify it later to have the output like i specified in the begening of the post: specially the number of the line that correspond to the intersecion

    If i have problems il will let you now

    You were very kind , thank you very mcuh

    Regards

[reply]

      Looks like your code had 2 loops, each counting to +6 million

      foreach my $che (@b){
  @aa=split(/\s/,$che);

  foreach my $kh (@a){
    @bb=split(/\s/,$kh);

    for ($l=0;$l<=$#bb;$l++){
      for ($m=0;$m<=$#aa;$m++){

      ## this code executes 6 million x 6 million times
      if(($bb[$l] eq $aa[$m]) ){
      ..

      }
    }
  }
}

      [download]

      But within the 6 million words, there are only few thousand different ones so your loops were checking the same word thousand of times more than required. By holding the unique words from file1 in a hash you don't have to loop through 6 million words every time to find a match with a word from file2

      poj
[reply]
[d/l]

        hi i don't understand this part of code

         my @match = grep $uniq1{$_}, @words; what i understand : here we search for $uniq1{$_} in @words

        we know that @words contain the current line, but $uniq1{$_} what does it contain? and if $uniq1{$_} contain the line of the file N°1 how to browse(iterate) it and how to change from a value to another

[reply]
[d/l]

In Section Seekers of Perl Wisdom