Not so much, it drops repeats until a change, though it sorts the contents of the file and strips the newlines, so it's almost certainly the wrong answer, definitely not scalable, and the $past isn't quoted correctly, but other than that it works perfectly ;)

~/$ perl  tmp.pl data
A different line
A line similar to the previous one
Apples are tasty
this is a line which contains the previous line  in it's entirety
this is not the same
~/$ cat data
A
A line similar to the previous one
A different line
Apples are tasty
this is a line
this is a line which contains the previous line
this is not the same
this is a line which contains the previous line  in it's entirety
[download]