join(1) is your friend.

Let's take for example

[am]king ~/a/tm$ cat first.csv 
apple,5
pear,4
ananas,6
watermelon,10
salad,5
carrot,6
peach,5
apricot,7
[am]king ~/a/tm$ cat second.csv 
peach,orange
watermelon,green
ananas,yellow,expensive
apple,red
banana,yellow
apple,red
pea,brown
apricot,orange
pear,yellow
spinach,green
salad,green
[download]

Then we have to sort them and use join to find the lines found only in the first or only in the second file:

[am]king ~/a/tm$ sort first.csv > first.s
[am]king ~/a/tm$ sort second.csv > second.s
[am]king ~/a/tm$ join -v1 -t, first.s second.s
carrot,6
[am]king ~/a/tm$ join -v2 -t, first.s second.s
banana,yellow
pea,brown
spinach,green
[download]

Update 2009 sep 2.

See Re^2: Joining two files on common field for a list of other nodes where unix textutils is suggested to merge files.