If you're on *nix or are using something like Cygwin, you don't even need to write a program. Use the sort utility:

sort -u -o file file
[download]

substituting for file as appropriate.

This has the side effect of sorting the file, but you didn't indicate whether that was desirable or not.