If you are on a Unix system that has a sort command that
supports the -u switch ( -u implies Unique ) you could do this

$ sort -u db.txt > outdb.txt
[download]

I know that Solaris's sort has -u and Red Hats cygwin does not.
I hope that helps :)