I would use a digest not of the file content, but of the file name (computationally cheaper and no collisions possible).

What makes you think that a filename has the special property of returning a unique value for each and every hash function? Hash functions always have collisions, by definition. You can't losslessly compress arbitary amounts of data into 128, 256 or 512 bits. Sure, it is unlikely that two short names share the same hash value, but it is not impossible. And with filenames near MAX_PATH, which is 4 KBytes on Linux and perhaps even larger on other systems, collisions become more likely.

If you are on a unix-filesystem I would use the inode-number of the file (you get that with stat).

The inode is not unique, it is just unique per filesystem. Together with the device number, it should be unique. Problems start when the filesystem layer of the OS kernel has to invent inode numbers for filesystems that do not have inodes. Linux does that for at least the FAT-based filesystems, in linux/fs/fat/inode.c.

Alexander

You are of course right regarding the collisons.

What I meant was that when using filenames rather then content for hashing, you avoid the case that you have 2 files with the same content, thus resulting in a (trivial) collision that can be avoided by hashing the name.

If you have 2 files with the same content and you've detected them, then you've saved some space

Not withstanding afoken's very good points above, the OP is combining his random number with part of the original filename.

I was suggesting he do the same with the MD5, which covers the possibility of there being two copies of the same content under different names in the same directory.

I would also combine the file length into the derived number, as probabilistically, each MD5 will repeat once in each set of files of length modulus 16.