The second option, generating a random root directory, can be easily and quickly set up, but runs the risk of calculation errors. It means that I have to concatenate the root to one or more made up relative paths. Concatenating a root with a relative path may be a trival algorithm, but, all the same, it is definitely vulnerable to stupid typos.

You're right there, but your options aren't totally exhausted here.

• Rely on an existing, well tested solution to do the concatenation for you
• you might be able to chdir into the newly created root dir and use only relative paths
• Maybe your API has an option to hand back the fully qualified paths. If that's the case, you can for example check if the file exists (-e), has some content (-s > 0), maybe even has the right magic numbers etc.
• Another option if you get back the fully qualified paths is to check if the made-up relative part of the path is a substring of the absolute path
• Again another option is to recursively search through your root, looking for files of the right name, or maybe just looking at the total number of files before and after a create operation.

Yes, you give up some precision in testing, but if you do fear the tautologies that might be a worthy trade-off. Or not, depending on your situation of course.

(Aside: I wonder how subversion and git do their testing...surely they have similar issues?)

SVN has different storage backends, and I guess you can pretty much black-box test them by using their API. You pump in lots of new revisions of entirely made-up data, and look if you can later retrieve it. Nobody really cares in which file the backend puts which data.

I guess with git it's pretty much the same: when you add a commit, there's no guarantee in which packfile it will end up; things like git-repack will change that anyway. The important thing is just the files on disk after the write still conform to their file format, and that it can be retrieved again.