If your files are really large, then they may exceed your available memory when you try to store data in the %seen hash. In that case, it might either crash or become painstakingly slow.

Please provide an estimate or your files' sizes. Right now you appear to store each file twice in memory, you could at least reduce this to only once, and this might be sufficient to get rid of the problem.