in reply to an efficient program?

You briefly mentioned that every entry has a unique ID, then went straight off into wanting to compare each entry with all entries that aren't in the same group without explicitly stating why the unique ID is relevant. If you meant to indicate that you're looking for cases in which the (supposedly) unique ID is duplicated in another group, then this becomes fairly simple: Either in an SQL query or a Perl hash, look for IDs which appear more than once, then check each such ID to see whether it appears in multiple groupings or just multiple times in a single grouping.

But, then, I'm making assumptions here about what I think you may have implied based on which statements are in proximity to each other. You may very well be trying to do something completely different...