When the file gets to 2 GB, it is possible that you'll have an OS problem.

After that, you're good until a couple of terabytes, or running out of disk.

I'd guess that your problem is due to a simple oversight on your part. A common suspect is forgetting to chomp data consistently - you save a key with a newline in it and then look for the same key without that, or vice versa. Or (more insidiously) you're reading a Windows file on Linux and can't see the \r that snuck into your key.

There are plenty of other possible suspects. But size limitations on BerkeleyDB is not one of them.