You may have generalized yourself into a corner.

The fastest way to deal with optional data is to make it required in the actual database but have a value that means "NO DATA". Then you can use a normal join, which will outperform an outer join. If you can't do that, you have to use an outer join or do multiple queries, one for each record. Multiple queries would be slower than what you're doing now, so I don't recommend it.

Be careful when generalizing your schema. One of the things databases do is enforce certain rules about your data relationships. If you don't let the database do that, you can get into tricky territory.