Well, good luck. I'll just note that I was not suggesting dumping Mongo. In fact I mentioned that you should be able to query your structured data there (https://docs.mongodb.com/manual/tutorial/query-embedded-documents/). I suggested marshalling your data for insertion into Mongo using a temporary SQL DB instead of the complex and hard-to-maintain Perl data manipulation routines you are struggling with.