I see.. I'm thinking once built I could also write that exact structure out to a .pm file to save building it again in the future until the source code changes.
As an optimization, sure. If I were doing this, I wouldn't spend much time on it until it paid off—getting something to walk the data structure or object graph and do the right thing is much more useful.
(The initial parsing phase probably won't be the source of performance problems or scaling, if you do it well.)