Surprisingly, or perhaps not, the problem you describe here and Challenge: 2D random layout of variable-sized rectangular units., are the same problem--and it's a tough one.

The hardest part is arranging for the branches of the directed graph to morph whilst maintaining the intersections between it and the matrices without requiring wholesale re-building of the entire graph.

I think the solution is to add an extra level of indirection. That is, instead of storing a reference to the taxon in the nodes of the graph and matrix, store a reference to a reference to a taxon. This allows either structure to be manipulated without forcing a full-scan reconstruction.

That extra level of indirection has a penalties though. Beyond the performance/complexity impacts, the biggest is that it is really very hard to visualise, which makes coding it very hard indeed--for me anyway.

I only mention it because the idea of the extra level of indirection might gel in your mind and give you ideas for a solution.

I've been trying to finish my attempt at my challenge, but life is being particularly interventionist at the moment, and it really requires an uninterrupted, sustained period of coding time to write down the solution that is half-formed in my mind.