I have not really understood your code, but sometimes it helps a lot to work with call by references instead of call by value.
Call by value will mean one additional copy of your data will be done when a function is called.
Given your amount of data this can speed things up big time.