Re: Hash of Arrays - Pushing Array references and memory impacts question

Rather than posting some non-functional pseudo-code, you should give us a working program that exhibits the (un)wanted effects. Especially with problems of speed and memory-consumption it is essential to see the actual code, otherwise you are unlikely to get useful answers.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Comment on Re: Hash of Arrays - Pushing Array references and memory impacts question

Replies are listed 'Best First'.
Re^2: Hash of Arrays - Pushing Array references and memory impacts question by alittlebitdifferent (Initiate) on Oct 31, 2011 at 23:29 UTC
A sincere thankyou to everyone for their assistance and ideas, To Moritz: Ofcourse!! Slaps Head I saw that reference in Dumper and thought it was wierd - Fixed! Thankyou To BrowserUK: Very smart. This was a great idea which I hadn't tried. I started to write it when I made the decision to drop HAshes. I think it would have worked really well, but I just decided that given the amount of seaches I am doing on this data set the SQLite idea would set me up for some further expansion. However, certainly joining the data would have allowed it to live 'inside' the hash and made it storable which would be fantastic. To ~~David ~~: Excellent Point. I had switched strict off temporarily and had forgotten to enable it again. Saw that ARRAY issue immediately. Thankyou. To Marshall: I am going to try using SQLite. I was concerned as I use Cygwin on Windows XP, that CPAN wouldn't install it correctly for me..however it did and so I think it might be a great solution. Its a little disappointing to drop the Hash as I had come up with a number of extremely create referecing mechanisms and subroutine calls to the arrays....but the amount of data I am dealing with is immense..(1.5GB of text per run) and I do agree having access to SQL queries will benift me in the long run. Thankyou for convincing me I should just bite the bullet. To CountZero: I was too embarrassed about other bits of my code to have you guys look at it but I appreciate your point and will provide a working program in future requests. Although I will say that thankfully, this time around, the answers here were very useful.	[reply]

Replies are listed 'Best First'.

Re^2: Hash of Arrays - Pushing Array references and memory impacts question
by alittlebitdifferent (Initiate) on Oct 31, 2011 at 23:29 UTC

To Moritz: Ofcourse!! *Slaps Head* I saw that reference in Dumper and thought it was wierd - Fixed! Thankyou

To BrowserUK: Very smart. This was a great idea which I hadn't tried. I started to write it when I made the decision to drop HAshes. I think it would have worked really well, but I just decided that given the amount of seaches I am doing on this data set the SQLite idea would set me up for some further expansion. However, certainly joining the data would have allowed it to live 'inside' the hash and made it storable which would be fantastic.

To ~~David ~~: Excellent Point. I had switched strict off temporarily and had forgotten to enable it again. Saw that ARRAY issue immediately. Thankyou.

To Marshall: I am going to try using SQLite. I was concerned as I use Cygwin on Windows XP, that CPAN wouldn't install it correctly for me..however it did and so I think it might be a great solution. Its a little disappointing to drop the Hash as I had come up with a number of extremely create referecing mechanisms and subroutine calls to the arrays....but the amount of data I am dealing with is immense..(1.5GB of text per run) and I do agree having access to SQL queries will benift me in the long run. Thankyou for convincing me I should just bite the bullet.

To CountZero: I was too embarrassed about other bits of my code to have you guys look at it but I appreciate your point and will provide a working program in future requests. Although I will say that thankfully, this time around, the answers here were very useful.

[reply]