manipulating a matrix

chexmix has asked for the wisdom of the Perl Monks concerning the following question:

(Title updated per mscharrer)

O Monks -

I fear may have spaghetti on my hands -- I hope not -- but am trying to limit my gaze (for now) to the part of the project in front of me -- if this small part proves impossible (for Perl? naah) I may have to address the larger mass of spaghetti and/or seek more help here.

Anyhoo. What I have is an array of anonymous arrays over which I am iterating to get a result set (I'll be more explicit about this in a minute, in order to help it make sense). The problem is that I now need to DELETE the anonymous arrays in the result set, so I can iterate BACK over the (modified) array of arrays to check whether there is a second set of results that still meet the criteria. In short, I want to remove the records that satisfied the first run so I don't find them again.

I looked at splice, but that seems to insist that one knows the indices or range of indices one wants to work on. Here, it's the contents of each anonymous array that are of interest.

I'm thinking there may be a way to sort the array in such a way that the result set appears first (or last) and then be popped or pushed off, but ... honestly, my brain is tired and I would like PerlMonkish input on this. I'm operating at the limit of my Perl baby talk (this is a good thing, really. I am learning -- I hope -- but it is also tiring) and may need a hand stepping forward.

To be more explicit about what this is doing (or why), the anonymous arrays contain three values: 0 is an x value, 1 is a y value, 2 is an ID number. x and y represent the coordinates of a single observation (referenced by the ID number) recorded by a CCD chip in a telescope. So the array of arrays represents all the observations recorded on that chip. I'm trying to find out whether the number of observations within strips of a given number of pixels exceed a set threshold value.

Then I want to remove those results and see if there is a second strip that _still_ exceeds the threshold.

The array of arrays look like this:


(
  [48, 219, 11021],
  [72, 190, 11006],
  [203, 177, 11005],
  [301, 186, 11013],
  [309, 119, 11015],
  [309, 216, 11017],
  [309, 147, 11000],
   ....

  [343, 180, 10990],
  [346, 179, 10989],
  [451, 258, 11008],
  [838, 162, 11014]
)
[download]

In this example set, there are a bunch of records between x=309 and x=338, enough to go over the threshold and cause the program to report them out. Now that I have found those, I want to delete them, so I don't find them again when I go back over the array of arrays.

What is the best way to accomplish this? As always I appreciate any guidance, and apologize for the mental fatigue (and other weaknesses) that prompted me to post this at all ...

Regards,

Comment on manipulating a matrix Download Code

Replies are listed 'Best First'.
Re: manipulating an array of anonymous arrays ... by citromatik (Curate) on Apr 25, 2008 at 14:36 UTC
If your work is based heavily on matrices manipulation I recommend you using PDL. Just for the record, if I remember well, the creator of PDL works on something related with astronomy, too. Maybe PDL::Matrix has something useful for you Update: There is also another module that can help you in the creation and manipulation of matrices, Math::MatrixReal. citromatik	[reply]
Re^2: manipulating an array of anonymous arrays ... by chexmix (Hermit) on Apr 25, 2008 at 16:18 UTC
I have gazed longingly at PDL for some time but am not sure my basic Perl knowledge is up to it (this is probably an excuse ... I can be good at excuses). The real restrictions are time (to learn and implement it) PDL may not be available to me in the environment this tool will run in I will have to check on the second item. I know others here use PDL, but the module situation here is complex. Thanks!	[reply]
Re: manipulating an array of anonymous arrays ... by mscharrer (Hermit) on Apr 25, 2008 at 15:05 UTC
You can use `grep` to generate a modified copy without the sub-arrays you don't want: use strict; use warnings; my @matrix = ( [48, 219, 11021], [72, 190, 11006], [203, 177, 11005], [301, 186, 11013], [309, 119, 11015], [309, 216, 11017], [309, 147, 11000], [343, 180, 10990], [346, 179, 10989], [451, 258, 11008], [838, 162, 11014] ); # e.g. filter out all sub-arrays with x <= 309: @matrix = grep { $_->[0] <= 309 } @matrix; local $, = " "; local $\ = "\n"; print @$_ for @matrix; __END__ Output is: 48 219 11021 72 190 11006 203 177 11005 301 186 11013 309 119 11015 309 216 11017 309 147 11000 [download] The code block you give `grep` gets called once for every array element with `$_` as an alias for the element. In this case the elements are references to the anonymous arrays, which can be accessed by using the dereferencing operator `->` like shown above. PS: In Perl a two dimensional array, aka an array of arrays aka a matrix, uses always anonymous arrays inside. So you don't need to mentioned that explicitly with "an array of anonymous arrays".	[reply] [d/l] [select]
Re^2: manipulating an array of anonymous arrays ... by chexmix (Hermit) on Apr 25, 2008 at 16:26 UTC
Thank you. I am going to try something like this with the modification that I am seeking to "toss out" the results that fell betweentwo values. I am not sure how to modify that use of grep but will have fun experimenting with it. I will try to have fun with it anyway. Working on this has involved a lot of encounters between me and my limitations (current knowledge, learning speed, and so on pretty much ad infinitum). I get lost sometimes between feeling humble ("I have so much to learn") and feeling like a deer in headlights ("I will never ever get it, my chops are those of a 3 year old and I will eventually be working at a fast food chain"). A constant diet of crow gets old, especially if like me you are vegetarian (the veggie crowcakes aren't very good, yet). :)	[reply]
Re^3: manipulating an array of anonymous arrays ... by mscharrer (Hermit) on Apr 25, 2008 at 16:33 UTC
`grep` returns all element for which the code block returns boolean true, so you have to change the code block to return true if the results are not between two values: `my @array = grep { $_->[0] <= $low \|\| $_->[0] >= $high } @array;` [download]	[reply] [d/l] [select]
Re^4: manipulating an array of anonymous arrays ... by chexmix (Hermit) on Apr 25, 2008 at 16:39 UTC
Re: manipulating an array of anonymous arrays ... by Fletch (Bishop) on Apr 25, 2008 at 14:08 UTC
Perhaps rather than attempting to modify the incoming data in place it'd make more sense to create a new, filtered version which doesn't contain the entries you've already processed. Copy the misses to your output copy, while recording the hits in another data structure. Then you can iterate over that filtered version of the information again (using the same sub); lather, rinse, repeat. The cake is a lie. The cake is a lie. The cake is a lie.	[reply]


Think about Loose Coupling
	PerlMonks