I am running a query on a database which I need to translate to a fixed format as follows:

select m, p from various_joined_tables_and_views where u = N group by 
+m, p, u
</code.

<p>The result looks like this:</p>

<code>
m1, p1
m1, p2
m1, p3
m2, p1
m2, p2
m3, p1
.
.
.
alot more

[download]

so basically for every mN there are multiple pMs.

I can change the SQL but the final data needs to be in this format:

[[m1, [p1,p2,p3], [m2, [p1,p2]], [m1, [p1]]]

[download]
The final perl data is passed to JSON::XS and written to a file. I'd dearly like to generate this structure from the SQL directly without any perl processing so I could fetchall_arrayref the result and pass it to JSON::XS but I don't think this is possible.

So far I've come up with:

my @data;
my %mkts;
my ($m, $p);
$cursor->bind_col(1, \$m);
# this is in the hope that DBD::Oracle will actually implement
# bind types in the future -
# see http://rt.cpan.org/Public/Bug/Display.html?id=49818
# because I'd really like this column to be scalar that looks like an 
+number
# rather than a string - as JSON::XS will write it as N instead of 'N'
$cursor->bind_col(2, \$p, {type => SQL_INTEGER});
while ($cursor->fetch) {
  if (exists($mkts{$m})) {
    foreach (@data) {
      if ($_->[0] == $m) {
        push @{$_->[1]}, $p + 0; # make it look like a integer - see a
+bove
        last;
      }
    }
  } else {
    $mkts{$m} = 1;
    my $n = @data;
    $data[$n][0] = $m;
    $data[$n][1] = [$p] + 0;
  }
}

[download]

Any ideas how to improve on this?

update I should have said there could be as many as 4000 m's and 600 p's.

In reply to Help with more efficient data translation from DB query by mje

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.