frasco has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks, some times ago I asked for Monks wisdom in order to find a convenient way to fetch data from a DB table. I had the following table structure holding words of a splitted text file:

|id   |line_n   |word          |word_position
--------------------------------------------
|1    |a1       |Lorem         | 1
|2    |a1       |ipsum         | 2
|3    |a1       |dolor         | 3
|4    |a1       |sit           | 4
|5    |a1       |amet,         | 5
|6    |b2       |consectetuer  | 1
|7    |b2       |adipiscing    | 2
|8    |b2       |elit.         | 3
|9    |c3       |Phasellus     | 1
|10   |c3       |non           | 2
|11   |c3       |erat          | 3
|12   |c3       |...           | 4

By retrieving data from db I needed that my output would be the same as follow (without GROUP_CONCT and similar database functions):

line 1    Lorem ipsum dolor sit amet,
line 2    consectetuer adipiscing elit.
line 3    Phasellus non erat ... 

That is the original layout of my text file.

A wise monk suggested this code to print the output:

my $sql = qq(select * from table order by id); my $data_col = $dbh->selectcol_arrayref($sql, { Columns => [ 2, 3 ] }) +; my %seen; my @list = @$data_col; while (@list){ my $line_n = shift @list; my $word = shift @list; if ($seen{$line_n}++) { print " "; } else { print "\n$line_n - "; } print $word; }

Of course, It was ok until I had to immediately print the output, but I cannot figure out how to store retrieved data in a complex data structure for further use (i.e. to use such output as json, etc). For instance:

my $text = [ "myText_01", [ a1, ["Lorem", "ipsum", "dolor", "sit", "amet,"] ], [ b2, ["consectetuer", "adipiscing", "elit."] ], [ c3, ["Phasellus", "non", "erat"] ], [ 4, ["...", "..."] ] ];
thanks, Francesco

Replies are listed 'Best First'.
Re: Fetching data from DB and complex data structures
by choroba (Cardinal) on Dec 14, 2015 at 10:32 UTC
    Just build the structure instead of printing the values obtained from the database. I used SQLite to test:

    Now the real work:

    my $select = $db->prepare('SELECT line_n, word, word_position FROM tex +t'); $select->execute; my $text; while (my ($line, $word, $pos) = $select->fetchrow_array) { $text->{$line}[$pos - 1] = $word; } print Dumper($text);
    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

      Thank you very much.

      Just a question. Since I'm pretty new to Perl I cannot understand that "strange construct" in the while loop:

      $text->{$line}[$pos - 1] = $word;

      Please, can you give some more details on it?

      Then here come the second question/problem. As far I can see keys are the original line numbers and values are references to arrays containing the words of each line. In that case, if I'm not in wrong, keys/lines do not respect the original order of my text. Is it true?

        $text->{$line}[$pos - 1] = $word; ^ ^ ^ | | | | hash | dereference array

        I used $pos - 1 as positions start from 1, but array indices start from 0.

        Note that the bracket types are the same as when creating the data:

        $VAR1 = { 'c3' => [ 'Phasellus', 'non', 'erat' ], 'b2' => [ 'consectetuer', 'adipiscing', 'elit.' ], 'a1' => [ 'Lorem', 'ipsum', 'dolor', 'sit', 'amet,' ] };
        ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
Re: Fetching data from DB and complex data structures
by chacham (Prior) on Dec 14, 2015 at 15:33 UTC

    without GROUP_CONCT and similar database functions)

    Do you mean without a special group concatenation function(), or without any functions at all? This can be easily done with a recursive query.

    WITH Builder(line_n, word, word_position, amount) AS ( SELECT line_n, '', 0, MAX(word_position) FROM table GROUP BY line_n UNION ALL SELECT Data.line_n, CONCAT(Builder.word, CONCAT(' ', Data.word)), Data.word_position, Builder.amount FROM Builder, table WHERE table.line_n = table.line_n AND table.word_position = table.word_position + 1 ) SELECT line_n, word FROM Builder WHERE word_position = amount;

    The double CONCAT() was used to avoid casting issues; you may be able to remove them. Depending on your RDBMS, you may need to specify the RECURSIVE keyword.

    Also, if you don't mind, a side point, please do not use * outside of COUNT(), EXISTS(), and ad hoc queries. Specifying the columns--id, line_n, word, word_position--is easy, self-documenting, and protects against column changes and reordering.

    Perhaps it is worth mentioning, the id column is redundant, as line_n and word_position must be unique anyway.

      Thank you for you help, but in fact I would like to avoid SQL functions at all. For instance, GROUP_CONCAT or string_agg would be great. I need to manage these data before their way to the HTML template. Moreover, just to clarify my question, neither line_n nor word_position are unique values.

        If line_n and word_position (together) are not unique, how in the world do you build the string? Well, unless id breaks the ties.

        If you want to see the work in progress, the recursive query is even easier. (No MAX in the anchor, and no WHERE clause is the main query.)