clintp has asked for the wisdom of the Perl Monks concerning the following question:

Traverse this.

I've been hacking on this problem on-and-off for almost a week now and only succeeded in giving myself a headache. It's time to try the "infinite monkeys" approach to this problem and see if anyone gets lucky.

The code below reproduces the structure I've got. (Yes, it's self-referential. Don't lecture me on memory leaks, please.) The "rules" are noted in the comments below. I hope there's no typos.

Good luck

#!/usr/bin/perl -w use strict; my $rdb={ 'table a' => { 'aa' => { data => { key1=>'val', key3=>'val',key2=>'val', }, parent => '', children => { 'table b' => { 'ab' => "TBD", }, 'table c' => { 'ac' => "TBD", }, } }, 'ba' => { data => { key1=>'val', key3=>'val',key2=>'val', }, parent => '', children => { 'table b' => { 'bb' => "TBD", }, 'table c' => { 'bc' => "TBD", }, } }, }, 'table b' => { 'ab' => { data => { nonsense => 'foo' }, parent => 'aa', children => {}, }, 'bb' => { data => { morenonsense => 'foo' }, parent => 'bb', children => {}, }, }, 'table c' => { 'ac' => { data => { keyi => 'val' }, parent => 'aa', children => { 'table d' => { 'ae' => "TBD", 'af' => "TBD", }, 'table e' => { 'ag' => "TBD", }, }, }, 'bc' => { data => { keyi => 'val' }, parent => 'bb', children => { 'table d' => { 'be' => "TBD", 'bf' => "TBD", }, 'table e' => { 'bg' => "TBD", 'bh' => "TBD", }, }, }, }, 'table d' => { 'ae' => { data => { k => 'v', k2 => 'v', k3 => 'v' }, parent => 'ac', children => {}, }, 'af' => { data => { k => 'x', k2 => 'x', k3 => 'x' }, parent => 'ac', children => {}, }, 'be' => { data => { k => 'v', k2 => 'v', k3 => 'v' }, parent => 'bc', children => {}, }, 'bf' => { data => { k => 'x', k2 => 'x', k3 => 'x' }, parent => 'bc', children => {}, }, }, 'table e' => { 'ag' => { data => { f => 'b', b => 'k' }, parent => 'ac', children => {}, }, 'bg' => { data => { f => 'b', b => 'k' }, parent => 'bc', children => {}, }, 'bh' => { data => { f => 'b', b => 'k' }, parent => 'bc', children => {}, }, } }; # Fixes the "TBD" above $rdb->{'table c'}->{ac}->{children}->{'table d'}->{ae}= $rdb->{'table d'}->{ae}; $rdb->{'table c'}->{ac}->{children}->{'table d'}->{af}= $rdb->{'table d'}->{af}; $rdb->{'table c'}->{ac}->{children}->{'table e'}->{ag}= $rdb->{'table e'}->{ag}; $rdb->{'table a'}->{aa}->{children}->{'table b'}->{ab}= $rdb->{'table b'}->{ab}; $rdb->{'table a'}->{aa}->{children}->{'table c'}->{ac}= $rdb->{'table c'}->{ac}; $rdb->{'table c'}->{bc}->{children}->{'table d'}->{be}= $rdb->{'table d'}->{be}; $rdb->{'table c'}->{bc}->{children}->{'table d'}->{bf}= $rdb->{'table d'}->{bf}; $rdb->{'table c'}->{bc}->{children}->{'table e'}->{bg}= $rdb->{'table e'}->{bg}; $rdb->{'table c'}->{bc}->{children}->{'table e'}->{bh}= $rdb->{'table e'}->{bh}; $rdb->{'table a'}->{ba}->{children}->{'table b'}->{bb}= $rdb->{'table b'}->{bb}; $rdb->{'table a'}->{ba}->{children}->{'table c'}->{bc}= $rdb->{'table c'}->{bc}; # Given $rdb and the key "table a", traverse the structures # so that you produce the records of: # # aa ab ac ag ae # aa ab ac ag af # ba bb bc bg be # ba bb bc bg bf # ba bb bc bh be # ba bb bc bh bf # # (These can be arrays of arrays or arrays of hashes # or arrays of strings or whatever.... # but any column MUST only contain things of the same kind) # # Note that: # * each a record from each table is represented in # each "record" produced # * multiple records within a table are noted once for # each permutation # (ae and af above) # * subsequent table names are not known: you're given # only the first one. # * the keys within the hashes are guaranteed to be # unique. # * the data is not important. # * the keys (aa, ab, bb) follow no set pattern and are # strictly coincidental just in this example. # * each record will join with the proper number of # tables. # * be prepared to have "table a" be the one and only # table (boundaries!) # # NO HARD WIRING, except for the initial table name. # # [I think I keyed these tables in right.]

Replies are listed 'Best First'.
Re: So you think you're good with structures?
by japhy (Canon) on Aug 03, 2001 at 22:41 UTC
    The first thing I see is the wasted use of nested references for the children. It'd be far easier to just keep an array reference:
    my $rdb={ 'table a' => { 'aa' => { children => { 'table b' => [ 'ab' ], 'table c' => [ 'ac' ], } }, }, };
    Like so. I think that will make the general approach far easier.

    _____________________________________________________
    Jeff japhy Pinyan: Perl, regex, and perl hacker.
    s++=END;++y(;-P)}y js++=;shajsj<++y(p-q)}?print:??;

      I think one of the original incarnations of the structure looked like this. It doesn't change the approach at all. If you want just the keys, the use: keys %{....} instead of @{...} when you get down there. The references are there just because they were there when it was built, you don't have to use them.

      Obviously, the structure wasn't BUILT with code like this. This was taken out of a 600-line program that did a lot of other things.

Q: So you think you're good with structures? A: Umm... Yes!
by dragonchild (Archbishop) on Aug 04, 2001 at 00:33 UTC
    Ok. After annoyingly hard work, I've got the following code that works! Now, obviously, it works on the data structure you've given and returns the output you desired. I make no anythings when it comes to any other data structures.

    my @tables_seen; sub traverse_nodes { my ($db, $start, $record) = @_; push @tables_seen, $start; my @records = ("$record"); CHILD: foreach my child (keys %{$db->{$start}{$record}{children}}) { next CHILD if grep /$child/, @tables_seen; my @child_values; foreach my $key (keys %{$db->{$start}{$record}{children}{$chil +d}}) { push @child_values, traverse_nodes($db, $child, $key); } my @temp = @records; @records = (); foreach my $rec (@temp) { push @records, map { "$rec $_" } @child_values; } } return @records; } my $starter = 'table a'; my @good_records; foreach my $record (keys %{$rdb->{$starter}}) { @tables_seen = (); push @good_records, map { join "", $_ } traverse_nodes($rdb, $starter, $record); } print "REC: $_\n" foreach @good_records;

    ------
    /me is the brightest bulb in the chandelier!

      Wonderful. This actually seems to work with a couple of sample datasets that I had laying around. I'll try it with real data from the application on Monday (I'm home now and all of that stuff's at work). A few minor modifications and I can put it in production code.

      I don't see anything wrong with it. (And I see one of my mistakes, *DOH*. I hate mental blocks.) I'll let you know how it works on Monday. Thanks!

      PS: does anyone have a need for an ad-hoc report writer? Outputs in Excel, PDF, text, HTML given some XML input and massaging... :)

        Yes! I could definitely use an ad-hoc report writer!!! Email it to dragonchild93@yahoo.com as soon as you can. :)

        That'll be your payment. *grins*

        ------
        /me wants to be the brightest bulb in the chandelier!

      Okay, I found a structure that doesn't get processed by the code well. On inspection, it looks like it should though. It follows shortly...It's just a Data::Dumper dump, so it might be a bit hard to read.
        Ooops. In the recursive function, change the last few lines to:
        pop @tables_seens; return @records; }
        This allows you to remove the @tables_seens = () line in the control section.

        I am sorry about forgetting that. I thought about doing a pop, but the dataset you gave at first didn't find that bug. *blushes*

        ------
        /me wants to be the brightest bulb in the chandelier!

Re: So you think you're good with structures?
by clintp (Curate) on Aug 03, 2001 at 22:32 UTC
    A small ASCII art picture that helps clarify how the output was arrived at given the structure and how the tables relate in this particular example.
    # # This is how the tables relate to each other, in this # xample. The records for each table are shown in parens. # # _____ table b (ab) # / # table a (aa) # \______ table c (ac) # | \____table d (ae, af) # | # |_____table e (ag) # # So traversing this tree once yeilds aa, ab, ac, ae, ag. # going down it again (because table d has two records) # gives ab, aa, ac, af, ag. #
Re: So you think you're good with structures?
by dragonchild (Archbishop) on Aug 03, 2001 at 22:47 UTC
    Very frankly, it sounds like you're trying to do the work of a relational database. Smart people have already invented perfectly good wheels to do this stuff.

    I tried working on it and your problem statement wasn't very clear. Would you try explaining how you got that 6x5 matrix up above?

    ------
    /me wants to be the brightest bulb in the chandelier!

      Very frankly, it sounds like you're trying to do the work of a relational database
      That's EXACTLY what I'm trying to do (to a 15-year-old DOS program) but I didn't want to distract from the problem at hand. I don't have an xy problem, I need to act the part of an RDB.

      The reason isn't important, can you traverse the structure or no?

      Did the ASCII art help? It's important to grok the data before attempting the solution.

        I think it's more important that you defined the structure you want to traverse versus design the traversal for a given structure. Most people don't realize this, but the data structure you design will make or break your sanity. It's not as easy as saying "Can you traverse this or not?"

        If I was going to design a data structure to traverse, I would do things completely differently. However, I suspect that traversal isn't the only thing you're planning on doing.

        In addition, you still haven't fully laid out the rules for the traversal. Here are the rules I've seen you lay out. You tell me if they make sense:

        1. Take a key from the table you're in and append it to the list.
        2. If there are no children, go to 1.
        3. Grab the list of child records.
        4. Append each child record to the list.
        5. For each child record, go to 1.
        Am I missing anything?

        ------
        /me wants to be the brightest bulb in the chandelier!

        The weird thing is the elements in the traversal:
        A / \ B C / \ D E
        This would yield "ABCD" and "ABCE", according to you. That seems odd to me -- are you saying you include all the child nodes of the previous node until the LAST ROW, where you just choose one?

        _____________________________________________________
        Jeff japhy Pinyan: Perl, regex, and perl hacker.
        s++=END;++y(;-P)}y js++=;shajsj<++y(p-q)}?print:??;

Re: So you think you're good with structures?
by suaveant (Parson) on Aug 03, 2001 at 22:19 UTC
    Ummm... what is your problem? I ran it and nothing happened, but then... I don't know what was supposed to happen.

                    - Ant
                    - Some of my best work - Fish Dinner

      The bug is, there's no code to do the traversal and produce the output. I've got 5 different versions of a subroutine to try and they all fail in some small, subtle way. This may not be as easy as first appears.