Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

(Ovid) Re: Confused about complex data structures.

by Ovid (Cardinal)
on Jan 18, 2001 at 05:14 UTC ( [id://52661]=note: print w/replies, xml ) Need Help??


in reply to Confused about complex data structures.

What you are doing is sound. To test this, I created two files called cp_123456 and cp_123457. I then modified your snippet to use my files and I use Data::Dumper to view the resulting data structure:
use strict; use warnings; use Data::Dumper; my %city_data = (); foreach my $filename ( 'cp_123456', 'cp_123457') { my ( $city, $id ) = split /_/, $filename; open THIS, "<$filename" or die "Can't open $filename for reading: +$!"; chomp ( my @tmp = <THIS> ); close THIS; push @{ $city_data{$city}{$id} }, @tmp; } print Dumper( \%city_data );
The output was the following:
$VAR1 = { 'cp' => { '123456' => [ 'this is some data', 'this is a test' ], '123457' => [ 'this is the second file', 'this is another line', 'this is the third line' ] } };
Whenever I want to create a complex data structure, I have to decide what's the best way to get at the data. One important issue is to avoid iterating, if possible. As these structures get larger, iteration can kill your performance.

After I have the basic idea of the structure laid out, I use Data::Dumper to output a subset of the structure so I can see that the results are what I expect. Using the debugger and entering the command x \%city_data has essentially the same effect.

However, I find that complex data structures in Perl are, for me, similar to regexes in that at times I tend to use them innapropriately. Often, as the amount or complexity of the data increases, a complex data structure can become unmanageable. Using a database to handle that data can solve some serious headaches if you're concerned about scalability issues.

Another issue to ask here is, what do you do if you wind up with two files with the same name? If someone accidentally copies a file to another folder that you also happen to read from, do you have duplicate data? You may wish to test for this.

Also, is there any data validation? I realize that you just posted a snippet, but don't forget to test for this. Plus, if there is any chance that another process will be accessing the files while you are reading them, consider using flock.

Cheers,
Ovid

Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

Replies are listed 'Best First'.
Personal preference
by flocto (Pilgrim) on Jan 18, 2001 at 06:58 UTC
    I fully agree with what you wrote above. I just prefere to have references for my whole complex data. So instead of using a named hash like %data; i prefere using $data = {};. Maybe it's because I dislike using stuff like ${ $data{$foo} }. $data->{$foo} just looks better and I am aware of what I am doing. But that's pretty much personal preference..
Re: (Ovid) Re: Confused about complex data structures.
by marius (Hermit) on Jan 18, 2001 at 21:22 UTC
    Favorite thing #101 about our little Monastery (wow, Monastery is a really odd looking word now that I've typed it a few times.. ah, you're still reading, moving along..) is finding new and neat uses of modules. I've seen Data::Dumper used before in stuff, but never to visualize !(Arrays)/(Hashes)/$1 of $1/$2 of $2/you get the idea!. Now that's cool!

    -marius
Re: (Ovid) Re: Confused about complex data structures.
by r.joseph (Hermit) on Jan 18, 2001 at 05:24 UTC
    I can't thank you enough - this is exactly what I was looking for. Thanks everyone for your help and the quick response - oh, and don't be discouraged to continue posting, I am still fuzzy in this area and will always welcome more help!

    R.Joseph

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://52661]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (4)
As of 2024-03-28 17:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found