Flat File Fun (or You Just Saved My Class)

abitkin has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks,
First, I must ask your forgiveness for not having code, but this project is still in the development stage.

I'm going to have a set of objects set up like:

/-------\
|Booking|
\-------/
    |    /----\
    |--->|User|
    |    \----/
    |     /--------\
    ----->|Schedule|
          \--------/
              |
/-----\       |
|Route|<-------
\-----/
   |    /------\
   ---->|Driver|
        \------/

This is by no means the complete diagram, only a representation of the major objects. Also each object will have extra data in it, as you would expect with any object. I must be able to save the whole thing into a text file. (flat file) What is the best way to store the entire structure?

Please Note: There can be multiple high level objects attached to lower objects. (For example, there can be more than one booking that can use the same user or schedule. Likewise there can be more than one schedule using the same route, and so on.) Also, I'm asking this before I've written any object code, so that I can look at, and try to decide which way will suit this project the best. Finally, because I have no control over the target system, no modules outside of the standard set with Perl 5.8 can be used.
==
Kwyjibo. A big, dumb, balding North American ape. With no chin.

Comment on Flat File Fun (or You Just Saved My Class)

Replies are listed 'Best First'.
Re: Flat File Fun (or You Just Saved My Class) by Abigail-II (Bishop) on Jul 16, 2003 at 14:48 UTC
This is not an easy question to answer. No doubt, you'll see quick posts saying "use Date::Dumper", "use YAML" or some other serialization technique. But then what. So you decide the pick YAML, what's next? That's the wrong time to start asking questions. First you have to know, what is that data I need to save? Why am I saving? Is it for persistence, that is, do I have to be able to reconstruct the object? It might be that all you need to save is everything reachable from the object ref. But what about class data? Is the class keeping track of the objects? Perhaps it's keeping count of how many objects are out there - perhaps you use fly-weight objects or Inside Out Objects. Then there will be (possible lexical) data in the classes that need to be saved too. So you first have to determine what you need to save to be able to reconstruct. Determine what kind of relations there are between the pieces of data that need extra attention. For instance, the class puts the various objects of that class in a hash, using the objects as keys. Which means, the keys are the stringified forms of the references. On restoring, you most likely get different addresses for your references, and hence the stringified form is likely to be different. Then your saved class data is useless, and you have a major data corruption. Having decided what to store, and what meta data to store, then you can focus your attention which technique to use. Not all techniques are perfect. Code references are hard to serialize. Some techniques turn strings into numbers on certain conditions, which can lead to data corruption again. (Numifying and then stringifying "5.000" gives you "5", not "5.000"). So, what the "best way" can't be determined until you have implemented more of it. In fact even the question "what is a correct way" can't be answered yet. Abigail	[reply]
Re: Re: Flat File Fun (or You Just Saved My Class) by abitkin (Monk) on Jul 16, 2003 at 15:09 UTC
Okay, so here's how the data breaks down: The idea is the project will be a rideshare program. The idea being that each one of these objects shown will be shown to the user as a unique ID (so that they can do specific actvities.) Each portion of the data structure is built by the user and it needs to persist between runs so that it can continue to be modified. For the scope of this project, I don't need to worry about more than 1 user accessing the program at a time. One posibility is to just add the unique IDs that I'm giving to the user to the actual objects and ignore inheritance all together; however, doing that would make the implimentation of this system much more difficult. It is true that certian "modes" of this program will not require the whole structure to be rebuilt; however, these repesent the minority of all cases. == Kwyjibo. A big, dumb, balding North American ape. With no chin.	[reply]
Re: Flat File Fun (or You Just Saved My Class) by Abigail-II (Bishop) on Jul 16, 2003 at 15:24 UTC
Given my previous post, you aren't thinking that I now tell you what to do, just based on two small paragraphs of description, one of which is just exploring a possibility? Make an implementation, then we'll see. Abigail	[reply]
Re: Re: Flat File Fun (or You Just Saved My Class) by waswas-fng (Curate) on Jul 16, 2003 at 17:05 UTC
Re: Flat File Fun (or You Just Saved My Class) by Abigail-II (Bishop) on Jul 16, 2003 at 20:01 UTC
Some notes below your chosen depth have not been shown here
Re: Flat File Fun (or You Just Saved My Class) by talexb (Chancellor) on Jul 16, 2003 at 14:31 UTC
I would head over to CPAN and look at Freeze::Thaw, YAML and DSM::Storable for starters. There are lots of modules that let you store/retrieve objects to/from text files. --t. alex Life is short: get busy!	[reply]
Re: Flat File Fun (or You Just Saved My Class) by waswas-fng (Curate) on Jul 16, 2003 at 14:30 UTC
You can use Data::Dumper to do it. the output would depend on what the actual data structure(s) look like. then you can eval to bring back in. -Waswas	[reply]
Re: Re: Flat File Fun (or You Just Saved My Class) by abitkin (Monk) on Jul 16, 2003 at 14:32 UTC
How well would that work with references to the parent objects? == Kwyjibo. A big, dumb, balding North American ape. With no chin.	[reply]
Re: Flat File Fun (or You Just Saved My Class) by l2kashe (Deacon) on Jul 16, 2003 at 14:33 UTC
Modules that will serialize data off the top of my head, Data::Dumper, Storable, FreezeThaw (I think that last is right). There is also Data::Serializer which is a higher level module that can use one of those other modules for storage, as well as providing for compression, and encryption if need be. MMMMM... Chocolaty Perl Goodness.....	[reply]
Re: Flat File Fun (or You Just Saved My Class) by wufnik (Friar) on Jul 16, 2003 at 16:22 UTC
greetz; i have previously had problems with data::dumper for large structs, and have had none since using XML::Dumper, which produces compressed xml descriptions of your data struct. typical usage is something like: `use XML::Dumper; # deserialize my $coffin = "resurrectme.xml.gz" my $xdumper = new XML::Dumper; my $lazarus = $xdumper->xml2pl($xml_gz); # ... $lazarus modifications sit somewhere here # serialize $xdumper->pl2xml($lazarus, $coffin);` [download] i have found this to be a practical (both space efficient and reliable) way to serialize many structs. i have used the admirable xml::twig to operate on the xml generated by xml dumper. obwarning: there may be issues with the xml generated in terms of well-formedness. i think diotalevi did not like this solution, and i am sure he had good reason, though, ermm, i'm not quite sure what it was: my forgetfulness. hope that helps. wufnik -- in the world of the mules there are no rules --	[reply] [d/l]


Clear questions and runnable code get the best and fastest answer
	PerlMonks