cmv has asked for the wisdom of the Perl Monks concerning the following question:

Perl Monks-

Reading How do I save a nested structure to a file (and read it later)? gives me several different ways to save a data structure to a file. I'm hoping for some advice on which might be the best to use for my particular application.

I have a perl script that runs commands on remote hosts. I want the user to generate their own list of login@hostnames so the GUI can provide them with a drop-down list to select from every time they run the script. This implies I need to save the data to a file.

Some users will just use the GUI menus to build their list, but I imagine others may want to simply edit the file directly, since it will be faster for bulk additions.

Assuming the save file is in human readable format (say I use Data::Dumper), how do I protect against clever little users who may assume I just eval the whole file in (I'm assuming I need to worry about this, don't I)? However, if I use Storable or FreezeThaw, they will loose the ability to edit the file directly (is that really true?). In this case I could always write a seperate bulk-transfer utility, but that seems awkward.

Any advice is greatly appreciated!

Thanks

-Craig

Replies are listed 'Best First'.
Re: Restoring a Data Structure from a File
by Fletch (Bishop) on Nov 02, 2007 at 13:59 UTC
[reply]
      fletch-

      GREAT idea! Many thanks. I knew someone here would shine a light down one or two of the dark alleys that I never bother to turn the lights on for.

      BTW: What is a hyoomon (and how do you pronounce it)? All my searches turn up folks using it, but no one with a definition.

      Never heard that one before...

      -Craig

      UPDATE:
      XML::Simple doesn't work because I can't use @ in my hash key names. As stated in the docs: hash keys from the data structure will be encoded as either XML element names or attribute names. Therefore, you should use hash key names which conform to the relatively strict XML naming rules..."

      YAML::Syck Seems to work fine.

      PS - Thanks for the help with huh-yoo-man too. I love it!

[reply]
Re: Restoring a Data Structure from a File
by KurtSchwind (Chaplain) on Nov 02, 2007 at 14:35 UTC
    Considering your specific example, I'd probably steer clear of XML. It's way over-kill for what you are looking for. There is no reason to not do something as simple as a text file where you put one address per line.
    Joe@Host1.fqdn
Schmoe@Host2.fqdn

    [download]
    and simply read the file in if it's there. It's super easy to edit in any given editor. And it's trivial to read in as well. You can loop through and build the array in what... 3 lines? And dumping it out is just as easy.
    Don't over-engineer this. And yes, you are wise to not just blindly eval a document into the program. It sounds like a good app.
    --
    I used to drive a Heisenbergmobile, but everyone I looked at the speedometer, I got lost.
[reply]
[d/l]
[select]
      Kurt-

      You did good in pointing out the KISS solution here, as I failed to properly state the scope of the problem in my original request.

      Actually, this is how I started doing things, but ended up with data-feature-creep, as I kept adding bits of information about each usr@host. Things now look something like this: 

      'me@hawaii' => {
    'SSHinfo' => {
        'Login' => 'myusr@warm.hawaii.com',
        'socks_proxy' => 'mysocksproxy.com',
        'socks_port' => 1234,
        'socks_protocol' => 4,
        'identity_files' => [ "$HOME/.sshspecial/toolauth" ],
        'user_known_hosts' => "$HOME/.sshspecial/knownhosts",
    },
    ...
},
...

      [download]
      Having all this data tied to each entry, it'll be better for me to save the structure.

      -Craig

[reply]
[d/l]
        I see what you mean by a structure now. At this point, you could consider XML, but I'd probably stick with something like CSV.
        As someone else mentioned, it's generally a good idea to store data format versioning in the file as well. If you go XML, it can be in a version= attribute. If you go CSV, make the first line have the col header information.
        --
        I used to drive a Heisenbergmobile, but everyone I looked at the speedometer, I got lost.
[reply]
Re: Restoring a Data Structure from a File
by NetWallah (Canon) on Nov 02, 2007 at 16:16 UTC
    Having been down this road before, I'd advise you to save, and verify data VERSION information.

    In other words, make sure the VERSION of the data, and the program structure the data imports into are compatible.

    This is critical to feature-creep applications. Also have suitable defaults for each feature you add, so older stored data can remain compatible.

         "As you get older three things happen. The first is your memory goes, and I can't remember the other two... " - Sir Norman Wisdom

[reply]
      NetWallah This is an excellent suggestion, which I will be adding to my code as soon as possible. I've already been tripped up with data version problems in some other code I've written, but never thought to do anything about it. Many thanks for the suggestion!

      -Craig

[reply]
Re: Restoring a Data Structure from a File
by dragonchild (Archbishop) on Nov 02, 2007 at 16:25 UTC
    Unless the data needs to be human-readable, why not just use DBM::Deep? That allows you to treat your serialization directly as a Perl data structure. And, for bulk-additions, just whip together a little script that adds the stuff to the Perl data structure that is your file.
    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
[reply]
Re: Restoring a Data Structure from a File
by holli (Abbot) on Nov 02, 2007 at 22:00 UTC
    Since this sounds to a ajaxian thingy i recommend JSON. It plays nice with Javascript, is easy to note and familiar for you (just replace => by : and you're mostly done). As for YAML, it is error prone for manual editing. eg, a whitespace/newline (or none) at the wrong place may lead to fatal parsing errors.


    holli, /regexed monk/
[reply]
[d/l]
      holli I was glad to get your suggestion to use JSON. As noted above, XML doesn't really work because of the restrictions on the characters in the hash keys (at sign is not allowed), and YAML works fine, but has a complicated copyright that I'm not sure I can live with.

      I'll take a look at JSON next week and see how that fares.

      Thanks

      -Craig

[reply]

        But YAML is JSON.

        And YAML::Syck and libsyck are under your usual open source MIT and BSD licensing respectively. If you're referring to the "D&R" license in the latter's COPYING file that's just why being his normal surrealist self. Syck is included in the core Ruby distribution so there's probably very little need for concern.

        We're looking for a Perl and Database Developer for Corporate Investments Group.

[reply]

Back to Seekers of Perl Wisdom