shaolin_gungfu has asked for the wisdom of the Perl Monks concerning the following question:

Hi fellow Monks,

Before I say anything I should say i'm pretty new to Perl, so sorry if this is a stupid question:-)

I wrote a message a little earlier about writing a normal perl array to disk, in the same way as the old dbmhash thing did for hashes. I got messages back telling me about tie() and storage.pm and i've looked at these all afternoon, but even after reading loadsa man pages and tutorials I just can't get anything to work. I'm using Activestate perl and it just complains about wrong versions when I try using storage.pm and I don't know anything about creating packages for the tie() thing.

I'm developing a web spider, at the moment my program keeps the list of URLs to visit in an array. I would simply like to store these values on disk so if my spider crashes I don't loose my list of URLs to visit, and I can carry on where my robot left off.

Any idea what the best way of doing this is? I could just keeping opening and closing file handles and writing to a normal text file, but that seems a bit slow and dodgy to me. I thought about using the dbmhash thing, but I know that's deprecated. Anyway, any code examples would be really appreciated.

Many thanks once again, Tom

Replies are listed 'Best First'.
Re: Writing Arrays to disk
by samtregar (Abbot) on Apr 16, 2002 at 23:26 UTC
    Check out the Storable module. You can do something like: 
      use Storable;

  # store the links array in links.db
  store(\@links, "links.db");

  # load the links array from links.db
  @links = @{ retieve("links.db") };

    [download]

    -sam

[reply]
[d/l]

      yeah thanks i've tried the Storable thing, it sounds like just what I need but I just get the message:

      Storable object version 1.012 does not match $Storable::VERSION 1.014 at C:/Perl/lib/DynaLoader.pm line 225.
      Compilation failed in require at storable line 1.
      BEGIN failed--compilation aborted at storable.pl line 1.

      I'm using Activestate Perl and their ppm thing isn't working at the moment so I can't even try reinstalling the module:-( ah well maybe i'll just keep on opening and closing the filehandles in a textfile instead, thanks Tom

[reply]
Re: Writing Arrays to disk
by rbc (Curate) on Apr 16, 2002 at 23:07 UTC
    A simple approach might look like this ... 
    my @arr = ("http://blahblahblah", "whatever blah", "something else"); 
open( FH, ">SAVEME" ) or die "oops! could not open it\n";
for my $url (@aar) {
    print FH, "$url\n";
}
close FH;

    [download]
    $ cat SAVEME
    http://blahblahblah
    whatever blah
    something else
[reply]
[d/l]
Re: Writing Arrays to disk
by DaWolf (Curate) on Apr 17, 2002 at 01:59 UTC
    Well, I'm not sure if I'm gonna sound obvious, but here it goes...

    To me the simplest solution is to do this: 
    open (FOO, ">>foo.txt");
foreach $element(@element)
{
    print FOO "$element\n";
}
close (FOO);

    [download]
    I'm not sure about performance issues on implementing this solution, but I'm sure that the others can clarify this.

    And, of course, it won't hurt you to test it :-)

    Hope it helps you.

    Best regards,

    Er Galvão Abbott
     a.k.a. Lobo, DaWolf
    Webdeveloper
[reply]
[d/l]
Re: Writing Arrays to disk
by JayBonci (Curate) on Apr 17, 2002 at 16:03 UTC
    If you're having trouble with Storable, also check out Data::Dumper. It tends to make things that are bigger than Storable would, but it's more useful for debugging things, as items end up in plain text (and all you need to do to restore the complex symbol is to eval them).

        --jb
[reply]

Back to Seekers of Perl Wisdom