Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Does Perl have the capablity to easily create database files with fields in records. I know files can be created with Perl that read, write and append. I know that databases in formats of SQL and INFORMIX etc can be read, but if I want to create my own database and access the fields or records how do I do it? I hope Perl can do this. If so then I will try to learn Perl? Help!
  • Comment on HOW TO CREATE A DATABASE FILE WITH FIELDS IN RECORDS

Replies are listed 'Best First'.
Learning Databases with Perl
by footpad (Abbot) on Dec 27, 2000 at 20:22 UTC

    As others have noted, Perl can easily handle databases of various flavors.

    I should point out that that SQL (Structured Query Language) is not a database format, per se, but a standard for accessing and manipulating data. Time spent learning SQL will serve you well with other tools, not just Perl.

    You may wish to consider obtaining a copy of Programming the Perl DBI, by Alligator Descartes & Tim Bunce (Feb 2000, O'Reilly and Associates, ISBN: 1-56592-699-4, $34.95/US), also known as the "Cheetah book." It provides a good introduction to the DBI module, an interface used to connect to a variety of backend databases and formats via Perl.

    This book provides a nice introduction to general database concepts, gives a reasonable introduction to SQL and the statements you'll use to maintain your databases, and provides several useful examples. It also provides a rather nice comparison of the features suported by various DBI drivers, ranging from CSV (comma-separated text files) to industry giants, such as Informix, Oracl, InterBase, Sybase, and so on.

    Since you're new to Perl, I also recommend the "Llama" book: Learning Perl, by Randal L. Schwartz & Tom Christiansen (2nd Ed., July 1997, O'Reilly & Associates, ISBN: 1-56592-284-0) if you're primary working on a Unix box. If you'll be using a Windows system, you may prefer the "Gecko" book: Learning Perl on Win32 Systems, by Randal L. Schwartz, Erik Olsen, and Tom Christiansen (Aug 1997, O'Reilly and Associates, ISBN: 1-56592-324-3) which prvides much of the same information as the Llama book, but oriented for those with a Windows background.

    Finally, if you're planning to run this from a web site, you may also want to look at the "Rat" book: CGI Programming with Perl, by Scott Guelich, Shishir Gundavaram, & Gunther Birznieks (2nd Ed, July 2000, O'Reilly and Associates, ISBN 1-56592-419-3). This contains a wealth of useful material regarding Perl CGI scripts, including a brief discussion of using the DBI module with your CGI scripts.

    It's a lot to read, but once you've gone through those three books, typed in the examples, and put together some sample database scripts, you should be well on your way to mastering databases with Perl.

    I would also recommend using tools provided by the Monestary itself to review previous discussions, Q&A's, and tutorials along these lines. While this can be somewhat time-consuming, you'll learn a great deal simply by reading the real-world questions and conversations posted by your fellow monks.

    --f

    Update: Fixed the animal for LP; thanks davorg. Mis-remembered the zoology lessons from the Colophons. Sorry.

    Update 2: Fixed the other reference, too. Sheesh; you'd think I hadn't had my coffee yet. Again, thanks, davorg.

      Since when was Learning Perl the 'Alpaca' book? I'm pretty sure that just about everyone calls it the Llama book - because it has a llama on the front cover.

      --
      <http://www.dave.org.uk>

      "Perl makes the fun jobs fun
      and the boring jobs bearable" - me

Re: HOW TO CREATE A DATABASE FILE WITH FIELDS IN RECORDS
by ichimunki (Priest) on Dec 27, 2000 at 07:42 UTC
    Yes. Perl has the ability to create and store incredibly complex data structures. It has numerous methods for using data on disks, one way or another. And it can certainly store both ascii and binary data in files and pack and unpack this

    See perlman:perldata, perlman:perlref and Data::Dumper for some hints as to just how easy it is to build some very complex records and store them to disk. Perl is not a highly optimized DB management system, though-- it is a programming language. But building small (which I suppose is pretty relative) DB packages would be one great way to start with Perl.

      I wouldn't suggest using pack and unpack to build a database. But you can build your own databases without having to buy an external database. You can use MySql or even build a database out of text files using DBD::CSV, for example, or use one of the DB_FILE modules. You might want to try http://search.cpan.org/ for more details and more ideas. Sorry, I don't actually do much of this kind of stuff.

              - tye (but my friends call me "Tye")
        "I wouldn't suggest using pack and unpack to build a database."

        I would. In fact, that is historically exactly what they are for. On the other hand, I wouldn't suggest building your own database when there are perfectly good ones out there so it is rather moot. =)

        If your needs are lightweight and the data sets are small you should likely try DBD::CSV or even just something like:

        #!/usr/bin perl -w use strict; use Data::Dumper; my @LoL = ( ["aa", 1, 2, 3, "Hi Mom!"], ["bb", 2, 3, 5, "No soup for you!"], ["cc", 3, 1, 2, "Not me!"], ["def", 4, 4, 4, "Like gag me with a spoon!"], ); { #block for localizing open FOUT, ">./datafile"; local $\=$/; #Record separator set to input separator. local $"=$;; #Array interpolation separator set to subscript separat +or. print FOUT "@$_" foreach (@LoL); close FOUT; } my @newLoL; { #block for localizing open FIN, "<./datafile"; push (@newLoL, [split m/$;/]) while (<FIN>); close FIN; } print Dumper(\@LoL, \@newLoL);

        As long as you don't want space efficiency, memory efficiency, power, or binary data, that way is blazingly fast. =P The field separator in your file will be "\034" and the record separator would be "\n" so those two chracters are verboten in the data.

        Useless triva for you, chracters \034-\037, the ones between the ESC and the SPACE chracters were once for data separation. Perl (thanks to AWK) uses \034 aka "FS" as its field separator. This is in fact a horrendous error since the original meaning of the chracters is: File, Group, Record, and Unit Separator. We really "should" be using the Unit separator "\037" for breaking up fields but so many people do it wrong that it isn't worth the fight.

        --
        $you = new YOU;
        honk() if $you->love(perl)

        I was using the words pack and unpack without regard for their Perl meanings, and then realized I might as well link them as one of the many ways Perl makes it easy to fetch and store data. I don't recommend using any Perl function or Perl itself for the heavy lifting part of database development. Although by carefully crafting your own indexes, and storing your data in fairly discrete, delimited text files, you can easily put together a relatively simple system, but it won't have near the flexibility that using Perl in conjunction with some other package would.
Re: HOW TO CREATE A DATABASE FILE WITH FIELDS IN RECORDS
by davorg (Chancellor) on Dec 27, 2000 at 14:58 UTC

    Take a look at the tie function. You may well be able to do what you want you want by tieing a hash to an MLDBM object.

    --
    <http://www.dave.org.uk>

    "Perl makes the fun jobs fun
    and the boring jobs bearable" - me

Re: HOW TO CREATE A DATABASE FILE WITH FIELDS IN RECORDS
by repson (Chaplain) on Dec 31, 2000 at 06:44 UTC
    The other replies to this node are about creating a database engine
    If you just want to create a new database within an existing database engine (which is how I - probably incorrectly - read this node) is not too hard.
    use DBI; my $dbh = DBI->connect("DBI:enginename"); $dbh->do("CREATE DATABASE foo"); $dbh->do("CREATE TABLE foo.blah (id INT, data BLOB)"); # then proceed with inserting and selecting
    You may or may not be able to use all this syntax depending on database engine, but you could probably implement the funtionality non-portably with $dbh->func('something'); or stored procedures. Read DBI docs together with docs for the driver used eg..DBD::Oracle