Re: Flat file DB

Hi,
I once wrote this code to learn some few things (I'm rather new to Perl, too).
Save it into an executable file (I named it "agenda"), and it should work as a flat DB reader and printer. It is intended as an address manager... Probably silly (grep and few other things can do the same), but you can maybe grab some simple idea for your DB needs.
ciao Roberto

#!/usr/bin/perl

use strict;

my $ref_values;
my $ref_names;

### Database file
my $db_file = './phone.txt' || $ARGV[1];

### Field eparator
my $separator = '#' || $ARGV[2];

### Set the primary key
my $pri = 1 || $ARGV[3];

my $primary = $pri - 1;

my $keyword = $ARGV[0] || die
    qq(     \n),
    qq(use: agenda keyword (databasefile separator pos.key)\n),
    qq(     \n),
    qq(     default database:  $db_file\n),
    qq(     default separator: $separator\n),
    qq(     default pos.key:   $pri\n),
    qq(     \n),
    qq(     examples of use:\n),
    qq(     agenda smith  (shows all '*smith*' in field 'pos.key')\n),
    qq(     agenda .      (shows  all records)\n),
    qq(     \n),
    qq(     To create a database, edit a text file writing in the firs
+t\n),
    qq(     line the names of the fields, separated by some *separator
+*\n),
    qq(     ('#' for example), and related entries in the following re
+cords\n),
    qq(     \n);

### Load the variables
($ref_values,$ref_names) = db_read($db_file, $separator, $primary);


### Print the database content
db_print($ref_values,$ref_names,$keyword);
############
sub db_read{
############

###
### This routine reads the content of an ASCII database
### with field names as first record, storing the content
### into an hash of arrays.
###
###

    use strict;

    my $db_file = shift();    ### database file name
    my $separator = shift();  ### field separator
    my $primary = shift();    ### primary key

    my %values;
    my @fields;
    my @names;
    my $i;

### Open the database for reading
    open DB, "<$db_file" or die "Can't open $db_file: $!\n";

### Read the first record containing the names of the fields
    my $first_record = <DB>;

### Remove the newline
    chomp $first_record;

### Split the record to get the field names
    @names = split($separator,$first_record);

### Now read the rest of the file and store
### the records into an hash of arrays

    while (<DB>) {

### Remove the new line character at end of record
        chomp;
### Split the record into fields
        @fields = split /$separator/;

### Populate the hashes defined by the fields in $first_record
        foreach $i (0..$#names-1) {
           $values{$fields[$primary]}[$i] = $fields[$i];
        }
    }

### Close database
    close DB;

### Return the the reference to the hash of arrays
### and the array of field names
    return (\%values,\@names);

}

#############
sub db_print{
#############

###
### This routine prints the content of an hash of arrays
### sorted by its keys. The hash of arrays is created by
### db_read subruotine.
###

    use strict;

    my %values = %{ shift() };
    my @names = @{ shift() };
    my $keyword = shift();

    my $item;
    my $features;

    for $item ( sort keys %values ) {

### Scan the hash of arrays searching for $keyword

         my $found = 0;
         for $features (0..$#names-1) {
             if ($item =~ /$keyword/i or
                 $values{$item}[$features]=~ /$keyword/i) {
                 $found = 1;
             }
         }
### Print the hash of arrays if anything was found

         if ($found) {
             printf qq(\n ** keyword: %s\n\n), $item;
             for $features (0..$#names-1) {
                 printf qq(    %s = %s\n), $names[$features],
                                           $values{$item}[$features];
             }
             $found = -1;
         }
    }

}
[download]

a sample phone.txt file:

1#COGNOME#NOME#ENTE#UFFICIO#FAX#CELLULARE#CASA
2#Smith#John#CNR ICG#012 306816#012 304056##
3#Doe#John#AAA Corp.#012 306816#012 304056#335 2321212#
[download]

Comment on Re: Flat file DB Select or Download Code

Replies are listed 'Best First'.
Re: Re: Flat file DB by Hofmator (Curate) on Aug 30, 2001 at 18:09 UTC
Allow me to comment a little bit on your code, you are sometimes making your life harder than it has to be ;-) I will make the comments sequentially with the source code. For multiline strings use a HERE document like this `my $keyword = $ARGV[0] \|\| die << "USAGE"; use: agenda keyword (databasefile separator pos.key) default database: $db_file default separator: $separator default pos.key: $pri and so on USAGE # the line above marks the end` [download] It is sufficient to `use strict` only once at the top of your script, you don't have to repeat it in every subroutine. `open DB, "<$db_file" or die "Can't open $db_file: $!\n"; # leave out the final \n open DB, "<$db_file" or die "Can't open $db_file: $!";` [download] Then perl appends automatically the line number where the error occured - very helpful for debugging. And don't worry, perl will put a newline at the end of the statement. Use less comments!! Yes, too many comments can be as bad as too little comments as the code gets cluttered and makes reading more difficult, e.g. `### Close database close DB;` [download] is really unnecessary, both lines read exactly the same. There has been quite some discussion about commenting correctly here at PM, have a look with Super Search. This can be contracted `my $first_record = <DB>; chomp $first_record; # in one line chomp(my $first_record = <DB>);` [download] You are throwing the last field of the every entry away, $#names contains the last index of the array @names, not the number of elements. To avoid such one-off errors don't use loops which use indices - if avoidable. Here you can write this instead of the loop: `@{ $values{$fields[$primary]} } = @fields;` Use compiled regexes with the /o modifier if the variable you are interpolating doesn't change, e.g. `$item =~ /$keyword/io;` Don't use printf unless necessary, in your cases you can simply use the string interpolation and write `print "\n ** keyword: $item\n\n"; # and print " $names[$features] = $values{$item}[$features]\n";` [download] There are more things, e.g. I don't think the hash of arrays is the best datastructure for this task, you are not using the main feature of a hash, the direct access to an element by a key. It has probably been some time since you wrote this script, your perl has improved in the meantime. So as a good exercise I'd suggest to rewrite the whole thing, rethinking your data structure and incorporating my advices. If you then post your script here you will get further suggestions for improvement. And at the end of the day you will have learnt a lot :-) -- Hofmator	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: Re: Flat file DB
by Hofmator (Curate) on Aug 30, 2001 at 18:09 UTC

Allow me to comment a little bit on your code, you are sometimes making your life harder than it has to be ;-) I will make the comments sequentially with the source code.

For multiline strings use a HERE document like this

my $keyword = $ARGV[0] || die << "USAGE";

use: agenda keyword (databasefile separator pos.key)

  default database:  $db_file
  default separator: $separator
  default pos.key:   $pri

and so on
USAGE

# the line above marks the end
[download]

It is sufficient to use strict only once at the top of your script, you don't have to repeat it in every subroutine.
```
open DB, "<$db_file" or die "Can't open $db_file: $!\n";

# leave out the final \n
open DB, "<$db_file" or die "Can't open $db_file: $!";
[download]
```
Then perl appends automatically the line number where the error occured - very helpful for debugging. And don't worry, perl will put a newline at the end of the statement.
Use less comments!! Yes, too many comments can be as bad as too little comments as the code gets cluttered and makes reading more difficult, e.g.
```
### Close database
    close DB;
[download]
```
is really unnecessary, both lines read exactly the same. There has been quite some discussion about commenting correctly here at PM, have a look with Super Search.

This can be contracted

my $first_record = <DB>;
chomp $first_record;

# in one line
chomp(my $first_record = <DB>);
[download]

You are throwing the last field of the every entry away, $#names contains the last index of the array @names, not the number of elements. To avoid such one-off errors don't use loops which use indices - if avoidable. Here you can write this instead of the loop: @{ $values{$fields[$primary]} } = @fields;
Use compiled regexes with the /o modifier if the variable you are interpolating doesn't change, e.g. $item =~ /$keyword/io;

Don't use printf unless necessary, in your cases you can simply use the string interpolation and write

print "\n ** keyword: $item\n\n";

# and
print "    $names[$features] = $values{$item}[$features]\n";
[download]

There are more things, e.g. I don't think the hash of arrays is the best datastructure for this task, you are not using the main feature of a hash, the direct access to an element by a key.

It has probably been some time since you wrote this script, your perl has improved in the meantime. So as a good exercise I'd suggest to rewrite the whole thing, rethinking your data structure and incorporating my advices. If you then post your script here you will get further suggestions for improvement. And at the end of the day you will have learnt a lot :-)

-- Hofmator

[reply]
[d/l]
[select]