Stamp_Guy has asked for the wisdom of the Perl Monks concerning the following question:

Ok, I have no access to any SQL server whatsoever, and I'm stuck using flat files. I have a database that is in the following format:
CAT #    DESCRIPTION    PRICE

It has 10,100 items. Currently, there are multiple listings with the same catalog number but different descriptions and prices. I want to be able to give the program two arguements: a first number and a last number and have it display everything in between. Right now I'm doing that by adding an extra field that is basically a line number in front of the catalog number and looping through until it finds the first number, then it prints everything until it reaches the last number. However, I'd like to use the catalog numbers as the numbers I use for the first and last arguements.

To see an example of what I'm doing, click here. Here's my code:

!/usr/bin/perl5 &parse_form; # Parse the form $first = $FORM{'first'}; # First number in range $last = $FORM{'last'}; # Last number in range $condition = $FORM{'condition'}; # Condition of each item $maxprice = $FORM{'maxprice'}; # Maximum price customer wants to p +ay $fields = 4; # Number of fields in each record $filename = "pricelist.txt"; # The database text file $results = 10000; # Maximum number of results to display &open_file("FILE1","",$filename); # Open the database file &mime; &print_top; $counter = 0; if ($first =~ /[^0-9]/) { &error; } if ($last =~ /[^0-9]/) { &error; } while (($line = &read_file("FILE1")) && ($counter < $results)) { # split the fields at the | character @tabledata = split(/\s*\|\s*/,$line ,$fields); &check_record; if ($found == 1) { $counter++; &print_record; } } close(FILE1); if ($counter == 0) { print "<TR><TD colspan=4><BR><B> Sorry, No Matches were found.</B> +</TD></TR>\n"; } &footer; ###################################################################### +### # # # Subroutines # # # ###################################################################### +### # Check the record sub check_record { $item_number = $tabledata[0]; $scottnum = $tabledata[1]; $description = $tabledata[2]; $descriptionurl = $tabledata[2]; $price = $tabledata[3]; $keywords = $tabledata[4]; $descriptionurl =~ s/\s/\+/g; $sfound = 0; $found = 0; $notfound = 1; if (($item_number >=$first) && ($item_number <= $last)) { if ($description =~ /$condition/i) { $sfound = 1; } elsif ($condition =~ /all/i) { $sfound = 1; } } else { $notfound = 0; } if ($sfound == 1 && $notfound == 1) { $found = 1; } } # Parse the Form sub parse_form { read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'}); if (length($buffer) < 5) { $buffer = $ENV{QUERY_STRING}; } @pairs = split(/&/, $buffer); foreach $pair (@pairs) { ($name, $value) = split(/=/, $pair); $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; $FORM{$name} = $value; } } # Open the File sub open_file { local ($filevar, $filemode, $filename) = @_; open ($filevar,$filemode . $filename) || die ("Can't open $filename"); } # Read and/or Write to the File sub read_file { local ($filevar) = @_; <$filevar>; }
Any suggestions on how I could make it so I could pass it the two Catalog numbers instead of the control numbers?

Replies are listed 'Best First'.
Re: More Flat-File Database Questions
by eg (Friar) on Feb 11, 2001 at 12:27 UTC

    In other words you want to "Scott Numbers" rather than "item numbers"? You need to edit the check_record subroutine. Change

    if (($item_number >=$first) && ($item_number <= $last)) {

    to

    if (($scottnum >=$first) && ($scottnum <= $last)) {

    For what it's worth, I think your script could do with a complete re-write -- it looks like it was originally written for perl 4.

      Complete Rewrite? You got that right! This was one of my first scripts. It's a modification of a freebie I found. I fully intend on rewriting it, but I'm trying to find out how to do what I said above in this post.

      I know I could change it from $item_number to $scottnum, but that wouldn't work because some of those numbers contain letters in them and there are multiple items for nearly each Scott number. Any other ideas?

        So what does $scottnum actually look like? Can you give some example of the mixing of letters and numbers? If the letters in $scottnum are irrelevant to the selection process, you can quash them out with

        $scottnum =~ tr/0-9//cd;

        or, if the irrelevant bit is at the end of the number, just do the comparisons on the int( $scottnum ).

        It doesn't matter that there are multiple entries for each scott number, since each line is treated independently in the selection process. If you have "100a" and "100b", they'll both be in the range [0, 200].

        Hi Stamp_Guy, i'm trying to make the same database you did, but i'm in troubles with some thimgs. Could you send me code of it, like de CGI and some lines of the txt database? Thanks Miguel
Re: More Flat-File Database Questions
by ichimunki (Priest) on Feb 11, 2001 at 20:59 UTC
    This is a long list of suggestions, only the last of which will start to answer your actual question.

    use strict -- you have subroutines assigning values to undeclared variables which the main body then assumes exist when it goes to the next statement. It woould be clearer to have things like:
    my %form; ($form{'First'}, $form{'Last'}) = parse_form("First", "Last");
    and this brings up use CGI for parsing forms. It looks like this script attempts to parse form input on its own. A practice most Monks encourage avoiding when possible.

    Finally, there is a module, DBD::CSV to treat CSV files as SQL capable databases. By using a module you make it possible to switch to a more powerful SQL backend at any time.

    In spite of all this, I still don't understand why you can't simply abandon the line number scheme altogether. As you loop throught the lines of data (assuming they are ordered in the file), you should simply be able to start pulling out lines where the catalog numbers are within the range.
Re: More Flat-File Database Questions
by Stamp_Guy (Monk) on Feb 12, 2001 at 07:04 UTC
    I wish I could just use the numeric value of the catalog numbers, but the problem is that the numbers go something like this:

    • 1-3400 numerical.
    • C1-C131
    • E1-21
    • Q1-12 etc. etc. etc.
    Also, there are groups of multiple items, such as 3119-3125, and those don't work so well with the numbering scheme. It looks like I'm pretty much stuck. Basically I guess I need a memory-efficient, quick way to find the line number of the first and last Scott numbers.

    However, I am intrigued by this DBD::CSV module. Can anyone who has used it give me some examples that I could use to learn it?

      With numbers like that you are in a bit of a spot for trying to use them as ordered constraints. Still, though, if your flat file is in the correct order, you can simply start grabbing rows on the first match and then keep grabbing until you get a non-match.

      To use a SQLish DB (even DBD::CSV) if you cannot abandon the catalog numbers and they are not unique to a row, you have a serious issue to solve in terms of assigning a unique ID number to each item at some point and translating the catalog number into the unique ID. If the catalog numbers are class-oriented, then you could assign sets of items a class ID and (with DBD::CSV) use a SQL statement like "select * from tablename where class = 'stamp_type_one' order by catalog_number;" to grab all the rows you're looking for sorted by catalog number.

      The perldoc for DBD::CSV is pretty straightforward, and simple SQL is not so tough that you probably wouldn't be able to pick up most of what you need from just reading the module docs. If not, searching google or about.com for SQL should find you some decent tutorials pretty quickly.
Re: More Flat-File Database Questions
by Uma (Initiate) on Feb 15, 2001 at 10:45 UTC
    #!/usr/local/bin/perl -w #=========================================================== # Get/Organize Parm & Variables #=========================================================== if ($ARGV[0]) { $parm = $ARGV[0];} else { print "Prameter missing.... \n"; die "Program Terminated... \n"; } chomp($parm); @parm_list = split(/\|/,$parm); $a = @parm_list; if ($a != 2) { die "Number of parameters not satisfied\n";} else { ($first_item_num, $last_item_num)= @parm_list; } $first_item_num =~ s/^\s+|\s+$//g; #trim $last_item_num =~ s/^\s+|\s+$//g; $seq = 0; # create a variable %cat_key_hash =(); # init hashes %cat_dtl_hash =(); #=======================================================================================# # M A I N P R O G R A M #=======================================================================================# open_files(); load_hash(); print_list(); close_files(); #=======================================================================================# # E N D O F P R O G R A M # #=======================================================================================# sub open_files{ $datadir = "/dir"; open (INP, "<$datadir/$flat_file")|| die "Couldn't open Flat file, $!\n"; open (OUTRPT, ">$datadir/$file_rpt"); } #=======================================================================================# sub load_hash{ while(<INPEDI>) { chomp; $inp_str = $_ ; ## Let inp_str =>CAT_NUM DESCRIPTION PRICE ($inp_cat_num, $inp_desc, $inp_price) = split(/\|/,$inp_str); if ($cat_key_hash{$inp_cat_num}) { $seq = $cat_key_hash{$f_cust_po} $seq ++ ($cat_key_hash{$inp_cat_num}) = $seq; # concatenate sequence number with catalog number # to make the key to be unique ($cat_dtl_hash{$inp_cat_num.$seq}) = $inp_str; } elsif if ($cat_key_hash{$inp_cat_num}) { $seq = 1; ($cat_key_hash{$inp_cat_num}) = $seq; ($cat_dtl_hash{$inp_cat_num.$seq}) = $inp_str; } } #end while } #end sub #=======================================================================================# sub print_list{ # order your key list @key_list = sort {$a <=> $b} (keys %cat_key_hash); foreach $key (@klist){ if (( $key >= first_cat_num) and ( $key <= $last_cat_num)) { $seq = $cat_key_hash{$key}; #find how many records for that catalog# for (i=1, i <=$seq, i++) { #print all the details # You may split and format the output $dtl_key = $key.$i; # create key print "$cat_dtl_hash{$dtl_key}\n"; } # end for } #end if } # end print_list #=======================================================================================#