Bismark has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, I have done a good amount of searching and as of yet, am unable to determine the answer to my question. I am hoping that someone out there might have attempted to do what I would like to accomplish. I have a form that I built as an executable( Using FlashMX). I know that I can use Perl to communicate with other db's. I also know that my form data from this application can be submitted to a Perl script. What I would like to attempt, if there is a way, is this. Take the data from the form and have it saved in its own db that can then be queried. The form has about 100 fields that would need to be saved and sorted. I would like it to create its own db so that I would not need a server to run the program on. Is this possible with Perl? I hope that my question is clear enough. If not, please send blessings to a new member to the Monastery and I will repent the errors of my ways. Thank you for any help in finding the right direction. Kerry

Replies are listed 'Best First'.
Re: Can Perl create its own searchable db?
by derby (Abbot) on Jan 10, 2003 at 18:06 UTC
    so that I would not need a server to run the program on

    do you mean so that you would not need a database server? Or webserver? If database server, there are several cpan and core modules that can save your data to a "database like" storage (DB_File, DBI and DBD, Text::CSV, etc). You may want to also do a search on HTTP persistence to see how to keep state across multiple requests. If you meant web server, I'm not sure what you're trying accomplish.

    -derby

    update: I meant of course DBD::SQLite and DBD::CSV or DBD::File.

      Maybe I need more experience with Perl to formulate my question properly. I will explain what the goal of this program is and how it will be used. Perhaps that will help. As I said in the first post, I have an executable that is called First Article Report. Here at work we have a common network drive. Lets call it the 'M' drive. I would like this program to reside on the M drive so that the users can just go into the folder and execute the program. Using the program, the user would fill out the information fields and then click the 'submit' button. The submit button would then send the data to the script. I would like the data to then be stored in a sub-folder that resides within the same folder as the program. Everything would be local to this one folder on the M drive. The plan was to then be able to search this folder's data to be able to pull out the information submitted. The reports would be filed by a report number and then that would be one of the fields that could be queried. So I guess the answer to your question would be that I do not need a web server for the program. But, would I need a web server for the script to run or can it stand alone. I hope that this made it a little clearer. I will definately check into the sources that you listed in your reply. Thank you
        To answer your title question - Yes perl can create a searchable db.

        To more fully answer your question would require a whole lot more dialog about your requirements, your goals and your expectations. You can easily create something like this in perl with a single (or few scripts) but it may not scale well (mutliple people modifying the report, large number of reports can kill simplistic searches).

        Do you need a web server? I don't know. From you original post about using Flash, I immediately thought web. But since I have a healthy disdain for flash, I have no idea wether or not it can create standalone apps or not. The part about "sending" data to the scripts implied CGI (and hence web), but once again, maybe Flash has hooks that allow you to create standalone apps that are CGI-like. Or maybe not - I don't know. You could do the while thing as a GUI (Tk,WK,Win32,etc) app if all the client machines accessing the data have perl (and the necessary modules) installed.

        A simple thing like this can quickly bloom into a full scale Document Management System. Most of the monks here make their living doing such work and while most (if not all) would be glad to help you tweak or understand such a system, few have the time to help you build one from scratch.

        My advice would be to start small using a webserver. Create the forms and scripts to just save data to a centralized data store. Work on adding, deleting and updating this data and then move on to searching. You're first going to run into configuration issues such as where can I save data on the webserver? (Hint it may not be that m drive you talk about or maybe the webserver sees the m drive as something else). You may want to peruse the book stores and check out CGI Programming - especially the chapters on Data Persistence and Maintaining State.

        Sorry if I sound a bit preachy but that's a tall order you're asking for (even though it sounds quite simple).

        -derby

        If your form is going to 'submit' to a Perl script then yes, you will need a web server. If you already have a machine acting as a file server for your 'M' drive then you might be able to run the web server on that machine.
Yes... sort of.
by khudgins (Sexton) on Jan 10, 2003 at 18:23 UTC

    If your host is a UNIX-based server, then you can use the Berkely-style database that's inherent to the system. DBD::File is where you want to go, as referenced above.

    Since you're just storing records in a flat table (I'm assuming you don't want a lot of tables - Database guys love them, but unless you're comfortable with DB normalization, I'd stick with a flat table. - http://www.devshed.com has a good lesson on database structure, but their website is broken ATM and I can't get you a direct link.

    The other possibility is to just write all your fields to a delimited text file. Just append to the file every time the form is submitted, and then you can open your datafile up in a spreadsheet and sort to your heart's content.

    Here's a tweakable example. I fully admit that this code can be optimized and it *will* have to be edited for your form. I'm doing this on purpose wo you learn how it works better. =)

    use CGI; $outputString = param('formfield1') . "|" . param('formfield2'); open OUTPUTFILE, "filename.txt"; print OUTPUTFILE $outputString . "\n";
    I'd also spit them out a second page saying thanks for filling out your 100-field form. One possible easy way is HTML::Template. Hope this helps.

    UPDATE: Corrected bad CPAN Link

      So if I am understanding your answer, this method would create a text file that would be appended everytime the form is submitted. Then if say the first time it is submitted I have report #100, then it is submitted again so now I have report #101. These would be in a delimited text file. Now I want to search the reports so that I can see report number 101. So I would want to load just that one section of the text doc. Or can I create a seperate text doc each time the form is submitted and then just load that one file back into the program. It seems to me that I could maybe set the text filename as a variable and then fill it with the report number which the program generates each time a new report is filed. The purpose of this form is to replace a paper version that we now use when ever we perform a first article inspection on a new product. That is the reason for the large size of the form.
        Yes, you can get a timestamp in UNIX time format by calling time(). It'll give you an integer value of seconds since Jan 1, 1970, so you can be fairly sure that you'll get a unique value. So when you call your open( FILEHANDLE, "filename.txt"); you can first say:
        $filename = "prefix-" . time() . ".txt"; open( FILEHANDLE, $filename );
        The only problem you're going to have is when you need to compare reports. It's going to be a beast to write a comparison program to read all these files and give you relevant data. If you never need to do that, then separate files would probably be the ideal solution for you. If you *do* need comparison ability, then a real database is probably the best idea. If so, I'd either install a database server (MySQL is free and can be installed on most any platform. Postgres SQL is considered by many to be superior, and even more free. I *know* it runs on Linux and the BSDs, but I'm not sure what other platforms it's been ported to... good oods on any *NIX)
      <quote>Since you're just storing records in a flat table (I'm assuming you don't want a lot of tables - Database guys love them, but unless you're comfortable with DB normalization, I'd stick with a flat table.</quote>

      Well.. hmm... Perl is so great at text searching that this advice is less barmy than it might be elsewhere - but I still say learn normalisation and enjoy a happier life! It's not hard and one day, when you want to search on a combination you never thought of before, you'll be glad you did.

      Later... nuts! My markup's wrong but it makes the point!

Re: Can Perl create its own searchable db?
by blokhead (Monsignor) on Jan 10, 2003 at 19:18 UTC
Re: Can Perl create its own searchable db?
by CountZero (Bishop) on Jan 11, 2003 at 09:22 UTC

    My personal opinion is that you are going to run into problems at some time or another because your Flash executable on the shared "M"-drive will at some time or another be in use by two or more persons at the same time and it has no way of knowing it.

    This could lead to all kinds of subtle or not so subtle race conditions were your program tries to save the answers to your form data to the same file (leading to a "cannot use this file as it is already in use by someone else"-type of error if you save the answers to a single file, or two users trying to save the answers to new file with the same name whereby the second one overwrites the first one's file).

    If your Flash program can guarantee to use an unique filename to save its results into, you could run a Perl-script at regular intervals to reap the newly created files, parse their contents, store the data in a database-file and delete the original files (or store them somewhere else). If this PERL-script runs at moments when no-one is inputting new data (say at night when the office is closed), you will avoid a lot of problems.

    Still, of course it is only a kludge in order to avoid running a real web-cum-database-server.

    Believe me, I tried all these simple solutions first too and now have come to use a web-and-database server and find new uses for it every day: e.g. you could show the results of all the queries in a dynamic webpage.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: Can Perl create its own searchable db?
by theorbtwo (Prior) on Jan 11, 2003 at 09:51 UTC

    I don't know enough about FlashMX to really say much, but I'll try anyway.

    I see two basic ways (well, perhaps three or four, but two applicable ways) to get the data from the flash form to the perl script. Either the perl script can be accessable via HTTP (which doesn't neccessarly mean running a full webserver, but I'll get to that in a moment), or you can set up the frontend (the FlashMX program) to run the perl program with some arguments. They each have their own set of problems.

    I suspect running a webserver is what you want to do. You can either set up a web server, and get your script to run as a CGI, or you can write a web server in perl. Either is fairly easy -- Apache and IIS are both easy to setup (for a simple server like this), and HTTP::Daemon is rather easy to use. Doing it this way avoids the contention and locking problems you'd have with doing everything on the client side.

    You can also have the frontend simply call the backend with a bunch of parameters. However, if you do it this way, you'll end up with a long commandline, which windows may not like. Also, you'll have to deal with locking issues (file locking over a network isn't the easiest thing in the world, though you do have the advantage of a relitavely homogonious installed base, and only one file server). However, you won't have the administrative and security problems associated with yet another server.

    I'm sure I'm forgetting somthing, but I normaly am. For that matter, "am" can refer to either being sure, or forgetting somthing... no what did I do with my brain again? It's around here somewhere, I'm sure.


    Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

      If you are adventurous you could try to make your exe a perl script instead of a flash projector.

      Using Win32::OLE and/or this package Win32::GUI::AxWindow you should be able to coax perl into firing up a flash movie (swf) as a windowed activex control.

      Then you can get/put variables between the two without a webserver using fscommand() (from flash) here is a tutorial covering moving vars from flash to vb. I can't remember the method for moving vars from vb to flash but I have tried it based on another tutorial whose url escapes me right now.

      At least you can know it is possible.

Re: Can Perl create its own searchable db?
by mohadib_the_nasty (Initiate) on Jan 13, 2003 at 01:27 UTC
    I would agree with others who suggest using some
    sort of a SQL database. Flash could make a text file
    for perl to parse. Have perl populate a local
    sql database using a sql SEQUENCE to ensure non-dups.
    I believe this could all be done with out a webserver
    as long as each client has the standalone player.

    That would get info into the data base. Using the
    reverse...have perl create a text file from the database
    for flash to use...would get info out of the database.
    However...its been my exp...with flashplayer6 on linux
    that flash is slow and awkward at displaying text on
    the fly. I have a small example...

    http://www.taproot.bz/~van (requires flash....duh)

    this is swf pulling text from the "news letter" every
    time its is viewd. Granted this is web-enabled...I think
    its slow to build the movie...and the webserver is on my
    LAN.

    jd