I would also go with what other Monks suggested: to use a sub for each of your proposed scripts' logic. One of their input parameters should be the DB handle. Once you have the three subs, you can still create 3 scripts, as per your original proposal, which reads in DB credentials, create a DB handle and call the appropriate sub. So you have both worlds (for whatever reason). (Edit: or call the three subs from one script which at the beginning asks or reads DB credentials once.)

One point is unclear to me, you mention "join". Does it refer to SQL JOIN or to concatenate?

Regarding your proposed "temporary table". It may not be necessary to create temporary tables if all you need is to pull results from DB, filter or concatenate and save. Perhaps you can save these results in Perl variables and do the transformations in Perl, but only if you are operating from a single Perl script calling the 3 subs (and data is small, though SQL can do it better in the DB, if one can write it that is ;) ). I don't know if this will be preferable than using the DB as temporary storage, it depends on your SQL knowledge and size of results to be transformed.

Also, I have recently used Redis which is a nosql (temporary) data store, which more-or-less acts like a Perl Hash, but accessible from many programs within the same machine (or remote). I used it to share temp data between scripts (like your proposed 3 scripts) when I was too lazy to write SQL and do it within the DB. That data was like a string blob, JSON string etc. But Redis did not care about any structure. Which was very convenient in storing it and retrieving it, no questions asked regarding structure. If you find this idea interesting but you are not allowed to install Redis, here is a very lame implementation in pure Perl: Simple data-store with Perl . Note that you can store Perl data structures, e.g. nested hashes, into files or DB blobs by using Data::Serializer or Storable

In reply to Re: designing a program - your wisdom needed by bliako
in thread designing a program - your wisdom needed by SpaceCowboy

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.