It's a funny coincidence (but I'm not laughing yet), that I saw this post at the end of a day where I learned that I may need to switch from perl built without threads to perl built with threads, in order to get DBD::Oracle to work (cf. DBD::Oracle and threading on a web server).

So your findings about runtime and memory usage led me to ponder how likely it would be that some simple-minded perl script using DBI to do stuff with data from Oracle tables might get kind of ugly, just because a simple-minded perl hacker doesn't know about, think about, or have a choice regarding the kind of coding adjustment you demonstrated in your benchmarks.

For example, someone decides to load a lot of data from a file before connecting to the database -- and then DBD::Oracle starts doing stuff with threads "under the table", completely unbeknownst to the hapless programmer. (Maybe DBD::Oracle doesn't really do stuff with threads, but then I don't understand why it seems to need thread support...)

Anyway, part of what you're hoping for seems unattainable, unless you accept a trade-off: you can economize on memory and runtime if you can specify up front exactly how many threads you intend to use. That's great, but isn't there a whole class of apps whose defining trait is the ability to start new threads on an as-needed basis (not knowing in advance how many will be needed)?

I am not someone who can go into detail on this, but roughly speaking, it sounds like what is needed is a way to define some sort of initial minimal state -- like a snapshot at startup -- such that each new thread starts out with just the minimal stuff defined therein; the parent process might know of specific data that a given thread would need, and would explicitly enable the access (whether copied or shared), but without this action from the parent, the thread must simply accumulate its own data separately.

I don't have a clue how that would be implemented (for all I know, it might already be implemented!) -- but just in conceptual terms, that seems like what you'd want.

In reply to Re: threads: spawn early to avoid the crush. by graff
in thread threads: spawn early to avoid the crush. by BrowserUk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.