Hi Tilly, Thanks for your response. Yes we are looking at atleast 20 to 50 thousand responses, answering around 50-80 questions. Also the same respondents could be asked more questions in the future, so the question database would go on increasing over time. I looked at Data::Mining module, but it doesn't cater to what I need. I am looking at a perl hash of hash right now and storing them into a file using Storage.pm or something and loading it in memory when required, to reduce the time. There could be soooo many associations and rules, I am thinking of writing a rule-engine too, I researched on yagg yesterday. It could be like In year 1980 in city 'new york' maxSold fabrickType for age-group 20-25 ? Can I store this historical data in the db? or I am better off using hashes. I preferred XML, as my hash would easily form an XML DOM Tree. 
      <results>
         <result id='1'>
             <year = '1980'/>
                <sales>
                   <sale>
                     <location>New York</location>
                      <cotton>5.6</cotton>
                      <nylon>8.4</nylon>
                     </saled>
                </sales>
         </result>
      </results>

[download]
I will look into PDL and Statistic Package as well. Thanks All. Sandeep

In reply to Re^2: Research App Optimization by sskohli
in thread Research App Optimization by sskohli

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.