Hello monks,

I am trying to find modules for some advanced data mining projects in perl. In python they have iterator module with functions like pairwise etc. that does the task for me. However I need to do this perl. So trying to avoid reinventing the wheel. I am going to describe a sample problem so that it will give you an idea for any recommendations you may provide. I have a data set like below...

10/01/2016   99.71
10/02/2016   99.53
10/03/2016   100.10
10/04/2016   100.96
10/05/2016   100.99
10/06/2016   101.38
10/07/2016   100.74
10/08/2016   100.70
10/09/2016   99.88
10/10/2016   97.62
10/11/2016   97.55
10/12/2016   99.12

[download]
Now I need to find 5 consecutive records either increasing or decreasing by value next to date field. For e.g. I need the following 2 result sets in my output. 
10/02/2016   99.53
10/03/2016   100.10
10/04/2016   100.96
10/05/2016   100.99
10/06/2016   101.38

10/07/2016   100.74
10/08/2016   100.70
10/09/2016   99.88
10/10/2016   97.62
10/11/2016   97.55

[download]
Please note that I am looking for exactly 5 consecutive records with either increasing or decreasing patterns, any other number of consecutive records with those patterns should not qualify.

I know this is a little advanced for me to write from scratch given my perl experience. Appreciate any inputs on the modules that I may use to accomplish this task or atleast any ideas to write the code efficiently given there are high volumes of records in each file.

Again I am not expecting one of you to write code for me although I would not complain about it :-)

Thanks! Monk

In reply to Recommendations for perl modules to work on data sets ? by raghuprasad241

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.