Hm. By way of experiment, I tweaked your code to read from a file:open DATA, '<', $ARGV[ 0 ] or die $!;

I then generated some random data:

c:\test>head 854367-11m.dat
aaaaa  5,12,11,15
aaaab  23,26,12,27,17
aaaac  18,1,5
aaaad  20
aaaae  29,6,6,31,4
aaaaf  27,11,29,4,3
aaaag  12,4,6,28
aaaah  19,29,18,26,21
aaaai  11,22,6
aaaaj  8

c:\test>tail 854367-11m.dat
zzzyv  26
zzzyw  7
zzzyx  29,17,14
zzzyy  26,6,24,21,3,25
zzzyz  9,24,30
zzzza  5,26,3,17,28
zzzzb  16,1,6,14,9,9
zzzzc  22,2,1,28,7,21
zzzzd  17
zzzze  2

11/08/2010  18:23             1,787 854367-100.dat
11/08/2010  18:15            17,311 854367-1000.dat
11/08/2010  18:15           174,511 854367-10000.dat
11/08/2010  18:15         1,747,330 854367-100000.dat
11/08/2010  18:15       207,709,322 854367-11m.dat
11/08/2010  18:23             3,532 854367-200.dat
11/08/2010  18:36            34,777 854367-2000.dat
11/08/2010  19:02            52,181 854367-3000.dat

[download]

And ran a few smaller sets: 

c:\test>854367 854367-1000.dat >nul
Check mem, cpu & time: Terminating on signal SIGINT(2)
c:\test>854367 854367-2000.dat >nul
Check mem, cpu & time: Terminating on signal SIGINT(2)
c:\test>854367 854367-3000.dat >nul
Check mem, cpu & time: Terminating on signal SIGINT(2)

[download]

The results:

  1. N        Mem(MB)         Time(secs)        CPU(billion cycles)
  2.  1000    178             267                 622
  3.  2000    678             1105               2671
  4.  3000    1501            2595               5970 ## Updated with actual timings.
  5. 10000   14000            n/avail.          n/avail.

What do you recon: O(N2) for time and O(N3) memory?

In reply to Re: Gay Dating; or, Thrifty Set Intersections with an Accumulator by BrowserUk
in thread Gay Dating; or, Thrifty Set Intersections with an Accumulator by Xiong

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.