I was trying to find the performance of Perl's hash insert in practice (as i assumed it was O(1) or very near) and hit this A short meditation about hash search performance . Though it's 20 years old almost. Avoiding the dereferencing of the array as choroba suggests will speed things up especially because you will eliminate this round-about way:

                my @urlList=@{$ipURL{$ip}};
                push (@urlList,$url);
                $ipURL{$ip}=\@urlList;

[download]

I may be wrong on how I think Perl handles the above but I will say it creates a new array every time and copies the old one in there. Whereas push @{$ipURL{$ip}}, $url; "magically" doesn't, but it pushes into the reference so-to-speak. As the arrays grow, that's likely the cause of your exponential degradation in performance. You may also find that pushing into an array is slower than inserting into a hash and so you can find johngg's suggestion useful.

bw, bliako

In reply to Re: Hash Search is VERY slow by bliako
in thread Hash Search is VERY slow by rtjensen

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.