in reply to Re: Hash Search is VERY slow
in thread Hash Search is VERY slow

I think you'll find that it was choroba's advice to process the file on a line-by-line (CSV-record-by-record in this case) basis that did the trick. :)


Give a man a fish:  <%-{-{-{-<

Replies are listed 'Best First'.
Re^3: Hash Search is VERY slow
by NERDVANA (Priest) on Sep 29, 2021 at 22:37 UTC
    I’m going to bet it was bliako’s observation that the array was getting cloned every time an element was added. That’s where the N^2 behavior came from.
      Oh I missed that.

      => Re: Hash Search is VERY slow

      Brilliant!

      > that the array was getting cloned

      I think it's not so much the copying alone but the allocation of a new array each time.

      Because the arrays are growing, old released memory often can't be reused, leading to a fragmentation and increasing waste of space.

      Fun! :)

      Tho I might be wrong, the reserved space for an array grows by doubling IIRC, this should be easier to reuse ...

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery