Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
Not sure, why to expect this gain, if instead of matching for sequence of non-zero bytes we just trying to match for non-zero byte followed by sequence of zeroes?

Mostly because it avoids the need for capturing parens (required for the back reference and needed if objects can abut rather than being separated by 0 bytes as you described). My crude image generator doesn't guarantee at least one zero byte between objects.

That is probably the difference between your profiling on your real images and mine against my simulated ones.

And there's additional work to do: xor-ing, chopping, etc.

xoring strings is a very fast operation; it forms the basis of many of my own faster algorithms operating on strings. chop simply decrements a single internal integer.

Plus, what if object "32" happens to be in 1st column?

Good call. For your data, using a null byte instead of a space would be important.

Well, for now, "buk3"'s speed is "good enough" (amazingly good compared to 'elegant' PDL-only solution)

In the end, you choose what works for your application. For me, this took on a life of its own because it is so similar to several applications I've written in the past; and will undoubtedly need to revisit in the future. I've made something of a career out of finding ways to speed up pure perl solutions to problems over the last few years, and each time something like this comes up here on PM, I like to throw the guts of the problem out to room, because it invariably throws up new approaches and new twists on old ones, that become useful down the line to myself and others alike.

The basis of the gain of the buk3 approach (over your original) is simply to allow the regex engine to skip over runs of similar characters at each iteration, rather than discovering them one byte at a time. But the need to use a back reference to make that happen -- in my slight redefinition of the problem where null bytes between objects isn't guaranteed -- forces the use of capturing brackets which imposes a significant cost.

eily's approach uses the fast xor operation to transmogrify the data such that (even without the null bytes) no back references or capturing parens are required.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^6: Faster regex to split a string into runs of similar characters? by BrowserUk
in thread Faster regex to split a string into runs of similar characters? by BrowserUk

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (5)
As of 2023-12-02 12:27 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (16 votes). Check out past polls.