Re^6: Faster regex to split a string into runs of similar characters?by BrowserUk (Patriarch)
|on Nov 22, 2016 at 14:52 UTC||Need Help??|
Not sure, why to expect this gain, if instead of matching for sequence of non-zero bytes we just trying to match for non-zero byte followed by sequence of zeroes?
Mostly because it avoids the need for capturing parens (required for the back reference and needed if objects can abut rather than being separated by 0 bytes as you described). My crude image generator doesn't guarantee at least one zero byte between objects.
That is probably the difference between your profiling on your real images and mine against my simulated ones.
And there's additional work to do: xor-ing, chopping, etc.
xoring strings is a very fast operation; it forms the basis of many of my own faster algorithms operating on strings. chop simply decrements a single internal integer.
Plus, what if object "32" happens to be in 1st column?
Good call. For your data, using a null byte instead of a space would be important.
Well, for now, "buk3"'s speed is "good enough" (amazingly good compared to 'elegant' PDL-only solution)
In the end, you choose what works for your application. For me, this took on a life of its own because it is so similar to several applications I've written in the past; and will undoubtedly need to revisit in the future. I've made something of a career out of finding ways to speed up pure perl solutions to problems over the last few years, and each time something like this comes up here on PM, I like to throw the guts of the problem out to room, because it invariably throws up new approaches and new twists on old ones, that become useful down the line to myself and others alike.
The basis of the gain of the buk3 approach (over your original) is simply to allow the regex engine to skip over runs of similar characters at each iteration, rather than discovering them one byte at a time. But the need to use a back reference to make that happen -- in my slight redefinition of the problem where null bytes between objects isn't guaranteed -- forces the use of capturing brackets which imposes a significant cost.
eily's approach uses the fast xor operation to transmogrify the data such that (even without the null bytes) no back references or capturing parens are required.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
In the absence of evidence, opinion is indistinguishable from prejudice.