Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^5: Faster regex to split a string into runs of similar characters?

by vr (Curate)
on Nov 22, 2016 at 14:14 UTC ( #1176338=note: print w/replies, xml ) Need Help??


in reply to Re^4: Faster regex to split a string into runs of similar characters?
in thread Faster regex to split a string into runs of similar characters?

I'm not getting any speed gain with this method, for 'real' image. Not sure, why to expect this gain, if instead of matching for sequence of non-zero bytes we just trying to match for non-zero byte followed by sequence of zeroes? And there's additional work to do: xor-ing, chopping, etc.

Plus, what if object "32" happens to be in 1st column?

$s = join '', map { chr } qw/ 32 0 1 1 0 0 2/; $t = ' ' . $s ^ $s; chop $t; while ( $t =~ m[[^\0]\0*]g ) { printf "%d\t%d\t%d\n", ord( substr $s, $-[0], 1 ), $-[0], $+[0] - +1; }

It looks we have to prepend a zero byte (because, it seems, original eily's solution was strictly for alphabetical strings). Then speed drops below that of "buk3" - loop spends lot of time finding bbox of background. Inserting "next unless $c", leads to, again, the same performance as "buk3".

Well, for now, "buk3"'s speed is "good enough" (amazingly good compared to 'elegant' PDL-only solution), thank you everyone who contributed.

  • Comment on Re^5: Faster regex to split a string into runs of similar characters?
  • Download Code

Replies are listed 'Best First'.
Re^6: Faster regex to split a string into runs of similar characters?
by BrowserUk (Patriarch) on Nov 22, 2016 at 14:52 UTC
    Not sure, why to expect this gain, if instead of matching for sequence of non-zero bytes we just trying to match for non-zero byte followed by sequence of zeroes?

    Mostly because it avoids the need for capturing parens (required for the back reference and needed if objects can abut rather than being separated by 0 bytes as you described). My crude image generator doesn't guarantee at least one zero byte between objects.

    That is probably the difference between your profiling on your real images and mine against my simulated ones.

    And there's additional work to do: xor-ing, chopping, etc.

    xoring strings is a very fast operation; it forms the basis of many of my own faster algorithms operating on strings. chop simply decrements a single internal integer.

    Plus, what if object "32" happens to be in 1st column?

    Good call. For your data, using a null byte instead of a space would be important.

    Well, for now, "buk3"'s speed is "good enough" (amazingly good compared to 'elegant' PDL-only solution)

    In the end, you choose what works for your application. For me, this took on a life of its own because it is so similar to several applications I've written in the past; and will undoubtedly need to revisit in the future. I've made something of a career out of finding ways to speed up pure perl solutions to problems over the last few years, and each time something like this comes up here on PM, I like to throw the guts of the problem out to room, because it invariably throws up new approaches and new twists on old ones, that become useful down the line to myself and others alike.

    The basis of the gain of the buk3 approach (over your original) is simply to allow the regex engine to skip over runs of similar characters at each iteration, rather than discovering them one byte at a time. But the need to use a back reference to make that happen -- in my slight redefinition of the problem where null bytes between objects isn't guaranteed -- forces the use of capturing brackets which imposes a significant cost.

    eily's approach uses the fast xor operation to transmogrify the data such that (even without the null bytes) no back references or capturing parens are required.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
    In the absence of evidence, opinion is indistinguishable from prejudice.
      For your data, using a null byte instead of a space would be importeant.

      Actually that's why I chose /.\0*/gs instead of /[^\0]\0*/s. This means that the first character can be a 0, and every iteration after that it will never start with null because of the way regex work. Or technically, the idea crossed my mind, I realized it would still work by chance, and then forgot to mention it :)

        Or technically, the idea crossed my mind, I realized it would still work by chance, and then forgot to mention it :)

        The significance went right over my head.

        But, I'm really impressed that vr picked up on it. It is so nice to see an OP applying their own thought processes to the posted code, rather than using it verbatim and then blaming us for not having considered every last nuance and detail of their problem, if it goes wrong.

        I really like your offset-and-xor technique. It is similar to a process I've used for edge detection in images in the past.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1176338]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2023-06-01 15:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?