why it stops after a word with a trailing ','

(\w+[.,;\s]?)+ means to match one or more word characters, followed by zero or one [.,;\s] character, and the outer (...)+ means that the next thing must be one or more word characters, and so on. However, in the string "Orinda, which", the , character isn't followed by a word character, it's a space. So it isn't really the difference between comma and space that's causing it to stop, it's more than one non-word character (Update: more precisely, a [.,;\s] character followed by a non-word character) - e.g. .. or ;- have the same effect.

I need to understand how to stop it at that first comma, (it stops when it should not) as well as how to let it run on to some other anchor.

If you wanted to stop it at the first comma, then you could remove the comma from the [.,;\s] group. A common way to say "match until some character", using the example of ;, is [^;]*; - this assumes that those characters can't be escaped.

One test case isn't really enough though, see Re: How to ask better questions using Test::More and sample data.

Update 2: Minor edits for clarity.