Apart from the negative character class, there's also negative lookahead:

(?![;:])\W
[download]

Remember that if you want to add quantifiers (like "*" and "+"), you will have to group this:

(?:(![;:])\W)+
[download]

I haven't benchmarked it, but I expect the negative charclass as the other people have shown you, is likely faster.