from perlre:

A "\w" matches a single alphanumeric character (an alphabetic 
character, or a decimal digit) or "_"...
[download]

Thus your earlier use of [^\w\d] had the set of digits in it twice, which suggested to me that you thought that \w means [A-Za-z].

[^\w\d] works, but is redundant and equivalent to \W

Update: You asked what the speed difference between the two passes and one pass:

s/(?:^\W+)|(?:\W+$)//g;
# versus
s/\W+$//g;
s/^\W+//g;

# my unscientific benchmark
                Rate  single_pass    two_pass
single_pass  15829/s           --        -11%
two_pass     17737/s          12%          --
[download]

Doing it in two passes seems to be about 10-15% faster.