++ Discipulus for the single regex. Please note that regex for parsing XML is not always the best idea, although there are some exceptions. I'm not so sure about your problem, and maybe considering a XML module would be a good idea.

Anyway, what you need here is more context, either what's before your number, after, or both to decide what to do with your digits. I think the easiest is to use look-ahead assertions, because they are more flexible than look-behind, and I think easier to understand. I don't know what your input data looks like, so maybe this:

qr{ 
  \b \d{3,4} \b # 3 or 4 digits
  (?!           # not followed by
    </b></a>
  )
}xms;
[download]

The interesting thing about look ahead is that they will peak at what is present after a certain point, but not include it in the match. So with my regex only 1234 (for example) will match, though perl will have checked that no </b></a> is present just after.

It won't work if the numbers you want to change are in links with another format (ex, no <b> or more nested tags). In which case you might want to start checking that the first link tag after your number is an opening one, and not a closing one. But at this point it would really start to look that regex are not the best solution, so while you can try it, I highly recommend considering another way.