some try and fail and I got solution myself:

$line=~/<a href=\"([^<br>]*?)\.htm\">/ig;

I don't see how the code in your OP did not work, it seems to me it should. And you're saying that your new latest code matches what you want (but I don't think it does that for the right reason, it is probably a happy coincidence -- more on this below). So, there isn't much more help to be provided, since your problem is solved.

I would suggest however that you probably want to avoid quantifiers such as * or + with the match-all dot (i.e. the .* and .+ patterns) when possible, and even also .*? and .+?, although these latter two are much less dangerous in terms of matching more than what you want.

It is often safer and better to be more specific on the characters you want to match, using the appropriate character class. In the case in point, using a regex like /<a href=\"(\w*?)\.htm\">/ig would probably be safer, because, with \w+ or \w*, you're guaranteed to match only alphanumeric characters (and possible underscores), so you know for sure that you're not gonna match any tag-opening and tag-closing characters (angle brackets), backslashes, quote marks, etc.

As for your latest code you posted, [^<br>]*? doesn't do what you probably think it does, even though you report that the result happens to be what you want. [^<br>] is a negative character class that matches everything except the following individual characters: < > b r. I doubt somewhat that what you really meant it to be. The fact that it contains the < character (and will therefore stop matching at the first tag-opening character) is probably enough to save your day in this specific case, but be aware that it won't work if the string that you want to retrieve contains any "b" or any "r."

HTH.

Maybe the following program helps you debug the regex:

#!perl -w
use strict;
# use Regexp::Debugger;

for my $line (<DATA>) {
    while ( $line=~/<a href=\"(.*?)\.htm\">/ig ) {
        print "$1\n";
    };
}

__DATA__
<a href="test1.htm"> test1</a><br> <a href="test2.htm"> test2</a><br><
+a href="test3.htm"> test3</a><br>
[download]

For me this outputs:

test1
test2
test3
[download]

okay, once again sorry I am really tired, I stripped my code with all unnecessary stuff and this is what I come up with, the result is very strange for me

$line='<p><a href="test1.htm"> test1</a><br> <a href="test2.htm"> test
+2</a><br> <a href="test3.htm"> test3</a><br> <a href="test4.htm"> tes
+t4</a><br>';

while ( $line=~/<a href=\"(.*?)\.htm\">/ig ) {
    $tmp=$1;

    print "LINE: $line\n";
    print "TMP: $tmp KK\n\n";

    $line=~s/<a href=\"$tmp.htm\">/<a href=\"\/xxx.html\">/i;
}
[download]

on the other hand Regexp::Debugger - amazing tool, thank you. It shows what the problem and it is what I suspected, but I have no idea why is it matching this way and I have no idea how to fix this. As you see in my code I have a s/ replace and after first replace the matching is weird