in reply to LWP gives funky characters

Turns up the solution for me was this:
use Encode; require LWP::UserAgent; my $ua = LWP::UserAgent->new; my $response = $ua->get("$ARGV[0]"); $_=$response->content; $_=decode('utf-8',$_); $result=""; for $r (/<div class="result-text-style-normal">\s*(.*?)<\/div>/isg, /< +td\s+class\s*=\s*"?multipassage-box[^>]*>(.*?)<\/td>/isg) { #($r)=/<div class="result-text-style-normal">\s*(.*?)<\/div>/is; $r=~s/<sup>.*?<\/sup>//gi; $r=~s/<h[45]>.*?<\/h[45]>/ /gi; $r=~s/<(.*?)>//g; $r=~s/&nbsp;/ /gi; $r=~s/\s*(.*?)\s*$/$1/s; $r=~s/\s+/ /g; $r=~s/([\x{0080}-\x{ffff}])/'\\u'.sprintf('%04x',ord($1))/ge; $result.= ($result?' ':'') . $r; }
The main points being: (1) I needed to use Encode and decode('utf-8',$_) my response content, and (2) I replaced high characters with their javascript quoted version. This way I can completely avoid unicode in the database and all encoding transmission issues, and just un-quote them in the AJAX client.