Re^2: Sanity Check

As far as I can tell these out of range ascii characters are introduced by the user copying from MS Word into a long description. For example

60 TON AIR-COOLED ROTARY SCROLL CHILLER
Carrier
Model no.30RAP060
Chiller Manufactured in Charlotte, NC
&#xF2; AHRI Performance: 56.0 Tons, Full Load &#xFB; 10.2 EER, IPLV &#
+xFB; 14.5
EER
&#xF2; Electrical Requirements for 208/3: MCA &#xFB; 279.7, MOCP &#xFB
+; 300, Rec
$37,855.00 $37,855.00
Quote ref: TII/CA/1214/2895 Page 1 of 5
Fuse Size &#xFB; 300
&#xF2; Electrical Requirements for 460/3: MCA &#xFB; 134.6, MOCP &#xFB
+; 150, Rec
Fuse Size &#xFB; 150
&#xF2; AHRI STANDARD 550/590 CERTIFIED
&#xF2; ASHRAE 90.1 COMPLIANT
&#xF2; 7.5 HP Constant speed, single-pump package (61.3&#xC6; ext. hea
+d @
134 GPM)
[download]

I do not care about the content of the long description. I would much rather have a fully automated solution to keep a human out of the loop. W/O the pre-command... my ETL (Informatica) will not digest the unknown character and aborts then I get a phone call!.

Comment on Re^2: Sanity Check Download Code

Replies are listed 'Best First'.
Re^3: Sanity Check by choroba (Cardinal) on Jan 28, 2015 at 13:42 UTC
I see. What about the entities in non-starting positions, then? `Fuse Size û 150` [download] لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply] [d/l]
Re^4: Sanity Check by kcinmd (Initiate) on Jan 28, 2015 at 15:07 UTC
Good catch. Thank you. Changed and result. `perl -pi -e 's/&#x.+;//g' ./*.xml 60 TON AIR-COOLED ROTARY SCROLL CHILLER Carrier Model no.30RAP060 Chiller Manufactured in Charlotte, NC 14.5 EER 300, Rec $37,855.00 $37,855.00 Quote ref: TII/CA/1214/2895 Page 1 of 5 Fuse Size 300 150, Rec Fuse Size 150 AHRI STANDARD 550/590 CERTIFIED ASHRAE 90.1 COMPLIANT ext. head @ 134 GPM)` [download]	[reply] [d/l]
Re^3: Sanity Check by Anonymous Monk on Jan 28, 2015 at 14:17 UTC
Perhaps it would be better to just skip the lines with encoded chars `perl -ne 'print unless /&#x/'` [download] Stuff like "134 GPM)" is on a separate line; I don't know if that's a problem.	[reply] [d/l]