Re^4: Safely removing Unicode zero-width spaces and other non-printing characters

in reply to Re^3: Safely removing Unicode zero-width spaces and other non-printing characters
in thread Safely removing Unicode zero-width spaces and other non-printing characters

Yes, the RSS reads fine of course.

The problem is with the pages which the RSS points to. HTML and XHTML is a hot mess. Even when a respectable CMS is used, the authors can still paste in something weird. It is looking like I may have to treat each site individually and making individual filters might not be worth the effort. However, I am hoping for an automated way to normalize incoming text.

Comment on Re^4: Safely removing Unicode zero-width spaces and other non-printing characters

Replies are listed 'Best First'.
Re^5: Safely removing Unicode zero-width spaces and other non-printing characters by haukex (Archbishop) on Dec 05, 2019 at 05:49 UTC
I am hoping for an automated way to normalize incoming text. Well, my suggestions for guessing encoding still apply, plus looking at the meta tags in the HTML might help (with the same caveat that it might be wrong). But again, for specific help with the specific issue that you wrote about in the root node, you'll have to show us some debug output.	[reply]

In Section Seekers of Perl Wisdom