Regex For Removing Emoji

Beaker has asked for the wisdom of the Perl Monks concerning the following question:

Someone has just started throwing emoji into text in my site, For the most part I don't mind but I want to strip it from article and thread titles. Either to change them to plain text completely, or use a regex to strip the emoji specifically. After looking into this and trying out a bunch of regex I found on stack overflow, it seems there's many types of emoji and I couldn't get any of these to work.

$text =~ s/[\x00-\x1F\x80-\xFF]+\s*//g;

$text =~ s/\xEE[\x80-\xBF][\x80-\xBF]|\xEF[\x81-\x83][\x80-\xBF]+\s*//
+g;

$text =~ s/([0-9#][\x{20E3}])|[\x{00ae}\x{00a9}\x{203C}\x{2047}\x{2048
+}\x{2049}\x{3030}\x{303D}\x{2139}\x{2122}\x{3297}\x{3299}][\x{FE00}-\
+x{FEFF}]?|[\x{2190}-\x{21FF}][\x{FE00}-\x{FEFF}]?|[\x{2300}-\x{23FF}]
+[\x{FE00}-\x{FEFF}]?|[\x{2460}-\x{24FF}][\x{FE00}-\x{FEFF}]?|[\x{25A0
+}-\x{25FF}][\x{FE00}-\x{FEFF}]?|[\x{2600}-\x{27BF}][\x{FE00}-\x{FEFF}
+]?|[\x{2900}-\x{297F}][\x{FE00}-\x{FEFF}]?|[\x{2B00}-\x{2BF0}][\x{FE0
+0}-\x{FEFF}]?|[\x{1F000}-\x{1F6FF}][\x{FE00}-\x{FEFF}]|[\x{D83E}][\x{
+DD13}-\x{DD14}]?+\s*//g;

$text =~ s/([0-9|#][\x{20E3}])|[\x{00ae}\x{00a9}\x{203C}\x{2047}\x{204
+8}\x{2049}\x{3030}\x{303D}\x{2139}\x{2122}\x{3297}\x{3299}][\x{FE00}-
+\x{FEFF}]?|[\x{2190}-\x{21FF}][\x{FE00}-\x{FEFF}]?|[\x{2300}-\x{23FF}
+][\x{FE00}-\x{FEFF}]?|[\x{2460}-\x{24FF}][\x{FE00}-\x{FEFF}]?|[\x{25A
+0}-\x{25FF}][\x{FE00}-\x{FEFF}]?|[\x{2600}-\x{27BF}][\x{FE00}-\x{FEFF
+}]?|[\x{2900}-\x{297F}][\x{FE00}-\x{FEFF}]?|[\x{2B00}-\x{2BF0}][\x{FE
+00}-\x{FEFF}]?|[\x{1F000}-\x{1F6FF}][\x{FE00}-\x{FEFF}]?+\s*//g
[download]

Comment on Regex For Removing Emoji Download Code

Replies are listed 'Best First'.
Re: Regex For Removing Emoji by Corion (Patriarch) on Nov 12, 2016 at 14:12 UTC
Also see Text::Unidecode and especially for sanitizing titles for URLs, Text::CleanFragment. Both err rather on the side of leaving things out rather than keeping things in. It seems your regular expressions attempt to remove whole Unicode character planes. Personally, I would explicitly allow some character planes or look at the unicode properties (maybe via Unicode::Tussle to find out whether a character is part of a script. Also consider what you want to do with character art: (╯°□°）╯︵ ┻━┻	[reply]
Re^2: Regex For Removing Emoji by Beaker (Beadle) on Nov 12, 2016 at 17:02 UTC
Thanks I will check out those modules you mentioned. I do a lot of manual text sanitization and manipulation so I should probably try and "third party" some of it.	[reply]
Re: Regex For Removing Emoji by LanX (Saint) on Nov 12, 2016 at 13:46 UTC
> found on stack overflow Where this? Did you ask there? Does it mean you are cross posting now? > and I couldn't get any of these to work. what does this mean? Could you provide sample input to test? Personally I would restrict with a positive filter (white listing) what does it help to block emojis and allow arbitrary Unicode to appear? Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Je suis Charlie!}	[reply]
Re^2: Regex For Removing Emoji by Beaker (Beadle) on Nov 12, 2016 at 14:18 UTC
Sample input 🦂 🍷✨⛓ 🔪🐷 ⏳🎣 Shows full color icons in firefox and chrome, white only in IE and some of the icons in Safari, also white only.	[reply]
Re^3: Regex For Removing Emoji by LanX (Saint) on Nov 12, 2016 at 14:59 UTC
Better dump the $text your application handles, before and after applying the regexes. I suspect the data you are processing doesn't look like it is supposed to do. (Probably Html escaped) Anyway, in general providing us with a complete test script showing input and output would help you and us. Often errors are spotted by the authors when preparing a post like this. Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Je suis Charlie!} Fwiw, for those interested the monastery transforms the provided emojis 🍷✨⛓ 🔪🐷 ⏳🎣 into `🍷✨⛓ 🔪🐷 ⏳🎣` within code blocks	[reply] [d/l]
Re^4: Regex For Removing Emoji by Beaker (Beadle) on Nov 12, 2016 at 17:00 UTC
Re^4: Regex For Removing Emoji (browser) by tye (Sage) on Nov 14, 2016 at 20:05 UTC
Re^2: Regex For Removing Emoji by Beaker (Beadle) on Nov 12, 2016 at 14:17 UTC
No I didn't post on stack overflow, I just found similar questions posted and regex provided by response (in other languages).	[reply]


Keep It Simple, Stupid
	PerlMonks