Illegal character CTRLCHAR code 27

ultranerds has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I'm working on a perl script to insert records into a Solr (lucine) install. However, for some of the records I'm getting an error about a "code 27"

Request error : 400 Illegal character CTRLCHAR code 27 at rowcol unknownsource 1778 at /home/user/domain.com/www/admin/Plugins/Solr/Users/Update.pm line 164.

I've done a little searching on google, and it seems like a "code 27" is the "excape" charachter.

Is there a simple way of filtering out these "control" charachters, so this doesn't even happen again?

At the moment, I'm doing it with a regex + tr{}

        $user->{$col} =~ s/\n+/\./sig;;
        $user->{$col} =~ tr{&[]юбдЮБДгГийхкИЙХКолнОЛНжтрЖТРэшыЭШЫA-Z?!
+;,.╚╩()" '}{   aaaaaacceeeeeeeeiiiiiioooooouuuuuua-z           __};
[download]

...but thats obviously not filtering out those "special charachters" :(

Any suggestions are much appreciated :)

TIA

Andy

Comment on Illegal character CTRLCHAR code 27 Select or Download Code

Replies are listed 'Best First'.
Re: Illegal character CTRLCHAR code 27 by moritz (Cardinal) on May 03, 2010 at 09:14 UTC
ultranerds, You have 108 writeups here at perlmonks - you still don't know how to write a regex/substitution to remove a character from a string? Or if you don't remember how to do it, you don't know how to find the documentation that explains it? Then let me point you at http://perldoc.perl.org/ and hope that you bookmark and/or remember the address.	[reply]
Re^2: Illegal character CTRLCHAR code 27 by ultranerds (Hermit) on May 03, 2010 at 09:19 UTC
Hi, I know how to remove a charachter, but wasn't sure what to use for the control charachter (as I've never had to do that) This seems to do the trick though: $user->{$col} =~ tr/\000-\037/ /; Cheers Andy	[reply]
Re^3: Illegal character CTRLCHAR code 27 by moritz (Cardinal) on May 03, 2010 at 09:26 UTC
I think you're not testing your code properly: `$ perl -wE '$_="\01\02"; say length; tr/\000-\007//; say length' 2 2` [download] Nothing removed. (Update: But replaced, see Re^5: Illegal character CTRLCHAR code 27) Still you're on the right track. For deleting with tr///, you need a trailing /d: `$ perl -wE '$_="\01\02"; say length; tr/\000-\007//d; say length' 2 0` [download] Perl 6 - links to (nearly) everything that is Perl 6.	[reply] [d/l] [select]
Re^4: Illegal character CTRLCHAR code 27 by ultranerds (Hermit) on May 03, 2010 at 09:32 UTC
Re^5: Illegal character CTRLCHAR code 27 by FunkyMonk (Chancellor) on May 03, 2010 at 09:36 UTC
Some notes below your chosen depth have not been shown here
Re: Illegal character CTRLCHAR code 27 by CountZero (Bishop) on May 03, 2010 at 13:28 UTC
You are adressing the symptoms but did you check where exactly those escapes were in your file / records? Perhaps there was a very good reason for them to be there? CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply]


Problems? Is your data what you think it is?
	PerlMonks