What you can do however is to pick one encoding, say UTF-8, and consequently apply it to everything:

• Character encoding of your pages
• charset= attribute to the HTTP Content-Type header
• List only UTF-8 in the accept-charset attribute of the input forms
• Use UTF-8 for URL encoding.

I'm not readily seeing how the character-set used to encode the request is supposed to be an issue. As far as I know, it's UTF-8, with HTML-escapes being used to encode all of the special-characters that you might need.

The browser, then, informs the server what character-sets it will accept, whereupon the server either delivers the information as-promised, or informs the client that it cannot be done.

I do not profess to be a wizard on this one.

The request itself is always ASCII - that's the spec.

However the URL-Encoding scheme with %DE%AD%BE%EF encodes only bytes, so you need to pick a character encoding.

When I used latin-1 for this encoding some browsers sent me some requests with latin-1 encoded data, even though the pages themselves were encoded in UTF-8 (and declared as such). So I guess they decoded the URLs and did some encoding guesswork, and used that for GET requests.

Which is why I recommend consistency ;-)

I recommend taking a look at this page, which is a mirror of a (now-defunct but highly useful) original for background: http://niwo.mnsys.org/saved/~flavell/charset/form-i18n.html

In short, you never *really* know exactly what you're getting back from a browser, and the very best belt-and-suspenders thing you can do is to have a "magic cookie" hidden form field on each form that contains a smattering of odd characters with an unambiguous representation in each encoding you suspect. That is, you put a bunch of CJK and other stuff, output as HTML entities, into your form field, and you use a heuristic based upon what the browser sends back to determine what encoding the browser has decided to use.

You can also look at Encode::Guess, which makes some informed guesses based on the content, but doesn't require making the round-trip the way the form field technique does.

But what most people do, and what's probably most tractable unless you're trying to solve a problem for a large, diverse user-base, is to figure out what the top 3-4 user-agents you serve do, and just bank on that...

If you don't trust the browser, I'd try to find a named entity that's represented as a different value in each of the two encodings, and set it as a value in a hidden field.

If there aren't any such characters, you could at least use a character that's in unicode, but not iso-latin, and see what comes back from the browser.

This document describes how a browser can request a document in a desired code. This is how a CGI script should encode its response.

But I am looking for the encoding of the request. How can a CGI script find out in which encoding a request was send from the browser to the server. This is: how is the CGI input encoded.

This is of importance when a request contains form input. Is this form input send utf-8 encoded or iso-8859-1.

Rather: how can a CGI.pm based script find out hoe the input was encoded.

Thanks

Thedi gerber@id.ethz.ch

The RFC specifies how both client/server should behave, and all information is in HTTP headers.