lgn8412 has asked for the wisdom of the Perl Monks concerning the following question:

Hello again, I've been having this problem with a formfu created page (through a YAML file), I add the elements like anyone would, except for one of them:

    - type: Text
      name: username
      label: Nombre

    - type: Password
      name: password
      label: Contraseña

[download]

As you can see, the ñ character is spanish, and should be handled correctly, turns out that in the webpage it displays a � instead. I've narrowed down the problem and think that it has to do with the YAML file and the subroutine that loads the config file, any suggestions would be greatly appreciated, thanks in advance

Replies are listed 'Best First'.
Re: problem with UTF-8/YAML/Formfu
by Corion (Patriarch) on Jan 04, 2011 at 14:00 UTC

    Find out whether (and how) YAML allows an encoding specified for its files. Also make sure your file actually is in the encoding it should be.

    As a third thing, also make sure that your page is actually served in the encoding you think it should be and has the matching Content-Type header.

    Update: Googling for YAML encoding claims that all YAML files are implied to be UTF-8 if there is no BOM, so you either found a bug in whatever YAML library you use (likely) or your file is not UTF-8 (also likely). Check both.

[reply]
[d/l]

      Thanks for the quick response, I've investigated a bit about this, so I've got this

      1. Taken from HTML::FormFu::Manual::Unicode: "If you're using YAML config files, your files will automatically be decoded by load_config_file|HTML::FormFu/load_config_file and load_config_filestem|HTML::FormFu/load_config_filestem."

      2. My YAML file is encoded as UTF-8

      3. My webpage has the charset set to UTF-8 and that doesn't fix it.

      My guess is that somewhere in the yaml file that the formfu decodes and encodes (I think), somewhere in the load_config_file|HTML::FormFu/load_config_file has to be a encode or decode option that needs to be set to UTF-8

      So the real question is.. is there a way to get into the load_config_file thingie to set some options like the one I need?

[reply]

        My webpage has the charset set to UTF-8 and that doesn't fix it.

        Did you actually encode your webpage using UTF-8?

        When UTF-8 is expected, I would expect exactly that output if the one outputs U+00F1 as byte F1 instead of bytes C3 B1.

        In a UTF-8 terminal: 

        $ perl -E'say "\N{U+00F1}";'
?          <--- Actually U+FFFD, what you posted

$ perl -MEncode -E'say encode("UTF-8", "\N{U+00F1}");'
ñ

        [download]

        If it was an lack of decoding on input as you suspect, one would get multiple gibberish characters rather than the coding error indicated by the character you posted. 

        $ perl -MEncode -E'say encode("UTF-8", "\xC3\xB1");'
ñ

        [download]

        It sounds to me like you are properly decoding the text on input while failing to encode it on output.

[reply]
[d/l]
[select]

        Hi, if you're not using Catalyst and setting everything advised in HTML::FormFu::Manual::Unicode, you need to read the whole document, and apply the same principles to your project. In particular, see the "HTML sent to the browser" section of that page.

        btw, I only noticed a couple days ago that DBIx::Class::UTF8Columns is no longer advised, so that part of the manual will need updating - in the meantime, you should see the DBIx::Class docs for details.

[reply]
Re: problem with UTF-8/YAML/Formfu
by Your Mother (Archbishop) on Jan 04, 2011 at 16:53 UTC

    Possible solution: Make sure you have the most up to date YAML libs. I don't know when (what version) the cutoff is but in the past there were definitely bugs here.

[reply]

Back to Seekers of Perl Wisdom