in reply to malformed UTF-8 character in JSON string

The server is returning data in a rather strange format which is encoded JSON data inside a JSON structure. But the character encoding is only done once.

decode_json includes a UTF-8 decoding step. Since your code is using it twice, it is trying to do the UTF-8 decode twice in addition to the JSON decoding.

If you replace the second decode_json call with a from_json call, it should avoid the additional broken character decoding. It may be preferable to change all of the JSON parsing calls with from_json for consistency, and do the UTF-8 decoding yourself before doing JSON parsing. This can be done with decode_utf8 from Encode.

Replies are listed 'Best First'.
Re^2: malformed UTF-8 character in JSON string
by cavac (Prior) on Jul 31, 2023 at 06:03 UTC

    JSON within a JSON gives a lot of trouble regarding UTF-8/Unicode with the JSON and JSON::XS modules. I pretty much resigned to ban this constellation from all my projects. No matter what i tried, i couldn't get it to work.

    The problem seems to stem from the fact that the first (outer) decode also decoded the UTF-8 from the inner JSON, which made it an invalid JSON string for the second (inner) decode.

    PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP