in reply to iso-8859-1 code converter

You can still get at kanjiworld.com via the wayback machine, but alas, only a few pages are archieved. Still a complete EUC-JP table is on the internet.

That said, it is not clear to me what you are trying to do, since there is no equivalent in iso-8859-1 for EUC-JP (in bytes, two bytes are just two bytes). There's only a unicode equivalent, which is straight forward to get from the table you refer to. E.g. の as HTML entity is written as の converting the numeric part to hex you get the unicode point: printf "%x", 12398 gives 306e. The perl unicode representation as string would be "\x{306e}" which is stored internally as a sequence of three char values: 227, 129, 174: 

use HTML::Entities;
use Devel::Peek;
$c = decode_entities("&#12398");
Dump $c
__END__
SV = PV(0x91ecb00) at 0x91ebcdc
  REFCNT = 1
  FLAGS = (POK,pPOK,UTF8)
  PV = 0x91ff608 "\343\201\256"\0 [UTF8 "\x{306e}"]
  CUR = 3
  LEN = 8

[download]

The EUC-JP encodng for の is the two byte sequence a4ce (ascii 164,206 in decimal), but how is that iso-8859-1? On my terminal that renders as the Euro-Sign+Capital-I-Circumflex. So what is the sought outcome? what are you actually trying to do?

Thinking about it, your "somewhat strange request" sounds like a XY question...

Replies are listed 'Best First'.
Re^2: iso-8859-1 code converter
by GaijinPunch (Pilgrim) on May 05, 2009 at 10:26 UTC
    Cheers for the responses, guys.

    Okay, I didn't know the iso-8859-1/HTML representation is actually the Unicode representation. The overall goal or whatever is this:
    I've got a list of games I slurped off of Amazon.JP. I need their English equivalent names, but of course, want the computer to do the work for me. play-asia.com is a pretty good source. If you search for the Japanese name, the first hit is usually it, and the default display is it's English title (bingo). The caveat is that the URL uses this 5-digit representation for each string. What I did before when I needed to do something similar for a site who used a legit Japanese encoding, was simply examined each character, and got it's corresponding code (utf8 or euc) from a hash populated by one of the tables on the net, much like the one you linked.

    So, the search link for "任天堂" (Nintendo) is
    http://www.play-asia.com/paOS-19-71-6-49-en-15-%26%2320219%3B%26%2322825%3B%26%2322530%3B-43-6.html They use %26%23 as a character breaker, FYI.

    Your solution seems to be in the right direction, but I'm not quite sure how it comes full circle.

    -GP
[reply]

      It's not clear to me from your example what data you have got and what you would like to convert. If I unserstand you right, you already have as input data:

      1. The search link
      2. The Japanese name of the game (任天堂), as a sequence of unicode code points (20219, 22825, 22530)
      3. The English translation of this name (Nintendo)
      Given this data, what is the result you would like to have?

      -- 
      Ronald Fischer <ynnor@mm.st>
[reply]
[d/l]
        I only have the Japanese (and thus, the Unicode and/or EUC-JP encoding). I don't have the English. I was just giving that one as an example. I will get() a page from the search link, but I can't pass it a JPN-friendly encoding (ironically).
[reply]
      So you have a string like this: "任天堂", and you have a template for submitting a query containing that sort of string, which is: 
      http://www.play-asia.com/paOS-19-71-6-49-en-15-{your_string_here}-43-6
+.html

      [download]
      (I hope I'm parsing that correctly, but I have to say it looks a bit implausible.)

      And in order to plug that sort of string into that query template, you have to:

      1. break the string into separate characters;
      2. convert each character into its decimal numeric Unicode code point value
      3. embed each numeric character value between "&#" and ";"
      4. do the "uri encoding" that turns "&", "#" and ";" into "%26", "%23" and "%3B", respectively

      So, where's the problem? Have you tried something like this? 

      my $url = http://www.play-asia.com/paOS-19-71-6-49-en-15-XXX-43-6.html

my $string = "...";  # wherever you get your Unicode Asian string from

$string =~ s/([^[:ascii:]])/"%26%23".ord($1)."%3B"/eg;

$url =~ s/XXX/$string/;

      [download]
[reply]
[d/l]
[select]
        >>rovf
        "Or did I misundertand you here completely?"

        Yes. I will get that from the page listed. I need the ASCII representation to do so though, and that's what I'm after. >>graff
        That seems to be on the right track. Never knew what ord() did. However, as before, I'm using utf8 or euc encoded strings, so ord returns the numeric representation of each byte, not each character. The above gives 6 distinct codes, not 3. My assumption is if I passed it iso-8859-1 encoded strings it might work, but my text editor (Kate on Linux) says they're not valid for that encoding, and chokes.

        Wait, the values are actually just HTML representation (like © is the copyright sign). I guess I should just either plug my values into a calculator like below, or search for the table somwhere.
        http://www.pinyin.info/tools/converter/chars2uninumbers.html

        Sorry my confusion led to such a lengthy discussion. :?
[reply]
        Okay, how do you know when you're using one or the other?

        I change the character encoding in Kate. I actually copy, then change encoding, and then paste over.

        6 numerics instead of 3 for that 3-character string definitely means "not utf8" (so presumably euc, based on what you've said);

        You are right -- EUC gives 3, and utf8 gives 6. However, the ones EUC gives are not the ones I need. :)

        And what do you mean by "the values are actually just HTML representation"?

        Sorry, that sounds stupid. They are the decimal form of Unicode, which HTML can read as &#XXXXX;You can type in the above string into this little engine and have it spit out the result.
        http://www.pinyin.info/tools/converter/chars2uninumbers.html

        Are you really still having a problem with this?

        I think I've got it now, but I still don't know how I can get this decimal value in Perl. If I pass a utf8 string to ord(), it gives me a value for each byte, not the double-byte character. I could do something tacky and pass my string in question to the aforementioned website to get the decimal value, but that seems overkill, to say the least.

[reply]
        graff

        That worked. Thanks!
        Definitely some good nuggets in this thread which went on a bit longer than I had anticipated.

        Thanks to everyone for their suggestions and patience.
[reply]

In Section Seekers of Perl Wisdom