Hi Monks,

I would like your advice on how to store multi-lingual text in some HTML data. Due to other (legacy) constraints I have to fit this into a single database field.

My thoughts were along the lines of using XML tags <locale lang="xx"></locale> to mark up the language specific areas. This seems to work ok.

The code for processing this to extract a single language also needs to back out to the most appropriate language if the first choice is not available.

I have code that works and I have included a sample to show what I mean but it lacks eligence and requires two parse passing of the HTML if the first choice language is unavailable.

All suggestions and comments gratefully recieved as this one has me at a loss.

Thanks
UnderMine 

#!/usr/bin/perl -w

use strict;
use HTML::Parser;

my $langs={};
my $in_tag = 0;
my $tags={};
our $textout='';

my $start = sub {
    my ($tag, $attr, $text) = @_;
    our ($lang, $textout);
    if ($tag eq 'locale') {
        $langs->{$attr->{lang}}=1; # mark languages found
        if ($attr->{lang} eq $lang) {
            $textout.=$text;
        $in_tag=0; # override if already in locale tag 
    } else {
        $in_tag=1;
    }
    } else {
        $textout.=$text;
    }
};

my $end = sub {
    my ($tag, $attr, $text) = @_;
    our $textout;
    if ($tag eq 'locale' and $in_tag) {
    $in_tag=0;
    } else {
        $textout.=$text;
    }
};


my $p = HTML::Parser->new(
   default_h => [ sub { $textout.=shift unless $in_tag }, 'text'],
   start_h   => [ $start , 'tagname, attr, text'],
   end_h     => [ $end, 'tagname, attr, text'],
);


# Order of preference for languages
my $acceptable = [qw{ en fr de it }];

my $data;
while (<DATA>) {
  $data.=$_;
}

$textout.='';
$langs={};
foreach our $lang (@$acceptable) {
   next if (scalar keys %$langs && !(exists $langs->{$lang}));
   $textout='';
   $p->parse($data);
   last if (scalar keys %$langs && (exists $langs->{$lang}));
} 

print $textout."\n";

__DATA__
<html>
<body>
<locale lang="en">Some English</locale>
<locale lang="fr">Some French</locale>
<locale lang="de">Some German</locale>
</body>
</html>

[download]

In reply to Extracting appropriate language text from HTML data by UnderMine

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.