| Category: | Text Processing |
| Author/Contact Info | |
| Description: | we at wiki.perlchina.org translate lots nice english perl articles to chinese. there are a lot of articles submitted with badly formatted style, especially the case when chinese characters and english squeeze together. this is a quick hack to add one space to seperate english words/digits from chinese (before and after chinese characters). so that there will not be {chinese}etc{chinese}, instead will be {chinese} etc {chinese} |
#!/usr/bin/perl -w
use strict;
my $format;
while (<DATA>) {
my $line = $_;
# this line has alphbet, try to seperate
if ($line =~ /[a-zA-Z\d]/) {
LINE: while (1) {
# (([\x81-\xfe][\x40-\xfe])*) greedy match on chinese.
# line starts with chinese
if ($line =~ /(([\x81-\xfe][\x40-\xfe])*)/ && $1) {
my $e = $1;
# update line
$line = $';
$format .= ($line=~ /^\s/) ? $e : $e." ";
} elsif ($line =~ /(.*?)([\x81-\xfe][\x40-\xfe])/) {
# line starts with non-chinese chars
my $e = $1;
# the second match eats one chinese char, so get it ba
+ck.
$line = $2.$';
# add space after english char if it isn't ended with
+space.
$format .= ($e=~/\s$/) ? $e : $e." ";
} else {
$format .= $line;
last LINE;
}
}
} else {
$format .= $line;
}
}
print $format;
__DATA__
在 2004 年7月的开源大会时能比CPython更快的执行Python的字节码
“Perl现在生机勃勃
Perl 6被建议不应只作为Perl的新的实现
perl 5.8.x,仍然生机勃勃,Jarkko Hietaniemi今年的早些时候把
在2003年10月,发布了
|
|
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: format text which mixed english and chinese characters.
by graff (Chancellor) on Feb 20, 2005 at 21:49 UTC | |
by Qiang (Friar) on Feb 21, 2005 at 03:57 UTC |