Category: | Text Processing |
Author/Contact Info | |
Description: | we at wiki.perlchina.org translate lots nice english perl articles to chinese. there are a lot of articles submitted with badly formatted style, especially the case when chinese characters and english squeeze together. this is a quick hack to add one space to seperate english words/digits from chinese (before and after chinese characters). so that there will not be {chinese}etc{chinese}, instead will be {chinese} etc {chinese} |
#!/usr/bin/perl -w use strict; my $format; while (<DATA>) { my $line = $_; # this line has alphbet, try to seperate if ($line =~ /[a-zA-Z\d]/) { LINE: while (1) { # (([\x81-\xfe][\x40-\xfe])*) greedy match on chinese. # line starts with chinese if ($line =~ /(([\x81-\xfe][\x40-\xfe])*)/ && $1) { my $e = $1; # update line $line = $'; $format .= ($line=~ /^\s/) ? $e : $e." "; } elsif ($line =~ /(.*?)([\x81-\xfe][\x40-\xfe])/) { # line starts with non-chinese chars my $e = $1; # the second match eats one chinese char, so get it ba +ck. $line = $2.$'; # add space after english char if it isn't ended with +space. $format .= ($e=~/\s$/) ? $e : $e." "; } else { $format .= $line; last LINE; } } } else { $format .= $line; } } print $format; __DATA__ 在 2004 年7月的开源大会时能比CPython更快的执行Python的字节码 “Perl现在生机勃勃 Perl 6被建议不应只作为Perl的新的实现 perl 5.8.x,仍然生机勃勃,Jarkko Hietaniemi今年的早些时候把 在2003年10月,发布了 |
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: format text which mixed english and chinese characters.
by graff (Chancellor) on Feb 20, 2005 at 21:49 UTC | |
by Qiang (Friar) on Feb 21, 2005 at 03:57 UTC |