re + utf8 problem

maard has asked for the wisdom of the Perl Monks concerning the following question:

Greetings.

My script works well under perl interpreter, but dies when compiled with perl2exe (from Indigo Star). The message is: "Can't locate utf8.pm in @INC ...".

The error appears in HTML::Template module at the point where regular expression is applied to template string. The template string, passed to the module is decoded into perl's internal form with either of commands:
open(my $fh, "<:encoding($enc)", $filename);
or:
decode($content, $enc, 1);
and then the filehandle/scalarref is passed to HTML::Template->new.

When I don't decode (or when I use 2-argument open) there's no error, but output from template is displayed in wrong encoding. Input template can be in one of several encodings, so I want user to decide which encoding to use, rather then force them to use single encoding. That's why I need 3-argument open (or manual decoding).

I know that "use utf8" can't help, but I tried to include it in script and recompile the binary... It didn't help.

The only info I found about utf8 and regexps was in perlunicode: regexp engine creates 2 versions of compiled regexp - for byte and character strings.

It seems to me that perl for some reason tries to convert regexp into internal form (why it needs utf8 module then?), but fails to find it.
What's the solution? What module shuold I explicitly "use" in my script for this error to disappear? Or maybe there's deeper explanation & solution?

perl: 5.8.4

Comment on re + utf8 problem Select or Download Code

Replies are listed 'Best First'.
Re: re + utf8 problem by tachyon (Chancellor) on Sep 27, 2004 at 09:59 UTC
Try one of more of: `#perl2exe_include "utf8.pm" use Encode::Unicode; use bytes;` [download] cheers tachyon	[reply] [d/l]
Re^2: re + utf8 problem by maard (Pilgrim) on Sep 27, 2004 at 10:59 UTC
When I include utf8 either via "use" or via #perl2exe_include, I get lots of following warnings: `Use of uninitialized value in pattern match (m//) at PERL2EXE_STORAGE/ +utf8_heavy.pl line 185, <$fh> line 1. Use of uninitialized value in scalar assignment at PERL2EXE_STORAGE/ut +f8_heavy.pl line 201, <$fh> line 1. ... Use of uninitialized value in scalar assignment at PERL2EXE_STORAGE/ut +f8_heavy.pl line 317, <$fh> line 1. Use of uninitialized value in pattern match (m//) at PERL2EXE_STORAGE/ +utf8_heavy.pl line 318, <$fh> line 1.` [download] and finally HTML::Template dies with: `panic: swash_fetch at PERL2EXE_STORAGE/HTML/Template.pm line 1910, <$f +h> line 1.` [download] , so probably including utf8 in script isn't a good idea?	[reply] [d/l] [select]
Re^2: re + utf8 problem by maard (Pilgrim) on Sep 27, 2004 at 10:20 UTC
Hello. Thanks for reply. Yes, I "use" lots of modules in my script, some of which include: `use bytes; use Encode::Byte; use Encode::Unicode;` [download] Without "use bytes" compiled binary doesn't work at all. Everythins works without errors except pattern matches in HTML::Template module, when script is in perl's internal encoding of scalar. So maybe there's some other detail whick I didn't notice? Or probably some extracts from my script could help? I already told, I "use" some modules, which perl autoloads itself when not compiled. Maybe something else?	[reply] [d/l]
A reply falls below the community's threshold of quality. You may see it by logging in.


Think about Loose Coupling
	PerlMonks