in reply to Re: Unrecognized ICU conversion error
in thread Unrecognized ICU conversion error

It's fine to update your post; however, it's important to indicate that you've done so — especially when your update invalidates an existing response. See "How do I change/delete my post?" for more about that.

I also note that all lines of your log extracts end with " <br>". I suspect this doesn't reflect the original and were probably added initially to format the log data for paragraph text.

I am aware that this was your first post here. My comments are intended to be informational; not any kind of rebuke. :-)

— Ken

Replies are listed 'Best First'.
Re^3: Unrecognized ICU conversion error [after your updates]
by Sainathuni (Novice) on Aug 11, 2023 at 21:53 UTC
    'Unrecognized ICU conversion error' disappeared when the vertica client was upgraded to 23.3.0; However the 'Wide character in print at XXXX line XX' is appearing at line 137 in addition to line 951. This indeed looks like Unicode/UTF8 issue, appreciate any help providing solution.

      [Identity: I am getting a little confused regarding with whom I'm conversing: ewcarroll or Sainathuni. If these are two separate people, perhaps working on the same project, please advise. If these are two usernames registered by the same person, please read "Site Rules Governing User Accounts" and action appropriately. Thankyou.]

      "This indeed looks like Unicode/UTF8 issue, appreciate any help providing solution."

      My two previous posts in this thread were purely to help a new user. I have no knowledge of Vertica Analytic Database, vertica-client or UL_VERTICA.pm; furthermore, I have no access to CentOS or Fedora Linux systems. I can provide the following, very general, potential solution (on a Cygwin system which was fully updated yesterday).

      $ uname -a CYGWIN_NT-10.0-19045 titan 3.4.7-1.x86_64 2023-06-16 14:04 UTC x86_64 +Cygwin

      I created the module Dummy_Vertica intended to emulate what I think might be in UL_VERTICA:

      $ cat /home/ken/tmp/pm_11153782_unicode/Dummy_Vertica.pm package Dummy_Vertica; use v5.36; use utf8; sub test_wide_character_print { say 'SLAP example:'; print "SLAPØ\n"; say 'IMP example:'; print "\N{IMP}\n"; return; } 1;

      [IMP (U+01F47F) was chosen completely arbitrarily as a wide character. See the Unicode® PDF Code Chart "Miscellaneous Symbols and Pictographs" for further details.]

      I wrote the following script, test_1.pl, to emulate the type of thing you're seeing:

      $ cat /home/ken/tmp/pm_11153782_unicode/test_1.pl #!/usr/bin/env perl use v5.36; BEGIN { say "Perl version: $^V"; } use lib '/home/ken/tmp/pm_11153782_unicode'; use Dummy_Vertica; Dummy_Vertica::test_wide_character_print();

      Output:

      ken@titan ~/tmp/pm_11153782_unicode
      $ ./test_1.pl
      Perl version: v5.36.0
      SLAP example:
      SLAP▒
      IMP example:
      Wide character in print at /home/ken/tmp/pm_11153782_unicode/Dummy_Vertica.pm line 10.
      👿
      

      As you can see, there's two types of problems: instead of Ø; and, the "Wide character ..." message. Both can be resolved by using the open pragma. Here's test_2.pl which is identical to test_1.pl except for the addition of one "use open ..." statement:

      $ cat /home/ken/tmp/pm_11153782_unicode/test_2.pl #!/usr/bin/env perl use v5.36; use open OUT => qw{:encoding(UTF-8) :std}; BEGIN { say "Perl version: $^V"; } use lib '/home/ken/tmp/pm_11153782_unicode'; use Dummy_Vertica; Dummy_Vertica::test_wide_character_print();

      Output:

      ken@titan ~/tmp/pm_11153782_unicode
      $ ./test_2.pl
      Perl version: v5.36.0
      SLAP example:
      SLAPØ
      IMP example:
      👿
      

      — Ken

      As there are many scripts, we are looking for a solution at configuration/high level than in the script itself. Thank you!

        As i've said before in Re^2: Unrecognized ICU conversion error and kcott explained much better in Re^4: Unrecognized ICU conversion error [after your updates], your problems seem to stem from bugs in your perl scripts (improper unicode handling), not from the Perl interpreter itself. There simply is "no high level configuration" to work around the bugs in your software.

        There are basically three ways to fix the problems with your code:

        1. Fix the bugs yourself.
        2. Contact whoever wrote the code to have them fix it.
        3. Open Source the whole thing and hope it is useful enough to someone that they spend their spare time to fix your code for free.

        Short of that, you could just make sure that no Unicode enters your system in the first place.

        PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP