There is a BUG in the Windows handling of the %ENV hash since perl-5.18
Specifically if a UTF8 variable is used to delete an entry
an aberrant entry appears in the hash.
Is this bug known?
Can you help find a fix?
Or workaround(s)?
Thank you!
# -*- perl -*-
# badperl   25-Jan-2017 11:04 gweekly

# This script attempts to expose the bug that if a utf8 variable is
# used as the key to delete a hash entry (that doesn't already exist)
# a non-deletable key comes into existence, in violation of the
# documentation for delete that says "Setting a hash element to the
# undefined value does not remove its key, but deleting it does..."
# It also shows that something is definitely wrong, as the hash can
# end up with two identical keys.

# The DATA area shows the results for running this script, making it
# evident the bug was introduced in perl-5.18 and has persisted since.

use strict;
use warnings;
use Devel::Peek; # Dump
use Encode;      # decode

print "Running Perl $]\n";

# Pre-condition
if ( exists $ENV{SAM} ) {
    print "Note: removing SAM=$ENV{SAM} from the environment\n";
    delete $ENV{SAM};
}

my $utf8 = Encode::decode('utf8','SAM');

my $fixed = substr $utf8, 0;

# Confirm $utf8 and $fixed both are eq 'SAM';
die 'utf8 is not SAM'  if $utf8 ne 'SAM';
die 'fixed is not SAM' if $fixed ne 'SAM';
die 'fixed is not utf8' if $fixed ne $utf8;

print "Deleting ENV{\$utf8} where utf8 eq SAM\n";
delete $ENV{$utf8}; # Here badness happens

if ( defined $ENV{SAM} ) {
    die "WRONG: ENV{SAM} is defined: '$ENV{SAM}'\n";
}
if ( defined $ENV{$utf8} ) {
    die "WRONG: ENV{\$utf8} is defined: '$ENV{$utf8}'\n";
}

if ( exists $ENV{SAM} ) {
    die "WRONG: ENV{SAM} exists\n";
}
print exists $ENV{$utf8}  ? "WRONG: ENV{\$utf8} exists\n"
    :                       "OKAY: ENV{\$utf8} does not exist\n";

if ( my @sams = grep {$_ eq 'SAM'} keys %ENV ) {
    if ( @sams > 1 ) {
        die "Surpise: ENV has ". @sams ." SAM keys: @sams\n";
    }
    else {
        print "WRONG: ENV has the key 'SAM' - @sams\n";
    }
    if ( exists $ENV{$fixed} ) {
        die "Surprise: \$ENV{\$fixed} DOES exist\n";
    }
}

print "Now, assign a new value:\n";
$ENV{$utf8}='newVal';

if ( ! defined $ENV{SAM} ) {
    die " ENV{SAM} is not defined\n";
}
print defined $ENV{$utf8} ? "OKAY: ENV{\$utf8} is defined: '$ENV{$utf8
+}'\n"
    :                       "OKAY: ENV{\$utf8} is not defined\n";

if ( ! exists $ENV{SAM} ) {
    die " ENV{SAM} does not exist\n";
}

print exists $ENV{$utf8}  ? "OKAY: ENV{\$utf8} exists\n"
    :                       "OKAY: ENV{\$utf8} does not exist\n";

my @sams = grep {$_ eq 'SAM'} keys %ENV
    or die "No SAM keys?";

if ( @sams > 1 ) {
    print "WRONG: ENV has ". @sams ." SAM keys: @sams\n";
    for ( @sams ) {
        Dump $_;
        my $ans = $ENV{$_} || '<undef>';
        print "  Value = $ans\n";
    }
}

if ( ! exists $ENV{$fixed} ) {
    die "Surprise: \$ENV{\$fixed} does NOT exist\n";
}

print "Now, delete the entry\n";
delete $ENV{$fixed};

if ( exists $ENV{$fixed} ) {
    die "Surprise: \$ENV{\$fixed} does exist\n";
}

print "Done testing\n";

__END__

C:\Users\gweekly>c:\perl\bin\perl i:\bin\badperl
Running Perl 5.008008
Deleting ENV{$utf8} where utf8 eq SAM
OKAY: ENV{$utf8} does not exist
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
Now, delete the entry
Done testing

C:\Users\gweekly>c:\ActivePerl-5.14.2\bin\perl i:\bin\badperl
Running Perl 5.014002
Deleting ENV{$utf8} where utf8 eq SAM
OKAY: ENV{$utf8} does not exist
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
Now, delete the entry
Done testing

C:\Users\gweekly>c:\ActivePerl5.16.3\bin\perl i:\bin\badperl
Running Perl 5.016003
Deleting ENV{$utf8} where utf8 eq SAM
OKAY: ENV{$utf8} does not exist
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
Now, delete the entry
Done testing

C:\Users\gweekly>c:\ActivePerl5.18.1\bin\perl i:\bin\badperl
Running Perl 5.018001
Deleting ENV{$utf8} where utf8 eq SAM
WRONG: ENV{$utf8} exists
WRONG: ENV has the key 'SAM' - SAM
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
WRONG: ENV has 2 SAM keys: SAM SAM
SV = PV(0x2890b2c) at 0x5015b4
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK,UTF8)
  PV = 0x288ea78 "SAM" [UTF8 "SAM"]
  CUR = 3
  LEN = 0
  Value = newVal
SV = PV(0x2890b24) at 0x5015cc
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x288e988 "SAM"
  CUR = 3
  LEN = 0
  Value = newVal
Now, delete the entry
Done testing

C:\Users\gweekly>\\mathworks\hub\win64\apps\bat\perl\latest520\bin\per
+l i:\bin\badperl
Running Perl 5.020002
Deleting ENV{$utf8} where utf8 eq SAM
WRONG: ENV{$utf8} exists
WRONG: ENV has the key 'SAM' - SAM
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
WRONG: ENV has 2 SAM keys: SAM SAM
SV = PV(0x23556c8) at 0x3f1970
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x2361e78 "SAM"
  CUR = 3
  LEN = 0
  Value = newVal
SV = PV(0x23556b8) at 0x3f1910
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK,UTF8)
  PV = 0x2361f68 "SAM" [UTF8 "SAM"]
  CUR = 3
  LEN = 0
  Value = newVal
Now, delete the entry
Done testing

C:\Users\gweekly>c:\Strawberry-perl-5.20.2.1\perl\bin\perl i:\bin\badp
+erl
Running Perl 5.020002
Deleting ENV{$utf8} where utf8 eq SAM
WRONG: ENV{$utf8} exists
WRONG: ENV has the key 'SAM' - SAM
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
WRONG: ENV has 2 SAM keys: SAM SAM
SV = PV(0x25562b8) at 0x5dd698
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x2527e58 "SAM"
  CUR = 3
  LEN = 0
  Value = newVal
SV = PV(0x25562a8) at 0x5dd6c8
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK,UTF8)
  PV = 0x2527db8 "SAM" [UTF8 "SAM"]
  CUR = 3
  LEN = 0
  Value = newVal
Now, delete the entry
Done testing

C:\Users\gweekly>c:\perl64\bin\perl i:\bin\badperl
Running Perl 5.024001
Deleting ENV{$utf8} where utf8 eq SAM
WRONG: ENV{$utf8} exists
WRONG: ENV has the key 'SAM' - SAM
Now, assign a new value:
OKAY: ENV{$utf8} is defined: 'newVal'
OKAY: ENV{$utf8} exists
WRONG: ENV has 2 SAM keys: SAM SAM
SV = PV(0x3ed3d8) at 0x30f488
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK,UTF8)
  PV = 0x30b8b8 "SAM" [UTF8 "SAM"]
  CUR = 3
  LEN = 0
  Value = newVal
SV = PV(0x3ed3e8) at 0x30f470
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x23bc458 "SAM"
  CUR = 3
  LEN = 0
  Value = newVal
Now, delete the entry
Done testing

[download]

In reply to Windows %ENV and utf8 by gweekly

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.