Re^2: This looks like whitespace in my CSV but doesn't seem to be

That gives me:

[
  "My Company",
  "Gavin",
  "Henry\xC2\xA0",
  "ghenry\@ghenry.co.uk",
  "\xC2\xA1.00",
]
[download]

which isn't whitespace after all. Thanks for the Data::Dump tip. I've always used Data::Dumper. Swapping for that produces:

$VAR1 = [
          'My Company',
          'Gavin',
          'Henry ',
          'ghenry@ghenry.co.uk',
          ' 1.00'
        ];
$VAR1 = [
          'My Company',
          'Gavin',
          'Henry ',
          'ghenry@ghenry.co.uk',
          ' 1.00'
        ];
[download]

i.e.

        next unless $fields[4];
        print Dumper \@fields;
        $fields[4] =~ s/\s+//g;
        print Dumper \@fields;
[download]

Walking the road to enlightenment... I found a penguin and a camel on the way.....
Fancy a yourname@perl.me.uk? Just ask!!!

Comment on Re^2: This looks like whitespace in my CSV but doesn't seem to be Select or Download Code

Replies are listed 'Best First'.
Re^3: This looks like whitespace in my CSV but doesn't seem to be by tobyink (Canon) on Sep 30, 2012 at 08:44 UTC
`\xC2\xA0` is a Unicode non-breaking space encoded into UTF-8. If you decode your UTF-8 string into a native Perl Unicode string (see Encode), and then add the `/u` modifier to your regular expression, that ought to enable `\s` to remove non-ASCII whitespace. `perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'`	[reply] [d/l] [select]
Re^4: This looks like whitespace in my CSV but doesn't seem to be by ghenry (Vicar) on Sep 30, 2012 at 08:58 UTC
That will be why Vim isn't allowing me either to do :%s/\s//g in that field. Excellent catch! Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re^4: This looks like whitespace in my CSV but doesn't seem to be by ghenry (Vicar) on Sep 30, 2012 at 09:06 UTC
I need to get a newer perl as I'm on 5.10.1 and /u throws an error. Cheers though, will look how I can do this. Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re^5: This looks like whitespace in my CSV but doesn't seem to be by Anonymous Monk on Sep 30, 2012 at 09:35 UTC
You can always use unicode-regex-range-character-class.pl `space => [\u0009-\u000D\u0020\u0085\u00A0\u1680\u180E\u2000-\u200A\u20 +28-\u2029\u202F\u205F\u3000]` [download] so `$ perl -pe " s{\\u(....)}{\\x{$1}}g " [\u0009-\u000D\u0020\u0085\u00A0\u1680\u180E\u2000-\u200A\u2028-\u2029 +\u202F\u205F\u3000] [\x{0009}-\x{000D}\x{0020}\x{0085}\x{00A0}\x{1680}\x{180E}\x{2000}-\x{ +200A}\x{2028}-\x{2029}\x{202F}\x{205F}\x{3000}]` [download] Thus `#!/usr/bin/perl -- use warnings;use strict; use Data::Dump; $_ = qq{\xC2\xA01.00};; utf8::decode($_); dd[$_]; s{^[\x{0009}-\x{000D}\x{0020}\x{0085}\x{00A0}\x{1680}\x{180E}\x{2000}- +\x{200A}\x{2028}-\x{2029}\x{202F}\x{205F}\x{3000}]+}{}g; dd[$_]; __END__ ["\xA01.00"] ["1.00"]` [download] Although, in 5.10 you could probably just use `s{^\p{space}+}{}g;`	[reply] [d/l] [select]
Re^3: This looks like whitespace in my CSV but doesn't seem to be by Marshall (Canon) on Sep 30, 2012 at 10:12 UTC
I don't work with multi-byte characters or UTF-8 files. But Perl has an amazing ability to deal with these things. From the open() docs, it could be that all you need to do is open the file in the right encoding mode and that could enable even Perl 5.10.X to work. I don't know for sure, but this idea is worth a try. `open(my $fh, "<:encoding(UTF-8)", "filename") \|\| die "can't open UTF-8 encoded filename: $!";` [download]	[reply] [d/l]
Re^4: This looks like whitespace in my CSV but doesn't seem to be by ghenry (Vicar) on Sep 30, 2012 at 12:40 UTC
Of course Marshall!!! That was it, all solved and s/^\s// works as I'd expect. Thanks so much. Very basic of me not to do. Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]
Re^3: This looks like whitespace in my CSV but doesn't seem to be by ghenry (Vicar) on Sep 30, 2012 at 08:42 UTC
Or is that hex for whitespace? Will check... Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!	[reply]