My input is a text file that I created by copying the contents of a PDF and pasting to notepad and saving (which dropped unicode). I then FTPd the text file to my Linux box where I am running the code below. I am simply trying to read each line of the text file, split each line into words and load each word into a hash key with a value of '1' attached to it. Running the map version only loads my hash with LOW, RES and PDF while the for loop version seems to work fine. Why? Does it have something to do with the <> in the map?

The input looks like this, for example...

LOW-RES PDF
NOT PRINT-READY
MY BIG TOE
BOOK 1:
A WAKENING
Section 1
Delusion or Knowledge:
Is This Guy Nuts, or What?
Section 2
Mysticism Demystified
The Foundations of Reality
LOW-RES PDF
NOT PRINT-READY
The My Big TOE reality model will help
you understand your life, your purpose,
[download]

This seems to work:

#!/usr/bin/perl -w

use strict;
use Data::Dumper;
my %hash;

for my $line ( <> ) {
    for my $word ( split /(\s+|\W+)/, $line ) {
        chomp $word;
        $hash{$word} = '1';
    }
}

print Dumper(%hash);

dbmathis@bamboo [~/mbt_index]# cat new_ib1.txt | head -100 | ./validat
+ion_poc.pl | head -20
$VAR1 = '';
$VAR2 = '1';
$VAR3 = 'you';
$VAR4 = '1';
$VAR5 = ' ';
$VAR6 = '1';
$VAR7 = 'put';
$VAR8 = '1';
$VAR9 = 'dust';
$VAR10 = '1';
$VAR11 = 'my';
$VAR12 = '1';
$VAR13 = 'delivered';
$VAR14 = '1';
$VAR15 = 'business';
$VAR16 = '1';
$VAR17 = 'power';
$VAR18 = '1';
$VAR19 = 'Seeburg';
$VAR20 = '1';
[download]

But this doesn't:

#!/usr/bin/perl -w

use strict;
use Data::Dumper;
my %hash;
%hash = map { chomp; $_ => '1' } split /(\s+|\W+)/, <>;

print Dumper(%hash);

dbmathis@bamboo [~/mbt_index]# cat new_ib1.txt | head -100 | ./validat
+ion_poc.pl | head -20
$VAR1 = '';
$VAR2 = '1';
$VAR3 = '-';
$VAR4 = '1';
$VAR5 = 'LOW';
$VAR6 = '1';
$VAR7 = ' ';
$VAR8 = '1';
$VAR9 = 'PDF';
$VAR10 = '1';
$VAR11 = 'RES';
$VAR12 = '1';
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.