You may want to watch out for "unprintable" characters -- there may be delimiters that are not visible (e.g. null bytes, invisible control codes, etc). It would be worthwhile to follow earlier advice -- yank out the easy parts first, push the hard parts to a separate listing -- and then do a more careful diagnosis of the hard parts. A simple hex-dump of the character data can help, or printing some tabulation of byte values that occur in the data (which might make it easier to spot unexpected bytes) -- e.g.:
#!/usr/bin/perl
# chist.perl -- print histogram of byte values
# (useful for seeing if text contains invisible characters)
while (<>)
{
@chars = split //;
for $c ( split // ) {
$chist[ord($c)]++;
}
}
for ( $i=0; $i<256; $i++ ) {
printf("%d\tx%0.2x\n", $chist[$i], $i) if ( $chist[$i] );
}
# there are ways to reduce that to a fairly short one-liner,
# but you may want to add options...