Petruchio has asked for the wisdom of the Perl Monks concerning the following question:
Synopsis: I've reported what seems to be a
platform/version-dependant bug in Switch.pm, which
the author cannot reproduce. I'd like some
help in finding where it occurs and where it doesn't.
The
Switch
module is a nifty thing, IMO. It's a source filter,
molesting your code in strange and unspeakable
ways to provide the answer to all those Chatterbox
prayers... a switch statement in Perl. Saith
merlyn in On Parsing Perl, "Perl is extremely
difficult to parse. In fact, some would say
impossible." But the author, Damian Conway, seems to
be trying very hard to parse Perl in Perl.
Switch does not, however, work quite properly...
as Dr. Conway, says in the POD, "There are
undoubtedly serious bugs lurking somewhere
in code this funky :-)".
Quite so. I sent the good Doctor a (rather
strange) bug report, and in his reply he said,
If you want to patch your own copy immediately,
change line 91 from:
$text .= substr($source,$pos[2],$pos[18]-$pos[2]);
to:
$text .= " " . substr($source,$pos[2],$pos[18]-$pos[2]);
Conway gave me permission to share this fix
and others he might send. I suggest that if
you're using the module, you do this now.
Anyway, I found another bug, as strange as
the first. When I reduced my code to this:
#!/usr/bin/perl -w
use strict;
use Switch;
my $x = 2 ? 'foo' : 'bar';
switch('x'){
case 'y' {print "y\n"}
else {print "x\n"}
}
# ?
it generated the following errors:
Bareword "case" not allowed while "strict subs" in use at ./Shell.pl l
+ine
9.
Unquoted string "case" may clash with future reserved word at ./Shell.
+pl
line 9.
String found where operator expected at ./Shell.pl line 9, near "case
+'y'"
(Do you need to predeclare case?)
syntax error at ./Shell.pl line 9, near "case 'y'"
Execution of ./Shell.pl aborted due to compilation errors.
Funny thing is, if you remove either the trinary operator,
or the question mark in the final comment, the program works.
Conway replied:
I can't replicate this bug, but it seems to be an old
Perl parsing problem. Are you sure you have the latest
version of Text::Balanced installed?
In fact, I do; $VERSION = '1.83'; interestingly, in the POD
for *this* module Conway says, "There are undoubtedly serious
bugs lurking somewhere in this code".
I am, however, still using perl 5.005_03. To be more specific,
perl -V says:
Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
Platform:
osname=linux, osvers=2.2.15pre14, archname=i386-linux
uname='linux them 2.2.15pre14 #2 smp mon mar 13 14:29:00 est 2000
+i686 unknown '
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef useperlio=undef d_sfio=undef
Compiler:
cc='cc', optimize='-O2 ', gccversion=2.95.2 20000313 (Debian GNU/L
+inux)
cppflags='-Dbool=char -DHAS_BOOL -D_REENTRANT -DDEBIAN -I/usr/loca
+l/include'
ccflags ='-Dbool=char -DHAS_BOOL -D_REENTRANT -DDEBIAN -I/usr/loca
+l/include'
stdchar='char', d_stdstdio=undef, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+2
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lndbm -lgdbm -ldbm -ldb -ldl -lm -lc -lposix -lcrypt
libc=, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami
+c'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Built under linux
Compiled at Apr 30 2000 12:08:38
@INC:
/usr/lib/perl5/5.005/i386-linux
/usr/lib/perl5/5.005
/usr/local/lib/site_perl/i386-linux
/usr/local/lib/site_perl
/usr/lib/perl5
.
Which should pretty well clue you in as to my setup.
Anyway, I'd like to solicit a little help; if you
could run my little script on a platform unlike my
own and report the result, as well as the output
of perl -v, I'd appreciate it. You'll
need to edit what you get from d/l code a bit, since
I've been obliged to use code tags for formatting the
snippets, too. And, naturally, don't forget to apply
the aforementioned patch.
I strongly suspect Conway is running a newer version
of perl than I am, and I expect to find the answer
there... but I cannot say for certain.
Thank you!
Re: Help Debug Switch.pm!
by btrott (Parson) on Apr 23, 2001 at 10:35 UTC
|
Okay, I went through and did some debugging. I'm running in
5.005_03, like you, so I got the same error. I stepped through
the code using print statements, and it looks like the error is
basically that, in Switch.pm, when the filter is running over the
program source, it can't find any case statements.
Which means that the source isn't getting filtered properly,
and so the filtered source ends up with the same case statements
that are in your original source--but case isn't a valid Perl
bareword, so that's why you get that error.
But that's simply the symptom, really.
The *real* problem is this: Text::Balanced::_match_quotelike
is being called, and it matches from the first '?' to the last (the
one in the comment), and it sees that as a valid regex. In case
this doesn't quite make sense, it sees this:
? 'foo' : 'bar';
switch('x'){
case 'y' {print "y\n"}
else {print "x\n"}
}
# ?
Question marks ('?') are valid regex delimiters; valid in the
sense that you don't need a 'm' in front of them. They're like
'/'. You can just say:
if ($foo =~ ?bar?)
That's legal.
Here's a runthrough of what's going on, essentially; it makes
the most sense if you follow along in the original source.
- Switch.pm receives your program source, which looks like
this:
my $x = 2 ? 'foo' : 'bar';
switch('x'){
case 'y' {print "y\n"}
else {print "x\n"}
}
# ?
It steps through that source, matching against regular
expressions, in the form /\Gpattern/gc. This means that each
time the regex engine tries to do a match, it starts in the
source string *after* the last successful match. So using pos,
I tried to figure out what it was doing.
- The first match is at the end of the lexing loop. It's supposed
to match what looks like whitespace followed by non-whitespace,
and it matches this:
my
That's three characters: a newline, then 'my'. (I realize it's
hard to see the newline. :)
This puts pos at 3.
- The next thing it matches is done using
Text::Balanced::_match_variable, and it matches:
$x
Which is also three characters: a space, then '$x'. This puts pos
at 6.
- After this we go through a few other match attempts, then
are back down to the non-whitespace followed by whitespace,
which matches:
=
Which is two chars: a space, followed by '='. This puts pos at
8.
- Go through again, get back to non-whitespace followed
by whitespace. We match:
2
Which is two chars: a space, then '2'. This puts pos at 10.
- All of this matching is going on in a loop, and each time
through the loop we're trying to match against a bunch of
other things, as well. One rather important thing is each time
we try to match switch/case statements. We haven't done so
yet, because we've been doing other things. :)
We've also been trying to match "things that look like quotes",
using Text::Balanced::_match_quotelike. Nothing has matched
yet, but the next time we go through, we do:
In _match_quotelike we first match the '$pre' condition, which is
optional whitespace. This moves pos to 11. The next thing in
the string is '?': _match_quotelike looks at this and enters
an if condition, looking to see if this could be a regex match.
It looks to see if there's another '?' in the string--and there is!
It's hidden in a comment, but _match_quotelike doesn't notice,
and so the "quote" matched is:
? 'foo' : 'bar';
switch('x'){
case 'y' {print "y\n"}
else {print "x\n"}
}
# ?
This puts pos at 124. _match_quotelike returns this match,
and this puts us at the end of the string: we're done.
- Since we're done, that means we didn't match any of the
switch/case statements, which means that they got through
unfiltered. They get passed directly to Perl, which doesn't like
them, hence the error.
So this is my theory/observation on what's happening. I
haven't checked yet on 5.6 to see why it's working there; I
really have no idea why.
But it seems to fit pretty well.
UPDATE: I just tested this on Perl 5.6.0, and I got
errors there, as well. So it looks like it's matching in 5.6.0 as
a '?'-delimited regex as well. The error is slighty different:
syntax error at t.pl line 9, near "){"
String found where operator expected at t.pl line 10, near "case 'y'"
(Do you need to predeclare case?)
Execution of t.pl aborted due to compilation errors.
But it's certainly the same idea: it doesn't understand what
either switch or case is, which means that they didn't get
filtered properly. And stepping through using the same debugging
statements shows me that it's matching the same things in
5.005_03 as in 5.6.0. | [reply] [d/l] [select] |
(jeffa) Re: Help Debug Switch.pm!
by jeffa (Bishop) on Apr 23, 2001 at 09:34 UTC
|
ran it without the last line and it worked - put the
'commented' question mark in there and RAAALPH . . .
i have no idea, but i did try this at the end:
=for comment
why does this break?
=cut
and same error - WTF?^H
oh yeah - running version Perl 5.005_03 with version 1.83
of Text::Balanced
Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
Platform:
osname=linux, osvers=2.2.5-22smp, archname=i386-linux
uname='linux porky.devel.redhat.com 2.2.5-22smp #1 smp wed jun 2 0
+9:11:51 edt 1999 i686 unknown '
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef useperlio=undef d_sfio=undef
Compiler:
cc='cc', optimize='-O2 -m486 -fno-strength-reduce', gccversion=egc
+s-2.91.66 19990314/Linux (egcs-1.1.2 release)
cppflags='-Dbool=char -DHAS_BOOL -I/usr/local/include'
ccflags ='-Dbool=char -DHAS_BOOL -I/usr/local/include'
stdchar='char', d_stdstdio=undef, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+2
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -ldl -lm -lc -lposix -lcrypt
libc=, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami
+c'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Built under linux
Compiled at Feb 2 2000 15:35:58
@INC:
/usr/lib/perl5/5.00503/i386-linux
/usr/lib/perl5/5.00503
/usr/lib/perl5/site_perl/5.005/i386-linux
/usr/lib/perl5/site_perl/5.005
.
R-R-R--R-R-R--R-R-R--R-R-R--R-R-R--
L-L--L-L--L-L--L-L--L-L--L-L--L-L--
| [reply] [d/l] [select] |
(tye)Re: Help Debug Switch.pm!
by tye (Sage) on Apr 23, 2001 at 18:40 UTC
|
The hardest part about parsing Perl is that you have to track both entries in the symbol table and token types in the input stream in order to decide when / is division vs. a regular expression and when ? is the trinary operator vs. a regular expression.
I seriously doubt that Switch.pm will ever solve this problem completely for Perl5 simply because the effort involved is more than a sane person would do.
Anyway, I wanted to strongly discourage the use of such complicated magic for any serious (a.k.a. production) scripts. You could easily find no problem with it at first and then make a minor change and find yourself either making "voodoo" changes (like remove '?'s from comments) in order to make Switch.pm happy or having to do a major rewrite to remove your dependence on that module.
-
tye
(but my friends call me "Tye")
| [reply] |
Re: Help Debug Switch.pm!
by grinder (Bishop) on Apr 23, 2001 at 12:15 UTC
|
I've played with Switch.pm (version 2.01)
but have pretty much given up on it
for production work. It works well in the small but when I
use it in large script (>600 lines) I get weird errors,
usually involving code several lines after a switch
statement, containing a single-quoted string. The parser
complains about barewords, which in turn is because it
thought it had an single-quoted string open and so tries
to interpret the string's contents as Perl.
I'll see if I can dig up an example of this different
error message. In the meantime:
% perl petruchio
syntax error at petruchio line 8, near "){"
String found where operator expected at petruchio line 9, near "case 'y'"
(Do you need to predeclare case?)
Execution of petruchio aborted due to compilation errors.
% perl -MText::Balanced -le 'print $Text::Balanced::VERSION'
1.83
% perl -V
Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
Platform:
osname=solaris, osvers=2.6, archname=sun4-solaris
uname='sunos ouessant 5.6 generic_105181-05 sun4u sparc sunw,ultra-4 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
useperlio=undef d_sfio=undef uselargefiles=define
use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
Compiler:
cc='gcc', optimize='-O', gccversion=2.8.1
cppflags='-I/usr/local/include'
ccflags ='-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
stdchar='unsigned char', d_stdstdio=define, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, usemymalloc=y, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib '
libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
libs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under solaris
Compiled at Sep 26 2000 18:27:15
[...]
-- g r i n d e r
| [reply] |
Re: Help Debug Switch.pm!
by jmcnamara (Monsignor) on Apr 23, 2001 at 12:13 UTC
|
As requested here is the salient output from perl -V with Switch 2.01 and Text::Balaced 1.83:
Summary of my perl5 (revision 5.0 version 6 subversion 0) configuratio
+n:
Platform:
osname=linux, osvers=2.2.17-compact, archname=i686-linux-thread-mu
+lti
uname='linux sickle 2.2.17-compact #1 sun jun 25 01:10:11 mst 2000
+ i686 unknown '
config_args='-des -Dcc=gcc -Dcf_email=ActivePerl@ActiveState.com -
+Uinstallusrbinperl -Dusethreads -Duseithreads -Dprefix=/usr/local/Act
+ivePerl-5.6'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define usemulti
+plicity=define
useperlio=undef d_sfio=undef uselargefiles=define
use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=u
+ndef
Compiler:
cc='gcc', optimize='-O2', gccversion=2.95.2 20000220 (Debian GNU/L
+inux)
cppflags='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include'
ccflags ='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include -
+D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
stdchar='char', d_stdstdio=define, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+2
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksize=8
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -ldl -lm -lpthread -lc -lposix -lcrypt
libc=/lib/libc-2.1.3.so, so=so, useshrplib=false, libperl=libperl.
+a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami
+c'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL
+_IMPLICIT_CONTEXT
Locally applied patches:
ActivePerl Build 618
Built under linux
Compiled at Sep 12 2000 17:00:49
@INC:
/usr/local/ActivePerl-5.6/lib/5.6.0/i686-linux-thread-multi
/usr/local/ActivePerl-5.6/lib/5.6.0
/usr/local/ActivePerl-5.6/lib/site_perl/5.6.0/i686-linux-thread-mu
+lti
/usr/local/ActivePerl-5.6/lib/site_perl/5.6.0
/usr/local/ActivePerl-5.6/lib/site_perl
The error message is slightly different but it amounts to the same thing:
String found where operator expected at switch.pl line 9, near "ca
+se 'y'"
(Do you need to predeclare case?)
syntax error at switch.pl line 8, near "){"
Execution of switch.pl aborted due to compilation errors.
Update: I also checked this on Windows and got the same results. At first I couldn't find STDERR but I eventually found it redirected to a Word document in a folder called "My Favourite Errors". I also noticed that the Paper Clip had become more agitated than usual:
[Clip]: Cleep!! Cleep cleep cleep cleep!
[John]: What is it Clip? What is it girl?
[Clip]: Cleep! Cleep cleep cleep. Cleep cleep!
[John]: What! Uninitialised variable stuck under a tractor?
[Clip]: Cleep!
John.
--
| [reply] [d/l] [select] |
Re: Help Debug Switch.pm!
by Brovnik (Hermit) on Jun 14, 2001 at 13:58 UTC
|
Here is another bit of code that hits a similar problem.
#! /usr/local/bin/perl -w
use 5.006;
use strict;
use warnings;
use Switch;
my $exit = 0;
sub shell
{
print $/; # Error goes away if you remove this line, or change to "
+\n";
while(1)
{
switch ("cmd")
{
case /^quit/ { $exit = 1; }
case /^help/ { help(); }
}
last if $exit;
}
}
I just got bitten by this. BTW, this is a much cut down version of the code, but gives
the same error.
I guess we'll just have to wait for Perl6 (RFC here).
-- Brovnik | [reply] [d/l] |
|
|