Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Help Debug Switch.pm!

by Petruchio (Vicar)
on Apr 23, 2001 at 08:19 UTC ( [id://74641]=perlquestion: print w/replies, xml ) Need Help??

Petruchio has asked for the wisdom of the Perl Monks concerning the following question:

Synopsis: I've reported what seems to be a platform/version-dependant bug in Switch.pm, which the author cannot reproduce. I'd like some help in finding where it occurs and where it doesn't.

The Switch module is a nifty thing, IMO. It's a source filter, molesting your code in strange and unspeakable ways to provide the answer to all those Chatterbox prayers... a switch statement in Perl. Saith merlyn in On Parsing Perl, "Perl is extremely difficult to parse. In fact, some would say impossible." But the author, Damian Conway, seems to be trying very hard to parse Perl in Perl.

Switch does not, however, work quite properly...

as Dr. Conway, says in the POD, "There are undoubtedly serious bugs lurking somewhere in code this funky :-)".

Quite so. I sent the good Doctor a (rather strange) bug report, and in his reply he said,

If you want to patch your own copy immediately, change line 91 from:

$text .= substr($source,$pos[2],$pos[18]-$pos[2]);

to:

$text .= " " . substr($source,$pos[2],$pos[18]-$pos[2]);

Conway gave me permission to share this fix and others he might send. I suggest that if you're using the module, you do this now.

Anyway, I found another bug, as strange as the first. When I reduced my code to this:

#!/usr/bin/perl -w use strict; use Switch; my $x = 2 ? 'foo' : 'bar'; switch('x'){ case 'y' {print "y\n"} else {print "x\n"} } # ?

it generated the following errors:

Bareword "case" not allowed while "strict subs" in use at ./Shell.pl l +ine 9. Unquoted string "case" may clash with future reserved word at ./Shell. +pl line 9. String found where operator expected at ./Shell.pl line 9, near "case +'y'" (Do you need to predeclare case?) syntax error at ./Shell.pl line 9, near "case 'y'" Execution of ./Shell.pl aborted due to compilation errors.

Funny thing is, if you remove either the trinary operator, or the question mark in the final comment, the program works.

Conway replied:

I can't replicate this bug, but it seems to be an old Perl parsing problem. Are you sure you have the latest version of Text::Balanced installed?

In fact, I do; $VERSION = '1.83'; interestingly, in the POD for *this* module Conway says, "There are undoubtedly serious bugs lurking somewhere in this code".

I am, however, still using perl 5.005_03. To be more specific, perl -V says:

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration: Platform: osname=linux, osvers=2.2.15pre14, archname=i386-linux uname='linux them 2.2.15pre14 #2 smp mon mar 13 14:29:00 est 2000 +i686 unknown ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef useperlio=undef d_sfio=undef Compiler: cc='cc', optimize='-O2 ', gccversion=2.95.2 20000313 (Debian GNU/L +inux) cppflags='-Dbool=char -DHAS_BOOL -D_REENTRANT -DDEBIAN -I/usr/loca +l/include' ccflags ='-Dbool=char -DHAS_BOOL -D_REENTRANT -DDEBIAN -I/usr/loca +l/include' stdchar='char', d_stdstdio=undef, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +2 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldbm -ldb -ldl -lm -lc -lposix -lcrypt libc=, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami +c' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Characteristics of this binary (from libperl): Built under linux Compiled at Apr 30 2000 12:08:38 @INC: /usr/lib/perl5/5.005/i386-linux /usr/lib/perl5/5.005 /usr/local/lib/site_perl/i386-linux /usr/local/lib/site_perl /usr/lib/perl5 .

Which should pretty well clue you in as to my setup. Anyway, I'd like to solicit a little help; if you could run my little script on a platform unlike my own and report the result, as well as the output of perl -v, I'd appreciate it. You'll need to edit what you get from d/l code a bit, since I've been obliged to use code tags for formatting the snippets, too. And, naturally, don't forget to apply the aforementioned patch.

I strongly suspect Conway is running a newer version of perl than I am, and I expect to find the answer there... but I cannot say for certain.

Thank you!

Replies are listed 'Best First'.
Re: Help Debug Switch.pm!
by btrott (Parson) on Apr 23, 2001 at 10:35 UTC
    Okay, I went through and did some debugging. I'm running in 5.005_03, like you, so I got the same error. I stepped through the code using print statements, and it looks like the error is basically that, in Switch.pm, when the filter is running over the program source, it can't find any case statements.

    Which means that the source isn't getting filtered properly, and so the filtered source ends up with the same case statements that are in your original source--but case isn't a valid Perl bareword, so that's why you get that error.

    But that's simply the symptom, really.

    The *real* problem is this: Text::Balanced::_match_quotelike is being called, and it matches from the first '?' to the last (the one in the comment), and it sees that as a valid regex. In case this doesn't quite make sense, it sees this:

    ? 'foo' : 'bar'; switch('x'){ case 'y' {print "y\n"} else {print "x\n"} } # ?
    Question marks ('?') are valid regex delimiters; valid in the sense that you don't need a 'm' in front of them. They're like '/'. You can just say:
    if ($foo =~ ?bar?)
    That's legal.

    Here's a runthrough of what's going on, essentially; it makes the most sense if you follow along in the original source.

    • Switch.pm receives your program source, which looks like this:
      my $x = 2 ? 'foo' : 'bar'; switch('x'){ case 'y' {print "y\n"} else {print "x\n"} } # ?
      It steps through that source, matching against regular expressions, in the form /\Gpattern/gc. This means that each time the regex engine tries to do a match, it starts in the source string *after* the last successful match. So using pos, I tried to figure out what it was doing.

    • The first match is at the end of the lexing loop. It's supposed to match what looks like whitespace followed by non-whitespace, and it matches this:
      my
      That's three characters: a newline, then 'my'. (I realize it's hard to see the newline. :) This puts pos at 3.
    • The next thing it matches is done using Text::Balanced::_match_variable, and it matches:
      $x
      Which is also three characters: a space, then '$x'. This puts pos at 6.
    • After this we go through a few other match attempts, then are back down to the non-whitespace followed by whitespace, which matches:
      =
      Which is two chars: a space, followed by '='. This puts pos at 8.
    • Go through again, get back to non-whitespace followed by whitespace. We match:
      2
      Which is two chars: a space, then '2'. This puts pos at 10.
    • All of this matching is going on in a loop, and each time through the loop we're trying to match against a bunch of other things, as well. One rather important thing is each time we try to match switch/case statements. We haven't done so yet, because we've been doing other things. :)

      We've also been trying to match "things that look like quotes", using Text::Balanced::_match_quotelike. Nothing has matched yet, but the next time we go through, we do:

      In _match_quotelike we first match the '$pre' condition, which is optional whitespace. This moves pos to 11. The next thing in the string is '?': _match_quotelike looks at this and enters an if condition, looking to see if this could be a regex match. It looks to see if there's another '?' in the string--and there is! It's hidden in a comment, but _match_quotelike doesn't notice, and so the "quote" matched is:

      ? 'foo' : 'bar'; switch('x'){ case 'y' {print "y\n"} else {print "x\n"} } # ?
      This puts pos at 124. _match_quotelike returns this match, and this puts us at the end of the string: we're done.
    • Since we're done, that means we didn't match any of the switch/case statements, which means that they got through unfiltered. They get passed directly to Perl, which doesn't like them, hence the error.
    So this is my theory/observation on what's happening. I haven't checked yet on 5.6 to see why it's working there; I really have no idea why.

    But it seems to fit pretty well.

    UPDATE: I just tested this on Perl 5.6.0, and I got errors there, as well. So it looks like it's matching in 5.6.0 as a '?'-delimited regex as well. The error is slighty different:

    syntax error at t.pl line 9, near "){" String found where operator expected at t.pl line 10, near "case 'y'" (Do you need to predeclare case?) Execution of t.pl aborted due to compilation errors.
    But it's certainly the same idea: it doesn't understand what either switch or case is, which means that they didn't get filtered properly. And stepping through using the same debugging statements shows me that it's matching the same things in 5.005_03 as in 5.6.0.
(jeffa) Re: Help Debug Switch.pm!
by jeffa (Bishop) on Apr 23, 2001 at 09:34 UTC
    ran it without the last line and it worked - put the 'commented' question mark in there and RAAALPH . . .

    i have no idea, but i did try this at the end:

    =for comment why does this break? =cut
    and same error - WTF?^H

    oh yeah - running version Perl 5.005_03 with version 1.83 of Text::Balanced

    Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration: Platform: osname=linux, osvers=2.2.5-22smp, archname=i386-linux uname='linux porky.devel.redhat.com 2.2.5-22smp #1 smp wed jun 2 0 +9:11:51 edt 1999 i686 unknown ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef useperlio=undef d_sfio=undef Compiler: cc='cc', optimize='-O2 -m486 -fno-strength-reduce', gccversion=egc +s-2.91.66 19990314/Linux (egcs-1.1.2 release) cppflags='-Dbool=char -DHAS_BOOL -I/usr/local/include' ccflags ='-Dbool=char -DHAS_BOOL -I/usr/local/include' stdchar='char', d_stdstdio=undef, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +2 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -ldl -lm -lc -lposix -lcrypt libc=, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami +c' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Characteristics of this binary (from libperl): Built under linux Compiled at Feb 2 2000 15:35:58 @INC: /usr/lib/perl5/5.00503/i386-linux /usr/lib/perl5/5.00503 /usr/lib/perl5/site_perl/5.005/i386-linux /usr/lib/perl5/site_perl/5.005 .

    R-R-R--R-R-R--R-R-R--R-R-R--R-R-R--
    L-L--L-L--L-L--L-L--L-L--L-L--L-L--
    
(tye)Re: Help Debug Switch.pm!
by tye (Sage) on Apr 23, 2001 at 18:40 UTC

    The hardest part about parsing Perl is that you have to track both entries in the symbol table and token types in the input stream in order to decide when / is division vs. a regular expression and when ? is the trinary operator vs. a regular expression.

    I seriously doubt that Switch.pm will ever solve this problem completely for Perl5 simply because the effort involved is more than a sane person would do.

    Anyway, I wanted to strongly discourage the use of such complicated magic for any serious (a.k.a. production) scripts. You could easily find no problem with it at first and then make a minor change and find yourself either making "voodoo" changes (like remove '?'s from comments) in order to make Switch.pm happy or having to do a major rewrite to remove your dependence on that module.

            - tye (but my friends call me "Tye")
Re: Help Debug Switch.pm!
by grinder (Bishop) on Apr 23, 2001 at 12:15 UTC
    I've played with Switch.pm (version 2.01) but have pretty much given up on it for production work. It works well in the small but when I use it in large script (>600 lines) I get weird errors, usually involving code several lines after a switch statement, containing a single-quoted string. The parser complains about barewords, which in turn is because it thought it had an single-quoted string open and so tries to interpret the string's contents as Perl.

    I'll see if I can dig up an example of this different error message. In the meantime:

    % perl petruchio
    syntax error at petruchio line 8, near "){"
    String found where operator expected at petruchio line 9, near "case 'y'"
            (Do you need to predeclare case?)
    Execution of petruchio aborted due to compilation errors.
    % perl -MText::Balanced -le 'print $Text::Balanced::VERSION'
    1.83
    
    % perl -V
    Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
      Platform:
        osname=solaris, osvers=2.6, archname=sun4-solaris
        uname='sunos ouessant 5.6 generic_105181-05 sun4u sparc sunw,ultra-4 '
        config_args=''
        hint=recommended, useposix=true, d_sigaction=define
        usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
        useperlio=undef d_sfio=undef uselargefiles=define
        use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
      Compiler:
        cc='gcc', optimize='-O', gccversion=2.8.1
        cppflags='-I/usr/local/include'
        ccflags ='-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
        stdchar='unsigned char', d_stdstdio=define, usevfork=false
        intsize=4, longsize=4, ptrsize=4, doublesize=8
        d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
        ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
        alignbytes=8, usemymalloc=y, prototype=define
      Linker and Libraries:
        ld='gcc', ldflags =' -L/usr/local/lib '
        libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
        libs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
        libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
      Dynamic Linking:
        dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
        cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib'
    
    
    Characteristics of this binary (from libperl):
      Compile-time options: USE_LARGE_FILES
      Built under solaris
      Compiled at Sep 26 2000 18:27:15
      [...]
    

    --
    g r i n d e r
Re: Help Debug Switch.pm!
by jmcnamara (Monsignor) on Apr 23, 2001 at 12:13 UTC

    As requested here is the salient output from perl -V with Switch 2.01 and Text::Balaced 1.83:
    Summary of my perl5 (revision 5.0 version 6 subversion 0) configuratio +n: Platform: osname=linux, osvers=2.2.17-compact, archname=i686-linux-thread-mu +lti uname='linux sickle 2.2.17-compact #1 sun jun 25 01:10:11 mst 2000 + i686 unknown ' config_args='-des -Dcc=gcc -Dcf_email=ActivePerl@ActiveState.com - +Uinstallusrbinperl -Dusethreads -Duseithreads -Dprefix=/usr/local/Act +ivePerl-5.6' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemulti +plicity=define useperlio=undef d_sfio=undef uselargefiles=define use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=u +ndef Compiler: cc='gcc', optimize='-O2', gccversion=2.95.2 20000220 (Debian GNU/L +inux) cppflags='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include' ccflags ='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include - +D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' stdchar='char', d_stdstdio=define, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +2 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='gcc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -ldl -lm -lpthread -lc -lposix -lcrypt libc=/lib/libc-2.1.3.so, so=so, useshrplib=false, libperl=libperl. +a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynami +c' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL +_IMPLICIT_CONTEXT Locally applied patches: ActivePerl Build 618 Built under linux Compiled at Sep 12 2000 17:00:49 @INC: /usr/local/ActivePerl-5.6/lib/5.6.0/i686-linux-thread-multi /usr/local/ActivePerl-5.6/lib/5.6.0 /usr/local/ActivePerl-5.6/lib/site_perl/5.6.0/i686-linux-thread-mu +lti /usr/local/ActivePerl-5.6/lib/site_perl/5.6.0 /usr/local/ActivePerl-5.6/lib/site_perl
    The error message is slightly different but it amounts to the same thing:
    String found where operator expected at switch.pl line 9, near "ca +se 'y'" (Do you need to predeclare case?) syntax error at switch.pl line 8, near "){" Execution of switch.pl aborted due to compilation errors.

    Update: I also checked this on Windows and got the same results. At first I couldn't find STDERR but I eventually found it redirected to a Word document in a folder called "My Favourite Errors". I also noticed that the Paper Clip had become more agitated than usual:
    [Clip]: Cleep!! Cleep cleep cleep cleep! [John]: What is it Clip? What is it girl? [Clip]: Cleep! Cleep cleep cleep. Cleep cleep! [John]: What! Uninitialised variable stuck under a tractor? [Clip]: Cleep!


    John.
    --

Re: Help Debug Switch.pm!
by Brovnik (Hermit) on Jun 14, 2001 at 13:58 UTC
    Here is another bit of code that hits a similar problem.
    #! /usr/local/bin/perl -w use 5.006; use strict; use warnings; use Switch; my $exit = 0; sub shell { print $/; # Error goes away if you remove this line, or change to " +\n"; while(1) { switch ("cmd") { case /^quit/ { $exit = 1; } case /^help/ { help(); } } last if $exit; } }
    I just got bitten by this. BTW, this is a much cut down version of the code, but gives the same error.

    I guess we'll just have to wait for Perl6 (RFC here).
    --
    Brovnik

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://74641]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (3)
As of 2024-04-18 19:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found