perl-diddler has asked for the wisdom of the Perl Monks concerning the following question:

I have a bash shell pattern that works, but I can't get it to work when called from perl.

This pattern works in bash shell, but seems remarkably resistant to incorporate into perl --

echo mc-lang-+([^-])-+([^-])*.rpm<br>
(matches mc-lang-4.6.1-140.x86_64.rpm)

a "previous expression, like
echo mc_lang-4.6.1-140*.rpm works from perl with
$a=`echo mc_lang-4.6.1-140*.rpm`
--if the file isn't there, I get back the "*", (unexpanded), otherwise, I get:
mc-lang-4.6.1-140.x86_64.rpm

but the top expression, a bit more precise and what I want fails 13 ways from thursday.

Double quotes, single, adding bash -c 'echo...' in the backquotes, extra backslashes in front of the problematic parens. Part of the prob was I needed to set the extglob option (shopt -s extglob; echo <pattern>).

But every type of backslashed sequence either doesn't expand, or comes up completely empty, or errors pop out.

Why can't perl pass a simple shell pattern and have it expanded!

A work-around that I would prefer not to use that I _think_ would work would fork bash as a process with I/O connected to sockets but that seems like pathetic overkill.

Is there something I'm missing in this morass? Or do I have to go search for a module to debug and modify... -- which isn't really much preferable to launching bash as a co-process...?

Thanks, Linda

Replies are listed 'Best First'.
Re: getting shell expansion to work
by Corion (Patriarch) on Dec 27, 2007 at 09:29 UTC

    If you want "shell" expansion (csh to be exact), the glob function is what you want, or, if you prefer a module and saner expansion, File::Glob. I couldn't follow your description of things you tried, so I won't comment on why they failed for you. If you provide better code, possibly wrapped in <code>...</code> tags, they will be more readable for me and I can maybe tell you where you went wrong.

    You seem to be unclear about where "shell expansion" happens - it does not happen within Perl but within your shell. So, if you want shell expansion Just Like Your Shell Does, you will have to call your shell and ask it to expand the pattern.

    In many cases, reading the Perl documentation of the built-in functions (perlfunc) helps lots.

      Hasn't glob been using File::Glob for some time (somewhere in the 5.6.X releases)?

        Yes.

        Yes, but File::Glob allows you to use "BSD semantics" via bsd_glob, which allow for whitespace in the file specification instead of implying whitespace as the delimiter for multiple specifications.

      Sorry, my note got mangled by the perlmonk-text-input text mangler. I didn't know that square brackets were a reserved HTML character that would be deleted...(I managed to put in the paragraph breaks, but then missed the important "tree" (the expression I was focusing on) due to my focus on the forest (all of my text needing formatting to appear the way I wrote it).

      I'm sorry you are unclear about where I'm intending shell expansion to occur. I'm hoping, ney, desiring that text expansion occur in my shell, as determined by the "$SHELL" variable . I see that's a mistake -- that perl uses the more predictable "/bin/sh". Attempting to expand my expression in the shell was why I used backquotes. Unfortunately, I forgot that Perl ignores "$SHELL and uses /bin/sh (otherwise it would run differently on each end-user (if there were other users using this code)'s system).

      I really should (as you mention) redo the note using code tags to quote things. Le'me rewrite using better text for the input example as well, as the case I selected was poor in that it doesn't show all of my "matching" problems.

        I guess the simplest way is then to be explicit about what you want to happen, instead of hoping for some magic in Perl that you don't know about:

        my $pattern = 'mc-lang-+([^-])-+([^-])*.rpm'; my $results = `$ENV{SHELL} -c echo $pattern`;

        assuming that by $SHELL you mean the environment variable $SHELL and not the Perl variable $SHELL.

Re: getting shell expansion to work
by eserte (Deacon) on Dec 27, 2007 at 10:08 UTC
    Are you sure your example pattern works? I get only a syntax error with my bash version. But this works: echo mc-lang-[^-]*-[^-]*.rpm This pattern also works when used in perl's system().
      Sorry....my example got mangled by the plain-text mangler.

      The RE pattern you use, "mc-lang-[^-]*-[^-]*.rpm" says (converting to perl syntax): /mc-lang-[^-].*-[^-].*.rpm/

      The "*", unlike in perl, isn't an "affix count", but matches any string, including the null string.

      I re-edited the base note to use the code tag to display the bash pattern -- it included square brackets inside of the parens.

      I was trying to use the regular expression syntax from bash to match "one or more" of an item. From the bash man page:

      If the extglob shell option is enabled using the shopt builtin, + several extended pattern matching operators are recognized. In the f +ollowing description, a pattern-list is a list of one or more patterns s +eparated by a |. Composite patterns may be formed using one or more of +the fol- lowing sub-patterns: ?(pattern-list) Matches zero or one occurrence of the given patte +rns *(pattern-list) Matches zero or more occurrences of the given pat +terns +(pattern-list) Matches one or more occurrences of the given patt +erns @(pattern-list) Matches one of the given patterns !(pattern-list) Matches anything except one of the given patterns
      As near as I can tell, perl doesn't have a similar syntax for RE's. It might be better for me to repost the base note with better wording/phrasing and a better example, since the example I used doesn't demonstrate my problem (and doesn't actually solve my my problem either, I just found out...:-( ).

      What I want to do *may* not be doable, directly, using a bash expression -- since I'm not sure if the pattern matching in bash can control "greediness".

        I was trying to use the regular expression syntax from bash to match "one or more" of an item. (...) If the extglob shell option is enabled (...)

        Thing is, you have to explicitly enable extended pattern matching, which is not enabled by default when the shell is run in non-interactive mode.

        Both of these invocations do work for me:

        my $r = `/bin/bash -O extglob -c 'echo mc-lang-+([^-])-+([^-])*.rpm'`; my $r = `/bin/bash -c 'shopt -s extglob\necho mc-lang-+([^-])-+([^-])* +.rpm'`;

        (i.e., print $r would output mc-lang-4.6.1-140.x86_64.rpm, assuming a file with that name is in the current directory)

        Note that when enabling extglob within the command sequence itself (shopt -s extglob, as opposed to the startup option -O extglob), you have to use \n as the statement separator, because when using a semicolon, the shell would parse the line in one go, and produce a syntax error before the shopt command gets a chance to take effect...

        Interestingly, on Linux (where /bin/sh typically is a symlink to /bin/bash), the following does work too, for me:

        my $r = `shopt -s extglob\necho mc-lang-+([^-])-+([^-])*.rpm`;

        I say "interestingly", because when invoked through the name sh (as Perl does behind the scenes), bash will behave differently in several respects (i.e. it mimics the original bourne shell), which is why I would have expected it to not do extended pattern matching at all. Apparently, the latter is not the case.