ag4ve has asked for the wisdom of the Perl Monks concerning the following question:

I've got a 'config' file that a shell script sources - I can't change the config file format but need to use the data in perl. Currently, I have a shell script that exports the values and then runs the perl script and just use $ENV{val} from within perl. But this seems clunky. Is there a way of parsing something like this in perl:

A="foo" B="some other data" C="appending $A" D=" some list with more data and $B "

Which with my current implementation, I'd have a shell script:

#!/bin/bash - . foo.conf export A export B export C export D ./foo.pl

And then:

#!/usr/bin/perl my %exp_vars = ( A => 'SCALAR', B => 'SCALAR', C => 'SCALAR', D => 'ARRAY', ); my $env; foreach my $var (keys %exp_vars) { my $type = $exp_vars{$var}; my $val = lc $var; if (! defined $ENV{$var} or length($ENV{$var}) <= 1) { push @out, "# [$var] not exported"; next; } if ($type eq 'ARRAY') { @{$env->{$val}} = ($ENV{$var} =~ m/(?:\s+)?(\S+)/g); } elsif ($type eq 'SCALAR') { $env->{$val} = $ENV{$var}; } }

So, how do I remove the shell script from this?

EDIT

After looking at the messing with Shell::Parser and looking at the suggestions, I took what I needed and came got the below (basically, I didn't see that Shell::Parser was givin me anything, so I parsed out the variables myself and then look any shell values that need to be interpreted:

#!/bin/env perl use strict; use warnings; use Data::Dumper; sub parse_sh { my ($data) = @_; my $r; $data =~ s/((?: *#.*?)?\n+)/\n/gs; # Remove comment lines $data =~ s/(\n+)/\n/gs; # Remove blank lines $data =~ s/ *(\S+)=("?)(.*?)\2 *\n/$r->{lc($1)}="$3"/gse; return $r; } sub parse_sh_maint { my ($r, $noset) = @_; # Interpolate variable names my $max_passes = 5; my $not_substituted; for (1 .. $max_passes ) { for my $k (keys %$r) { next unless (ref(\$r->{$k}) eq 'SCALAR'); $r->{$k} =~ s|\$(\w+)| $r->{lc($1)} // do{ "\$$1" } |ge; $not_substituted += $r->{$k} =~ tr/$/$/; } last unless $not_substituted; } # Remove remaining variables and make variables with spaces and newl +ines arrays for my $k (keys %$r) { $r->{$k} =~ s/\$\S+ *?//gs; # Remove remaining varia +bles $r->{$k} =~ s/\s+/\n/gs; # Remove leading spaces if (not grep { $_ eq $k } @$noset) { my %dedupe = map { $_ => 1 } split(/ |\n/, $r->{$k}); $r->{$k} = (); @{$r->{$k}} = keys %dedupe; } } return $r; } open(my $fh, "<", $ARGV[0]); my $content = do { local $/ = <$fh> }; my $stuff = parse_sh_maint(parse_sh($content), [qw/scalar1 scalar2/]); print "Vars: " . Dumper($stuff), "\n";

Replies are listed 'Best First'.
Re: Parsing bash data
by Perlbotics (Archbishop) on Aug 23, 2013 at 15:40 UTC

    Personally, I would rather use YAML or one of the config-modules from CPAN, but it seems, other non-Perl programs are also depending on the environment, right?

    This imperfect script should do some very basic parsing and interpolation. However, it would fail in situations like Y=`echo fails` or where here-documents are involved. If you need every functionality that you shell provides, stay with it. You might be able to get rid of the wrapper script using exec or system, though.

    use strict; use warnings; use Shell::Parser; use Data::Dumper; sub parse_shellscript_from_fh { my ($fh) = @_; my %vars; my $parser = new Shell::Parser syntax => 'bash', handlers => { assign => sub { my ($self, %args) = @_; my ($name, $value) = $args{token} =~ /^\s*(\w+)\s*=\s*\"(.*)\"\s*$/s; if ( defined $value ) { #-- multi-line/array? $value =~ s/\n//msg; $vars{$name} = $value; } else { warn "No value for variable ", ($name//'?'), "\n"; } } }; #-- parsing my $text = join("", <$fh>); $parser->parse( $text ); $parser->eof; #-- rudimentary variable interpolation my $max_passes = 5; my $not_substituted; for (1 .. $max_passes ) { $not_substituted = 0; foreach my $varname ( keys %vars ) { $vars{$varname} =~ s|\$(\w+)| $vars{$1} // do{ "\$$1" } |ge; $not_substituted += $vars{$varname} =~ tr/$/$/; } last unless $not_substituted; } die "Recursion or too few parses ... " if $not_substituted; return \%vars; } print "Vars: ", Dumper( parse_shellscript_from_fh( *DATA ) ), "\n"; __DATA__ A="foo" B="some other data" C="appending $A" D=" some list with more data and $B " X=<<EOF does not work there EOF Y=`echo fails`

    Result:

    No value for variable ? No value for variable ? Vars: $VAR1 = { 'A' => 'foo', 'D' => ' some list with more data and some other data ', 'C' => 'appending foo', 'B' => 'some other data' };

      That's awesome. Thanks. I don't like that that module was last maintained ~8 years ago, but it works. (I've also got to figure out why $not_substituted is getting set with my data - seems to work when I remove that die)
Re: Parsing bash data
by choroba (Cardinal) on Aug 23, 2013 at 15:34 UTC
    It is possible. The following could get you started, but also warned: there are lots of things you can do in bash. The script cannot parse ${VAR}, values outside the double quotes, backquotes, etc.
    #!/usr/bin/perl use warnings; use strict; use Data::Dumper; my $VARNAME = qr/[A-Z_][A-Z_0-9]*/; my %env; while (<>) { if (/^($VARNAME)="(.*)"$/) { $env{$1} = $2; } elsif (/^($VARNAME=")$/) { my $var = $1; my @val; my $line; until (($line = <>) =~ /"/) { chomp $line; push @val, $line; } $line =~ s/".*//s; push @val, $line; $env{$var} = \@val; } else { warn "Invalid line $.\n"; } } while (my ($key, $value) = each %env) { if (ref $value) { s/\$($VARNAME)/$env{$1}/g for @$value; } else { $value =~ s/\$($VARNAME)/$env{$1}/g; } $env{$key} = $value; } print Dumper \%env;
    لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
Re: Parsing bash data
by kcott (Archbishop) on Aug 24, 2013 at 12:33 UTC

    G'day ag4ve,

    This works with the example foo.conf file you posted; however, I would recommend checking that your example data is representative of your real data.

    $ perl -Mstrict -Mwarnings -Mautodie -e ' my %conf; open my $fh, "<", "foo.conf"; my $data = do { local $/; <$fh> }; $conf{$1} = $2 while $data =~ /(\w+)="([^"]*)"/g; s/\$(\w+)/$conf{$1}/g for values %conf; use Data::Dumper; print Dumper \%conf; ' $VAR1 = { 'A' => 'foo', 'C' => 'appending foo', 'B' => 'some other data', 'D' => ' some list with more data and some other data ' };

    -- Ken

Re: Parsing bash data
by Laurent_R (Canon) on Aug 23, 2013 at 17:48 UTC

    If I understand correctly, you have a config file on the one hand and a shell script that reads it and source the environment on the other hand. If my comprehension is correct, then simply reading the config file within the Perl program might be easier.