IAperl has asked for the wisdom of the Perl Monks concerning the following question:

Hi Everyone

I am working on a Docker project which implements the following script (https://github.com/CAIsr/volgenmodel-nipype/blob/master/volgenmodel.py)

This script is written in python and contains a section of code which converts a certain conf_file written in perl syntax.

This conf_file is used by an external function 'Nlpfit' which reads perl syntax (https://github.com/BIC-MNI/minc-widgets/blob/master/nlpfit/nlpfit)

while running this code in docker environment, i keep getting error:

String found where operator expected at (eval 8) line 6, near "u'step + ' => 8, u'" and Error reading config from /scratch/volgenmodel-fast-example/workfl +ow/writeconf_01_/fit_stage_01.conf (fix it!)_

I have checked the conf_file for syntax error, but there doesnt seem to be any string error. My default conf_file looks like this

@conf = ( {'step' => 8, 'blur_fwhm' => 4, 'iterations' => 20}, {'step' => 6, 'blur_fwhm' => 3, 'iterations' => 20}, {'step' => 4, 'blur_fwhm' => 2, 'iterations' => 10}, {'step' => 2, 'blur_fwhm' => 1, 'iterations' => 10}, );

Note that this external function (Nlpfit) file uses version 1.2.0 but the docker environment uses Perl5Lib

Do you think this is due to compatibility issue between Perl version 1.2.0 and Perl version 5.0

I am new to perl and need guidance on what is causing this error and how can I fix it

Also note that this algorithm runs fine outside Docker environment

Cheers

Replies are listed 'Best First'.
Re: perl and Docker
by 1nickt (Canon) on Oct 25, 2016 at 11:53 UTC

    The conf file you are trying to load contains garbage characters.

    The relevant code in your external script is:

    my(@conf); if(defined($opt{config_file})){ my($buf); $buf = `cat $opt{config_file}`; # slurp if (eval($buf)){ print STDOUT "$me: Read config from $opt{config_file}\n" if $opt +{verbose}; } else{ die "$me: Error reading config from $opt{config_file} (fix it!)\ +n\n"; } } else{ @conf = @default_conf; }
    This is a horrible way of loading configuration, but it works:

    $ cat 1174639.conf @conf = ( {'step' => 8, 'blur_fwhm' => 4, 'iterations' => 20}, {'step' => 6, 'blur_fwhm' => 3, 'iterations' => 20}, {'step' => 4, 'blur_fwhm' => 2, 'iterations' => 10}, {'step' => 2, 'blur_fwhm' => 1, 'iterations' => 10}, );
    $ cat 1174639.pl use strict; use warnings; use feature 'say'; use Data::Dumper; my $file = '1174639.conf'; my $buf = `cat $file`; my @conf; eval( $buf ); say Dumper \@conf;
    $ perl 1174639.pl $VAR1 = [ { 'blur_fwhm' => 4, 'step' => 8, 'iterations' => 20 }, { 'step' => 6, 'blur_fwhm' => 3, 'iterations' => 20 }, { 'blur_fwhm' => 2, 'step' => 4, 'iterations' => 10 }, { 'blur_fwhm' => 1, 'step' => 2, 'iterations' => 10 } ];

    Do you think this is due to compatibility issue between Perl version 1.2.0 and Perl version 5.0

    Definitely not. The 1.2.0 version number you cite is just the version number of the Nlpfit software.

    Hope this helps!

    The way forward always starts with a minimal test.

      Careful. You might get nominated for the useless use of cat award, and actually get it.

      This is a horrible way of loading configuration

      Indeed. Instead of

      my $file = '1174639.conf'; my $buf = `cat $file`; my @conf; eval( $buf );

      I'd rather say

      my @conf = do '1174639.conf';

      since duplicating a static file name into a variable is pointless, as is allocating a buffer to eval a file, and invoking /bin/sh to read it.

      update: The award should go to the author of nlpfit, since lines 227-244 read as follows

      # set up the @conf array my(@conf); if(defined($opt{config_file})){ my($buf); $buf = `cat $opt{config_file}`; # slurp if (eval($buf)){ print STDOUT "$me: Read config from $opt{config_file}\n" if $opt +{verbose}; } else{ die "$me: Error reading config from $opt{config_file} (fix it!)\ +n\n"; } } else{ @conf = @default_conf; }

      and that's where 1nickt got that abomination. It shouldn't have been repeated without correction, though.

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: perl and Docker
by Corion (Patriarch) on Oct 25, 2016 at 06:54 UTC

    The first step would be to look at the eval statement and look at whatever code it is trying to run.

    The u'step' => 8, u' looks a lot like Python code to me, so maybe you are mixing Python and Perl in a very weird way.

      Hi,

      Yes I have the original code in python which tries to translate it into perl command in the main function using "to_perl" and "read_from_perl"

      The actual conf_file is a python dictionary.

      The external library is perl function.

      This entire algorithm (code) works fine outside docker. But within docker container it generates error.

      I have also set the container (shell) environment and locale to UTF-8 but doesnt seem to be working

      Cheers

        Without seeing any of your setup, code and "config" code, I have to guess here.

        Your Perl code is trying to run/eval the Python "config" code when it is run from within the container. Why it finds the Python config is your next task to find out.

Re: perl and Docker
by Laurent_R (Canon) on Oct 25, 2016 at 17:28 UTC
Re: perl and Docker
by soonix (Chancellor) on Oct 31, 2016 at 13:24 UTC
    Line 6 of your config file contains garbage (Python's "u"-prefix for unicode strings is foreign to Perl)

      Hi

      I belive so too that, python is setting the 'conf_file' to unicode.

      As a result, I set the locale of my Docker container to UTF-8, as follows ,

      RUN locale-gen en_US.UTF-8 ENV LC_ALL=en_US.UTF-8 ENV LANG=en_US.UTF-8 ENV LANGUAGE=en_US.UTF-8

      Which changes the container environment as such,

      "Env": [ "PATH=/opt/python-code/extra-scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "LC_ALL=en_AU.UTF-8", "LANG=en_AU.UTF-8", "LANGUAGE=en_AU.UTF-8", "PYTHONPATH=/opt/python-code", "PERL5LIB=/usr/local/perl" ],

      I also include the following in my Dockerfile

      ENV PERL_UNICODE "SD"

      I was of the belief that this would enable perl to read the unicode

      But on running, it still generates the same error.

      I am using python 2.7.12 and perl 5.22

      Cheers

        Enabling unicode for Perl doesn't make Perl able to read Python unicode notation.

        ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

        Hi IAperl,

        Unicode has nothing to do with this. As various monks have told you numerous times, the file is simply not in Perl syntax. Also, I don't think you've posted an example of the file you're actually trying to load, have you? (I didn't dig through all of the GitHub repos you posted.) I know what I mean. Why don't you?

        If the file is in Python syntax, and you want to read it in Perl, then I would suggest that you write a Python script that reads the file and outputs it in a format that both Python and Perl can understand, such as YAML or JSON.

        If the file is supposed to be in Perl syntax, then apparently whoever is generating the file is doing so incorrectly, and I suggest you fix that code. Otherwise, you'd have to code a solution that can read it, which is IMO not the best solution.

        To assist you further, we'll have to see the problem for ourselves - please post an example of the file you are trying to read. How do I post a question effectively?

        Regards,
        -- Hauke D