xajin25 has asked for the wisdom of the Perl Monks concerning the following question:

Hi Perl Community, I am trying to write a script that retrieves a list of hostnames from a mySql database, and then retrieve some information from them simultaneously with threads. The list of hostnames will be stored in a hash which will have the information about the host stored as a hash corresponding to each hosts. So the desired structure of my hash would be like so:
'1' => { 'hostname' => 'somehost@somewhere.com', 'host_info' => { 'info' => 'valueA', 'info2' => 'valueB', 'info3' => 'valueA' }, }, '2' => { 'hostname' => 'someotherhost@somewhere.com', 'host_info' => { 'info' => 'valueA', 'info2' => 'valueB', 'info3' => 'valueA' }, }, }
Here is my code that returns, "Invalid value for shared scalar" erros:
my %host_info : shared; my $host_info_REF = \%host_info; my $dbc= DBI-connect("DBI:mysql:database=mydb;IPADDRESS, 'user, 'passo +wrd"); my $SQL = "SELECT * FROM mydb.HOSTNAMES;"; my $st = dbc->prepare($SQL); $st->execute(); my $row = 0; while (my $tmp = $st->fetchrow_hashref()){ for my $key (keys %$tmp){ $host_info_REF->{$row}->{$key} = $tmp->{$key}; } $row++; } my @threads; my %hostHash; my $hostHashRef = \%hostHash; foreach my $current_host( keys %host_info ){ $hostHashRef->{"curHost"}=$current_host; $hostHashRef->{"href"}=$host_info_REF; push (@threads, threads -> create (\&get_info, $hostHashRef)); } sub get_info[ #Get some info and add to hash }

It crashes at the point where I am trying to assign the results from my query to my hash. "$host_info_REF->{$row}->{$key} = $tmp->{$key};". I believe it has something to do with my "$row" counter.

Replies are listed 'Best First'.
Re: Threads, DBI, Shared Variables problem
by kcott (Archbishop) on May 26, 2013 at 07:17 UTC

    G'day xajin25,

    Welcome to the monastery.

    You declare and initialise:

    my $row = 0;

    You then increment $rowRef which is neither declared nor initialised. Did you mean: ++$row? use strict; would have picked that up; I also recommend you add use warnings; (see strict and warnings). I don't think that's causing your error, though.

    You also have "my $keyRef = \$key;" but you don't subsequently make any use of $keyRef. What was your intent with this piece of code?

    Your code is incomplete and I don't know what you haven't included; however, I would draw your attention to this from perlthrtut - Shared And Unshared Data:

        In the case of a shared array, all the array's elements are shared, and
        for a shared hash, all the keys and values are shared. This places
        restrictions on what may be assigned to shared array and hash elements:
        only simple values or references to shared variables are allowed - this
        is so that a private variable can't accidentally become shared. A bad
        assignment will cause the thread to die.
    

    Update: This was a poor guess and has been stricken:

    I'm wondering if instead of

    $host_info_REF->{$row}->{$key} = $tmp->{$key};

    you perhaps want

    $host_info{$row}{$key} = $tmp->{$key};

    Here's the type of behaviour I think you're probably after:

    #!/usr/bin/env perl use strict; use warnings; use threads::shared; use Data::Dumper; my @db_data = ( { hostname => 'h0', hinfo0 => 'A', hinfo1 => 'B', hinfo2 => 'C' }, { hostname => 'h1', hinfo0 => 'X', hinfo1 => 'Y', hinfo2 => 'Z' }, ); my %host_info : shared; my $row = 0; for my $db_data (@db_data) { $host_info{$row} = shared_clone({ hostname => delete $db_data->{hostname}, host_info => $db_data }); ++$row; } print Dumper \%host_info;

    Output:

    $ pm_thr_compdat.pl $VAR1 = { '1' => { 'hostname' => 'h1', 'host_info' => { 'hinfo0' => 'X', 'hinfo1' => 'Y', 'hinfo2' => 'Z' } }, '0' => { 'hostname' => 'h0', 'host_info' => { 'hinfo0' => 'A', 'hinfo1' => 'B', 'hinfo2' => 'C' } } };

    You might also find useful information in threads and threads::shared.

    -- Ken

      Oops, sorry about the mismatched variable names, I was fiddling around to find a fix and thought the solution was to use a reference to "$row". These are only snippets so I understand the confusion. The ultimate goal of my script is to start with an existing hash containing hostnames, and then have threads to go ssh to each hostname, retrieve some information, and then add it to the hash. If this weren't threaded, it would have been simple, but it seems when you spin off a thread, it gets it's own local copy of variables and does not manipulate variables outside of it.
        I've updated the code to make it more clear. Also, would your example work with threads?
Re: Threads, DBI, Shared Variables problem
by Athanasius (Archbishop) on May 26, 2013 at 07:28 UTC

    As per the documentation quoted by kcott, shared variables are valid only if all parts (other than “simple values”, i.e., non-reference scalars) are explicitly shared. So, to declare and initialise the hash, you will need syntax something like this:

    my %host_info :shared = ( 1 => &share ( { hostname => 'somehost@somewhere.com', host_info => &share ( { info => 'valueA', info2 => 'valueB', info3 => 'valueA', } ), } ), 2 => &share ( { hostname => 'someotherhost@somewhere.com', host_info => &share ( { info => 'valueA', info2 => 'valueB', info3 => 'valueA', } ), } ), );

    See Re^3: How to share complex data structure in threads ? by BrowserUk.

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

      Hi, thanks for your reply. How would this assignment work if I was assigning my hash values from a DBI sql statement? More complete code is in my above example:
      my $row = 0; while (my $tmp = $st->fetchrow_hashref()){ for my $key (keys %$tmp){ $host_info_REF->{$row}->{$key} = $tmp->{$key}; } $row++; }
Re: Threads, DBI, Shared Variables problem
by Laurent_R (Canon) on May 26, 2013 at 09:52 UTC

    Just as a side note, looking at your sample data, I am wondering whether an array of hashes of hashes (AoHoH) would not be a better data structure for our data than a hash of hashes of hashes (HoHoH).