learningperl01 has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone I am hoping someone can point me in the right direction as I am a little confused at the moment with what I am trying to do. I am trying to perform a sql query which reads some input from another file which gets placed in the query I am performing. The problem that I am having or probably just not doing right is I want to save all results(rows) to an array which I will use to sort certain fields after all the results have been saved to the array. I am not sure if I am saving or how to save all results to an array for later use. I have tried the following but when I print to view the results I don't get all the rows just the first. How do I access all the rows at once or print all rows records? Or I guess what I need to figure out is how would I access rows in the array for sorting. I know how to sort but not really sure how to access the records stored in the array after the sql is run? Thanks for the help!!
my @lines = read_file('/tmp/test.txt'); chomp(@lines); my $sql = q|select * from table_name where srcaddr = ? or dstaddr = ?; foreach my $line (@lines) { $sth->execute($line, $line) or die "Can't execute SQL statemen +t: $DBI::errstr\n"; my @all_rows; while (($line) = $sth->fetchrow_array()) { push(@all_rows, $line); } foreach (@all_rows) { print "$_\n"; } }

Replies are listed 'Best First'.
Re: Saving sql results to an array all rows?
by kyle (Abbot) on Aug 18, 2008 at 15:40 UTC

    As dHarry says, you're probably best off having the database sort your results.

    If you want to get every row, use a DBI method that does that. For example:

    $sth->execute( $line, $line ) or die ...; my @all_rows = @{ $sth->fetchall_arrayref() };

    At that point, every element in @all_rows is an array reference. The first row is $all_rows[0]. The first column of the first row is $all_rows[0][0].

    If you want, you can get an array of hash refs instead. I find these more useful because you can refer to the fields by name.

    my @all_rows = @{ $sth->fetchall_arrayref( {} ) };

    In that case, $all_rows[0] is still the first row, but it's a hash ref instead. Then you can do $all_rows[0]{srcaddr} for that field in that row.

    If you want to output the whole structure, Data::Dumper is good for that, or see How can I visualize my complex data structure?.

      Krambambuli I followed your example and I get all rows/results correctly. Now the only problem/thing that I do not know how to deal with is the @{$row_ref} array? If I wanted to run the sorted code which runs against an array passing it the @{$row_ref} array does not work. I get an array code ARRAY(0x9019330) on the screen here is the code that I am using currently. Thanks for the help!!
      foreach my $line (@lines) { $sth->execute($line, $line) or die "Can't execute SQL statemen +t: $DBI::errstr\n"; my @all_rows; while ( my $ref = $sth->fetchrow_arrayref()) { push(@all_rows, $ref); } foreach my $row_ref (@all_rows) { print "@{$row_ref}\n"; } }
      Here is the sorting code that I use in a separate script which is what I am having trouble incorporating to the @{$row_ref} array. I am guessing that I can't use @{$row_ref} name when running my sort? Any ideas or help would be greatly appreciated.
      my @sorted = map {$_->[0]} sort {$a->[1] <=> $b->[1] || $a->[2] <=> $b->[2] || $a->[3] <=> $b->[3] || $a->[4] <=> $b->[4]} map { [$_, split(/\./, (split /\s+/, $_)[1])]} @lines; print "$_\n" for @{$row_ref};
      sample of the sql output below. 2008-07-26 10.168.1.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK
Re: Saving sql results to an array all rows?
by dHarry (Abbot) on Aug 18, 2008 at 15:27 UTC

    It's also possible to let SQL do the sorting for you by using the ORDER BY clause.

    Update:
    I was wondering how big is the result set? Can you store that in memory?

      Thanks for the quick reply. The results usually aren't big. Sorting in mysql query will not work since it will only sort for that one record not the final output. What I have done in the past is save all the resuls to a file then run another script which will sort based on the IP address in the results. Shown below. What I am hoping to do is eliminate having to run the second script and do the sorting and query on all one script(Also learn in the process). I am pretty new to Perl so I am still trying to work myself through some of the examples I find online. Here is the script which I use to sort the results.
      my @lines this is array of the file being read in) my @sorted = map {$_->[0]} sort {$a->[1] <=> $b->[1] || $a->[2] <=> $b->[2] || $a->[3] <=> $b->[3] || $a->[4] <=> $b->[4]} map { [$_, split(/\./, (split /\s+/, $_)[1])]} @lines; print "$_\n" for @sorted;
Re: Saving sql results to an array all rows?
by Krambambuli (Curate) on Aug 18, 2008 at 15:35 UTC
    With
    ($line) = $sth->fetchrow_array()
    you just take the first field out of each record, ignoring the rest.

    Try maybe to rewrite
    while (($line) = $sth->fetchrow_array()) { push(@all_rows, $line); } foreach (@all_rows) { print "$_\n"; }
    as
    while ( (@record) = $sth->fetchrow_array()) { push(@all_rows, \@record); } foreach my $row_ref (@all_rows) { print "@{$row_ref}\n"; }

    Krambambuli
    ---
      Krambambuli I followed your example and I get all rows/results correctly. Now the only problem/thing that I do not know how to deal with is the @{$row_ref} array? If I wanted to run the sorted code which runs against an array passing it the @{$row_ref} array does not work. I get an array code ARRAY(0x9019330) on the screen here is the code that I am using currently. Thanks for the help!!
      foreach my $line (@lines) { $sth->execute($line, $line) or die "Can't execute SQL statemen +t: $DBI::errstr\n"; my @all_rows; while ( my $ref = $sth->fetchrow_arrayref()) { push(@all_rows, $ref); } foreach my $row_ref (@all_rows) { print "@{$row_ref}\n"; } }
      Here is the sorting code that I use in a separate script which is what I am having trouble incorporating to the @{$row_ref} array. I am guessing that I can't use @{$row_ref} name when running my sort? Any ideas or help would be greatly appreciated.
      my @sorted = map {$_->[0]} sort {$a->[1] <=> $b->[1] || $a->[2] <=> $b->[2] || $a->[3] <=> $b->[3] || $a->[4] <=> $b->[4]} map { [$_, split(/\./, (split /\s+/, $_)[1])]} @lines; print "$_\n" for @{$row_ref};
      sample of the sql output below. 2008-07-26 10.168.1.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK
        Try to see if the following code might help you:
        use strict; use warnings; use Data::Dumper; my @lines = ( '2008-07-26 13.168.1.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK' +, '2008-07-26 10.169.1.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK' +, '2008-07-26 10.168.2.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK' +, '2008-07-26 10.168.1.1 11122 3.3.3.3 80 hxxp://3.3.3.3 200 OK' +, ); my @sorted = map { $_->[0] } sort { $a->[1] <=> $b->[1] || $a->[2] <=> $b->[2] || $a->[3] <=> $b->[3] || $a->[4] <=> $b->[4] } map { [$_, split(/\./, (split /\s+/, $_)[1] ) ] } @lines; my @all_rows = map { [ split( /\s+/, $_ ) ] } @lines; my @sorted1 = map { join( ' ', @{ $_->[0] } ) } sort { $a->[1] <=> $b->[1] || $a->[2] <=> $b->[2] || $a->[3] <=> $b->[3] || $a->[4] <=> $b->[4] } map { [$_, split(/\./, $$_[1] ) ] } @all_rows; print Dumper( @sorted1 );

        Krambambuli
        ---