Re: how to save data to new array after retrieving from sql server

Hello mhoang,

Well in this case I will agree with fellow monk poj the best/easiest solution for me is selectall_arrayref() but just for reference there is also another "more correct/efficient way" of fetching data than the way that you are using fetchrow_array(), see bind_columns().

Sample of code that I put together just for demonstration purposes, convert it based on your example:

my $sth = $dbh->prepare("SELECT `Test-Column-1`, `Test-Column-2` FROM 
+`"
            .$config{'MySQL.table'}."` WHERE 1");

if (!$sth->execute()) {
    die "Error: ". $sth->errstr ."\n";
}

my ($column_1, $column_2);

# Bind Perl variables to columns:
$sth->bind_columns(\$column_1, \$column_2);

my %row;
# Column binding is the most efficient way to fetch data
while ($sth->fetch) {
    $row{$column_1} = $column_2;
}

print Dumper \%row;

__END__

$VAR1 = {
          'value-1 Column1' => 'value-1 Column2',
          'value-2 Column1' => 'value-2 Column2',
          'value-3 Column1' => 'value-3 Column2'
        };
[download]

Complete code for replication purposes:

#!/usr/bin/perl
use DBI;
use strict;
use warnings;
use Data::Dumper;
use Config::Simple;

my %config = ();
my $path = 'conf.ini';

sub mysql {
    Config::Simple->import_from("".$path."", \%config)
    or die Config::Simple->error();

    my $dbh = DBI->connect("dbi:mysql::".$config{'MySQL.host'}.":".$co
+nfig{'MySQL.port'}."", 
               "".$config{'MySQL.user'}."",
               "".$config{'MySQL.pass'}."",
               { 'PrintError' => 1, 'RaiseError' => 1 , 'AutoInactiveD
+estroy' => 1 } 
    ) or die "Could not connect to ". $config{'MySQL.host'} .": ". $DB
+I::errstr ."\n";

    my $databases = $dbh->do("SHOW DATABASES LIKE '".$config{'MySQL.db
+'}."'")
    or die "Error: " .dbh->errstr. "\n";

    if ($databases eq 1) {
    printf "Database: ". $config{'MySQL.db'} ." exists not creating: "
+. $config{'MySQL.db'} ."\n";
    }
    else {
    printf "Database: ". $config{'MySQL.db'} ." does not exist creatin
+g: ". $config{'MySQL.db'} ."\n";

    my $checkExist = $dbh->do("CREATE DATABASE IF NOT EXISTS `".$confi
+g{'MySQL.db'}."`")
        or die "Could not create the: ".$config{'MySQL.db'}." error: "
+. $dbh->errstr ."\n";
    } # End of else

    $dbh->do("USE ".$config{'MySQL.db'}."")
    or die "Error: " .dbh->errstr. "\n";

    my $tables = $dbh->do("SHOW TABLES FROM `".$config{'MySQL.db'}."`
              WHERE Tables_in_".$config{'MySQL.db'}." LIKE
              '".$config{'MySQL.table'}."'")
    or die "Error: ".dbh->errstr. "\n";

    if ($tables eq 1) {
    printf "Table: ".$config{'MySQL.table'}." exists not creating: ".$
+config{'MySQL.table'}."\n";
    }
    else {
    printf "Table: ".$config{'MySQL.table'}." does not exist creating:
+ ".$config{'MySQL.table'}."\n";

    my $prepareTable = $dbh->prepare("CREATE TABLE ".$config{'MySQL.ta
+ble'}." (
                    `ID` INT NOT NULL AUTO_INCREMENT,
                    `Test-Column-1` VARCHAR(25) CHARACTER SET utf8 NOT
+ NULL UNIQUE,
                    `Test-Column-2` VARCHAR(25) CHARACTER SET utf8 NOT
+ NULL UNIQUE,
                    PRIMARY KEY(`ID`)
                    );");

    if (!$prepareTable->execute()) {
        die "Error: ". $prepareTable->errstr ."\n";
    }
    } # End of else

    my $checkExistInsert = $dbh->prepare("INSERT IGNORE INTO `".$confi
+g{'MySQL.table'}.
                "` (`Test-Column-1`, `Test-Column-2`) VALUES
 ('value-1 Column1','value-1 Column2'),('value-2 Column1','value-2 Col
+umn2'), ('value-3 Column1','value-3 Column2')");

    if (!$checkExistInsert->execute()) {
    die "Error: ". $checkExistInsert->errstr ."\n";
    }

    my $sth = $dbh->prepare("SELECT `Test-Column-1`, `Test-Column-2` F
+ROM `"
    .$config{'MySQL.table'}."` WHERE 1");

    if (!$sth->execute()) {
    die "Error: ". $sth->errstr ."\n";
    }

    my ($column_1, $column_2);

    # Bind Perl variables to columns:
    $sth->bind_columns(\$column_1, \$column_2);

    my %row;
    # Column binding is the most efficient way to fetch data
    while ($sth->fetch) {
    $row{$column_1} = $column_2;
    }

    $dbh->disconnect()
    or warn "Error disconnecting: $DBI::errstr\n";

    return \%row;

} # End of mysql sub

my $output_ref = mysql();

print Dumper $output_ref;

__END__

$ perl test.pl
Database: PerlMonks exists not creating: PerlMonks
Table: Data exists not creating: Data
$VAR1 = {
          'value-1 Column1' => 'value-1 Column2',
          'value-2 Column1' => 'value-2 Column2',
          'value-3 Column1' => 'value-3 Column2'
        };
[download]

Update: Minor update of creating hash of hashes including name of table for easier sorting, when using multiple tables.

my %row;
# Column binding is the most efficient way to fetch data
while ($sth->fetch) {
    $row{$config{'MySQL.table'}}{$column_1} = $column_2;
}

$ perl test.pl
Database: PerlMonks exists not creating: PerlMonks
Table: Data exists not creating: Data
$VAR1 = {
          'Data' => {
                      'value-1 Column1' => 'value-1 Column2',
                      'value-3 Column1' => 'value-3 Column2',
                      'value-2 Column1' => 'value-2 Column2'
                    }
        };
[download]

I am also using the module Config::Simple maybe you will find it useful as I did. Configuration sample bellow:

[MySQL]
user=user
pass=password
host=localhost
port=3306
db=PerlMonks
table=Data
[download]

Hope this helps, BR.

Seeking for Perl wisdom...on the process of learning...not there...yet!

Comment on Re: how to save data to new array after retrieving from sql server Select or Download Code

Replies are listed 'Best First'.
Re^2: how to save data to new array after retrieving from sql server by mhoang (Acolyte) on Jul 28, 2017 at 00:44 UTC
I love the way Pol builds data structure. Very clean and clear, and also like the bind column, I will try soon.Thanks heaps `code` Nirvana is Now or Never	[reply] [d/l]
Re^3: how to save data to new array after retrieving from sql server by thanos1983 (Parson) on Jul 28, 2017 at 08:06 UTC
Hello again mhoang, No problems no worries, to be honest I also prefer using selectall_arrayref instead of bind_columns. But in cases that you are retrieving huge amount of data the only way to proceed is through bind_columns. Alternatively you will be utilizing huge amount of memory until all data have been processed and stored to the selectall_arrayref, when the other way you are just retrieving line by line the data. If in your case the retrieved data are not huge you should be fine. From the documentation bind_columns: `# Column binding is the most efficient way to fetch data` [download] Hope this helps, BR. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]