Querying Hive tables using DBD::ODBC

astroboy has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to query data from a Hive table on a MapR platform. My query is:

use Modern::Perl;
use DBI;
use Data::Dumper;

my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError
+ => 1});
$dbh->{LongReadLen} = 1000000; 
$dbh->{LongTruncOk} = 1;


my $sth = $dbh->prepare(q{select * from table});
$sth->execute;

while (my $row = $sth->fetchall_arrayref) {
    say Dumper $row;
}
[download]

What I get is the following line repeated hundreds of times to STDERR

DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
[download]

Then I get the one row in the table, followed by a never-ending stream of the following to STDOUT

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];
[download]

And the following to STDERR

DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 ... repeat forever ....
[download]

I have to Control-C to stop this. Any explanation and way forward is appreciated

Comment on Querying Hive tables using DBD::ODBC Select or Download Code

Replies are listed 'Best First'.
Re: Querying Hive tables using DBD::ODBC by poj (Abbot) on Oct 29, 2018 at 07:34 UTC
`->fetchall_arrayref` fetches all records so you don't need to loop (unless you don't have enough memory to fetch and return all the rows in one go.) `my $rows = $sth->fetchall_arrayref; say Dumper $rows;` [download] Try fetching each row using `->fetchrow_arrayref`. #while (my $row = $sth->fetchall_arrayref) { while (my $row = $sth->fetchrow_arrayref) { say Dumper $row; } poj	[reply] [d/l] [select]
Re^2: Querying Hive tables using DBD::ODBC by astroboy (Chaplain) on Oct 29, 2018 at 20:20 UTC
Duh! You wouldn't think I'd been using DBI since the late 90s! Thanks for the fresh pair of eyes - I couldn't see the wood for the trees	[reply]
Re: Querying Hive tables using DBD::ODBC by afoken (Chancellor) on Oct 29, 2018 at 07:27 UTC
`use Modern::Perl; use DBI; use Data::Dumper; my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError + => 1}); $dbh->{LongReadLen} = 1000000; $dbh->{LongTruncOk} = 1; my $sth = $dbh->prepare(q{select * from table}); $sth->execute; while (my $row = $sth->fetchall_arrayref) { say Dumper $row; }` [download] What I get is the following line repeated hundreds of times to STDERR `DBD::ODBC::st fetchall_arrayref warning: no select statement currently + executing at conn.pl line 17.` [download] The code you are showing us is not the code you are running. You posted only 15 lines, so there is no line 17. Show the code that is really running. If it is to long (more than about 30 lines), copy it, shorten it to the relevant parts, run it. Post the shortened code and the error messages and warnings it generates. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l] [select]
Re^2: Querying Hive tables using DBD::ODBC by astroboy (Chaplain) on Oct 29, 2018 at 20:19 UTC
Yip, I had the code in one window and the output in another, I remved three lines to make the code more succinct but forgot to run the output again	[reply]


Perl: the Markov chain saw
	PerlMonks