I would like comments about the following code.

Maybe it could turn into some sort of dbd flat file driver?

It has been brought to my attention that there is the DBD::SQLite module. Contrary to SQLite, this doesn't claim to be a fully functional database. It simply saves the results of statement handle for future use.

A couple questions I would like to have answered is: Does this seem like something that should turn into a DBD module? What is the best way to get the meta information about the statement handle given that statement handles can be for many types of database drivers?

# You can execute a statement handle # and save it to a file with this. use SthFile; my $query = qq{ SELECT * FROM TABLE}; my $sth = $dbh->prepare($query) or die "Cant prepare query: $query\n $DBI::errstr"; $sth->execute() or die "Cant execute query: $query\n $DBI::errstr"; SthFile::sth_to_file({ sth => $sth, file_name => "/tmp/my_query.store", }); # And later, in some other program or place my $sth_from_file = new SthFile("/tmp/my_query.store") or die "Cant get some garbage.";
the fetchrow_array, fetchrow_arrayref, fetchrow_hashref all work. Other stuff hasn't been fully tested. *File name: SthFile.pm*

package SthFile; use strict; use warnings; use Storable qw(store_fd fd_retrieve); use IO::File; use Data::Dumper; =pod =head1 Synopsis use SthFile; my $query = qq{ SELECT * FROM TABLE}; my $sth = $dbh->prepare($query) or die "Cant prepare query: $query\n $DBI::errstr"; $sth->execute() or die "Cant execute query: $query\n $DBI::errstr"; SthFile::sth_to_file({ sth => $sth, file_name => "/tmp/my_query.store", }); # And later, in some other program or place my $sth_from_file = new SthFile("/tmp/my_query.store") or die "Cant get some garbage."; =head1 Mimicing Stuff This module makes an attempt to mimic the behaivors of a statement handle as stated in the documentation. fetchrow_array, fetchrow_arrayref, fetchrow_hashref, fetchall_arrayref, fetchall_hashref have been implemented. I'm aware those subs may be defined somewhere else and MAY be written better, but it was fun to just write them. =head1 Raison d'Etre I wanted to take a query from a database and save it for later, withou +t having to modify it in some way in order to store it or worry about how to store undefs. =head1 To Dos Have the ability to take a handle and write to that handle as opposed to just writing to the filename specified. Yeah, I threw in the 'active' stuff for fun. Don't know what to do with it. I haven't fully tested fetchall_hashref or fetchall_arrayref. =head1 Version 1 File format %meta_data ( version => 1 sth => { NAME => $sth->{NAME} sth_subs => { rows => $sth->rows()} rows => $sth->rows, maybe a date or the query or something. ) \@data =head1 Author Martin VanWinkle (uber_spaced) =cut sub sth_to_file { my ($param_ref) = @_; die "file_name must be specified" if ! $param_ref->{file_name}; die "No sth given..." if ! $param_ref->{sth}; my $fh = new IO::File ">$param_ref->{file_name}" or die "Cant open $param_ref->{file_name} for writing: $!"; store_fd({ sth => { NAME => $param_ref->{sth}->{NAME}, }, version => 1, sth_subs => { rows => $param_ref->{sth}->rows(), }, }, $fh); store_sth_data($param_ref, $fh); $fh->close() or die "Couldnt close $param_ref->{file_name}: $!"; } sub store_sth_data { my ($param_ref, $fh) = @_; my $array_ref; while ($array_ref = $param_ref->{sth}->fetchrow_arrayref()) { store_fd($array_ref,$fh); } } sub new { my ($class, $file_name) = @_; my $fh = new IO::File "<$file_name" or die "Cant open $file_name for reading: $!"; my $meta_ref = fd_retrieve($fh); die "Somethings wrong with , meta doesn't have version set..." if (!defined $meta_ref->{version}); my $sth = $meta_ref->{sth}; delete $meta_ref->{sth}; $sth->{fh} = $fh; $sth->{meta_ref} = $meta_ref; $sth->{active} = 1; bless $sth; return $sth; } sub finish { my ($self) = @_; $self->{fh}->close() if ($self->{active}); $self->{active} = 0; } sub fetchrow_arrayref { my ($self) = @_; if (eof($self->{fh})) { $self->finish(); return undef; } return fd_retrieve( $self->{fh} ); } sub fetchrow_array { my ($self) = @_; my $ar = $self->fetchrow_arrayref(); return @{$ar} if defined $ar; return (); } sub fetchrow_hashref { my ($self) = @_; my %hash; my @array = $self->fetchrow_array(); @hash{ @{$self->{NAME}} } = @array; return \%hash if scalar(@array); return undef; } sub fetchall_arrayref { my ($self, $slice, $max_rows ) = @_; my $row_count = 0; my $ar = []; my $thing_to_push; while ( $thing_to_push = $self->fetchall_arrayref_helper($slice) ) { last if (defined $max_rows && $row_count < $max_rows); $row_count++; push @$ar, $thing_to_push } return $ar; } sub fetchall_arrayref_helper { my ($self, $slice) = @_; if (ref($slice) eq 'HASH') { my $hr = $self->fetchrow_hashref(); return undef if !defined $hr; if (! scalar(keys %$slice)) { return $hr; } else { my %i_hash; @i_hash{keys %$slice} = @{%$hr}{keys %$slice}; return \%i_hash; } } elsif (ref($slice) eq 'ARRAY' || !$slice) { my $ar = $self->fetchrow_arrayref(); return undef if !defined $ar; if (! scalar(@$slice)) { return $ar; } else { my @sliced = @$ar[@$slice]; return \@sliced; } } else { die "I have no idea what you want me to do."; } } sub fetchall_hashref { my ( $self, $key_field ) = @_; die "2nd arg to fetchall_hashref must be a field to key on." if (!defined $key_field); my %hash; my $hr; while ($hr = $self->fetchrow_hashref()) { $hash{$hr->{$key_field}} = $hr; } return \%hash; } sub rows { my ($self) = @_; return $self->{meta_ref}{sth_subs}->{rows}; } 1;

Replies are listed 'Best First'.
Re: Save a statement handle to a file and open it again as a 'cursor' or something.
by mr_mischief (Monsignor) on Aug 23, 2007 at 18:22 UTC
    If this is an RFC for the need for such a module, would you mind pointing out where it's better, worse, different, or meant for a different purpose than DBD::AnyData, DBD::CSV, DBD::RAM, DBD::Mock, DBD:Sponge, DBD::Sprite, and a handful of others I'm probably missing?

    It's not that I'm knocking what you're doing. I'm just nto sure what we're being asked to evaluate and along what lines.

      DBD::Anydata appears to be for interacting with specified types of files as databases. SthFile just saves a statement handle as a cursor that can be opened at a later time.

      DBD::CSV changes NULL values to empty strings. SthFile saves the actual array returned from the statement handle. NULL values are undefs.

      DBD::RAM (from the documentation) pulls the entire data structure into memory. SthFile streams it.

      DBD::Mock just makes the a database handle that 'looks like' it worked.

      DBD::Sponge makes a handle out of perl data that is already in memory. SthFile's streams data into memory.

      DBD::Sprite appears to require you to create a table with definitions before saving stuff to the file. SthFile just streams a statement handle to disk, and then reads it from disk.

      A future modification will probably involve allowing an open handle of any type to be the output or input of SthFile.

      I already know something like this would be useful to me. One of the things I'm wondering is if I should make it as an extension for other dbd drivers that automatically save a statement handles meta data.

        It does sound handy when you state it like that.

        I'm not sure how you mean to make it an extension for other DBDs.

        Do you mean you'd make it so you can load this module separately, take the existing statement handle from any other DBD, and stuff it into a file with this for later retrieval, without modifying the other DBD? That would be very handy sometimes..

        If you mean you'd have this subclass another DBD and add this functionality, and you'd just have to specify in your constructor what DBD you're using, that could be handy, too. If you could get that into from DBI, that could be even better.

        If you're asking if you should try submitting patches to a bunch of other DBD authors, that's probably a bad idea if for no other reason than you'd never be able to keep up with all of them.

        Sorry if I'm dense today. I feel like I'm making this harder for myself to understand than is necessary. I'm not trying to do that. Maybe a little less pizza at lunch next time... I do feel a bit groggy.

Re: Save a statement handle to a file and open it again as a 'cursor' or something.
by f00li5h (Chaplain) on Aug 23, 2007 at 17:05 UTC

    Incase it's unclear, i'm being helpful and friendly in this post ;)

    I think your code is a bit short on documentation, for hwo the methods act, and what they are for.

    Also, you may be interested in reading up on Test::More and Test for the writing of some simple tests to show it works.

    (1) If you're planning on unleashing it on CPAN, perlnewmod, perlmodstyle, perlmod and perlmodlib quoth freenode's perlbot

    Also, your question is quite open ended, what are you looking for feedback on, specifically? the API, the implementation of the fetch method, or just the indenting style you're using?

    @_=qw; ask f00li5h to appear and remain for a moment of pretend better than a lifetime;;s;;@_[map hex,split'',B204316D8C2A4516DE];;y/05/os/&print;

    Updated
    1. In Re^4: Save a statement handle to a file and open it again as a 'cursor' or something., jdporter points out that some of my perldoc:// were cpan:// links. They aren't any more.
      I'll be reading these tonight. It looks like following links:

      http://search.cpan.org/search?mode=module&query=perlmod

      http://search.cpan.org/search?mode=module&query=perlmodlib

      arent functioning as you expected.

        Should have read the "Node text goes above here" thing. That's why the rant about sanity checks is above my statement. Whoops.