learnedbyerror has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I am continuing in my journey with DBIX::Class. Honestly, I am not sure that I see as much, if any, value in it relative to straight perl and DBI; however, I know enough to know that you sometimes cannot see value until you work at it long enough. So ...

My testbed is an application that hits various public web services to collect information and store it in a SQLite database. I have quite a bit working functionally at this time; however, insert performance into the database is horrible with the default of Auto_Commit = 1. My test code shows almost a 100X improvement if I went to native DBI and used transactions comitting every 1000 records in a total of 750K records.

My quandry, err question, lies into how best use transactions with DBIX::Class. The online documentation almost reads like we support transactions, but don't use them because our code is wonderful. But if you do use them, keep Auto_Commit = 1 and use the DBIX::Class::Schema txn_do method and we will still do wonderful things. And lastly, if this doesn't work for you, you can use txn_begin, txn_commit and txn_rollback but its absolutely you fault, which it is anyway, if anything goes wrong.

When performing a large number of inserts or updates with DBIX::Class, what is the fastest, recommend pattern to use?

I like the thought of txn_do for applications that don't have the speed criticality above; however, I cannot find much documentation on the advantages and disadvantages of it. I seem to only find the module documentation or snippets of it reformatted. Are there any good reads on this method?

Thanks in advance,

lbe

Replies are listed 'Best First'.
Re: DBIX::Class and transactions
by moritz (Cardinal) on Apr 06, 2011 at 13:03 UTC

    I'm not a die-hard DBIx::Class expert, so take my statement with a grain of salt.

    ... but I don't see a reason not to use: 

    while (things_to_process) {
   $schema->txn_do(sub {
       # insert your 1000 rows here
   });
}

    [download]
    Perl 6 - second systems done right
[reply]
[d/l]
      I apologize for not updating this in a more timely manner. I have been using txn_do and this works well as long long as my transactions are not too big and not in too large of a volume. However, I am not satisfied with its performance for larger transactions or transactions with larger volumes as I have not been able to identify get this to work with parallel transactions (threaded or forked) with SQLite. I will start a new thread with more information on this.
[reply]
Re: DBIX::Class and transactions
by Your Mother (Archbishop) on Apr 06, 2011 at 14:01 UTC
[reply]
[d/l]
Re: DBIX::Class and transactions
by duelafn (Parson) on Apr 06, 2011 at 20:30 UTC
[reply]

Back to Seekers of Perl Wisdom