It appears that the client-side DBI/DBD modules are not doing any sort of compression and the compression=gzip option is being ignored in the DSN string.

The reason I ask if this is at all possible is because I can't find any mention of compression in DBI, DBD::Proxy or any other module that I can think of.

A removed feature? A good idea that was only partially implemented perhaps?

I could dicker with the dbiproxy to uncompress only certainly messages (not the connection information) and then run Compress::Zlib::compress to compress the data before I execute, but I was hoping for something more transparent and less hacky.

Am I missing something obvious? Is this something folks use regularly?

Ben

Am I missing something obvious? Is this something folks use regularly?

From your description, it sounds like this might have been a feature that was contemplated but not completed. I expect you'll have to ask Tim Bunce or someone else in the DBI team to get the full story.

I don't bother doing compression for traffic going to a database -- in my case, the data's not that frequent or lumpy, I don't have that many servers, and the local network is a 100Mb/sec guy. However, I do have tasks that regularly rsync data between servers, and some of that is text and highly compressible. Happily, rsync takes care of that automagically, so I never have to think about it.

Are you trying this out as an experiment, or do you have good information that your system could greatly benefit from this feature? I'm just worrying you may be doing premature optimziation.

Alex / talexb / Toronto

"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

After a few days of digging, it really does seem that the data compression is probably just half-baked.

Yes, this is more of an experiment - perhaps "polishing the turd" is more apt. I have a large number of application servers with large amounts of logs from legacy applications. Without touching the application logging, I am aggregating the logs to a central database.

It is working fine. However, since our apps are not CPU-bound, I wanted to see if I received any sort of network benefit of adding compression to the mix.

This appeared to be low-hanging fruit that simply required dbiproxy configured and running and a change the DSN on the client side.

The data being sent to the DB is generally not that large, though it certainly can be - I wouldn't be surprised if the overhead of Compress::Zlib surpassed the transmission time savings, but I wanted to try it out.

I see that Tim Bunce co-wrote the book that mentioned the compression as an alternative - I'll try that route.

Thanks

Ben