in reply to Re: dbiproxy and compression
in thread dbiproxy and compression

Yes, I ran the dbiproxy with --compression. It seems that my problem is on the client side.

I ran the proxy in the debugger and stepped through the code as it received the connection. It certainly attempted to Compress::Zlib::uncompress on every message received from the socket. The problem was that the message was not compressed.

However, I can find no mention in the client-side connection libraries that compression was an option. I added compression=gzip to the DSN (as suggested in the O'Reilly book, but not in any PODs) to no effect. Is this just a pipe dream?

Has anyone successfully ever gotten DBI::Proxy to talk to dbiproxy (DBI::ProxyServer) with compression enabled?

Alternately, is there another way to transparently compress data with the DBI modules?

Ben

Replies are listed 'Best First'.
Re^3: dbiproxy and compression
by talexb (Chancellor) on Nov 06, 2007 at 17:29 UTC
      The problem was that the message was not compressed.

    OK -- so why wasn't the message compressed? Is Compress::Zlib not installed on the client side?

    Am I missing something obvious?

    Alex / talexb / Toronto

    "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
      That is the question. Compress::Zlib is installed all around.

      It appears that the client-side DBI/DBD modules are not doing any sort of compression and the compression=gzip option is being ignored in the DSN string.

      The reason I ask if this is at all possible is because I can't find any mention of compression in DBI, DBD::Proxy or any other module that I can think of.

      A removed feature? A good idea that was only partially implemented perhaps?

      I could dicker with the dbiproxy to uncompress only certainly messages (not the connection information) and then run Compress::Zlib::compress to compress the data before I execute, but I was hoping for something more transparent and less hacky.

      Am I missing something obvious? Is this something folks use regularly?

      Ben

[reply]
          Am I missing something obvious? Is this something folks use regularly?

        From your description, it sounds like this might have been a feature that was contemplated but not completed. I expect you'll have to ask Tim Bunce or someone else in the DBI team to get the full story.

        I don't bother doing compression for traffic going to a database -- in my case, the data's not that frequent or lumpy, I don't have that many servers, and the local network is a 100Mb/sec guy. However, I do have tasks that regularly rsync data between servers, and some of that is text and highly compressible. Happily, rsync takes care of that automagically, so I never have to think about it.

        Are you trying this out as an experiment, or do you have good information that your system could greatly benefit from this feature? I'm just worrying you may be doing premature optimziation.

        Alex / talexb / Toronto

        "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom