mr_mischief has asked for the wisdom of the Perl Monks concerning the following question:

I have a CGI program that needs its content as it is sent to the web server instead of all at once after the whole thing is sent. This works with my testing server, but I'm running Apache 2.0.something and the production server is Apache 1.3.37, and it has a couple of shared hosting-related modules on it, mod_bwlimited and mod_log_bytes.

What I'd like to know is if using mod_perl I could convince the server to give me the info when I need it regardless of what other cruft is happening to buffer the request. Does it depend upon in what order the modules are loaded? Is it just a problem inherent in Apache 1.3.37?

If anyone knows of an easier solution, like a directive for the sparsely documented modules above that tell them to do their jobs in a respecful manner and give my program the content it's trying to read, I'd love to know that, too.


Christopher E. Stith
  • Comment on Can mod_perl get me past a silly buffering in Apache?

Replies are listed 'Best First'.
Re: Can mod_perl get me past a silly buffering in Apache?
by perrin (Chancellor) on Aug 22, 2006 at 21:13 UTC
    Essentially you are asking if you can parse the HTTP protocol yourself instead of letting the web server do it. In Apache 2/mod_perl 2, you can, by writing an input filter. I don't know any way to do it with mod_perl 1 or CGI.
[reply]
      Not so much the whole HTTP protocol. Just the POST request body. I can do that in a CGI, and in fact CGI.pm does just that. ...and I don't really want to parse it, either. I'm just wanting to read it, count it, and pass it on to another program that does the parsing (using CGI.pm, in fact). In case you're wondering, yes it is an upload progress meter, complete with asynchronous Javascript network calls.

      I currently have code that, as a CGI, read()s its standard in and gets data that is so many bytes of the request in a loop.

      It works this way just fine, and on one server it gets a bit of the request, does its thing, and then gets a little more. On another server, it seems to never run until there's a full request and it reports that the request is done right away the first time that it reports anything. That's the buffering I'm trying to work around -- something in the one server is buffering up the whole request before opening my CGI's stdin and sending the data. Stock Apache 2.0 doesn't seem to do this, and I wasn't aware that stock Apache 1.3 might.

      I know there's a record inside mod_perl of the request. I just wanted to know if there's a way to get at it before the whole request is there. If the mod_perl environment doesn't get triggered until the whole request is complete, then it won't be able to do that just like the CGI can't now. If there's no way in mod_perl to address just a part of a request value, then there's no way to do it from mod_perl either.

      Thanks for the helpful answer. I'd rather have heard it was easy to do, of course, but informative is the main thing. :-)


      Christopher E. Stith
[reply]
[reply]
        As derby pointed out, Apache::UploadMeter does this for mod_perl 1. It patches Apache::Request in order to get C-level access to the internal state of the request in order to do this. I don't know enough about the apache internals to know if there's another way.

        For mod_perl 2, Apache2::UploadProgress does it with an input filter, as I mentioned.

[reply]
Re: Can mod_perl get me past a silly buffering in Apache?
by CountZero (Bishop) on Aug 22, 2006 at 21:02 UTC
    There are fundamental differences between Apache 1.3.x and Apache 2.x

    What works under mod_perl2 might not work under mod_perl for Apache 1.x

    That being said and although mod_perl gives you access to all stages of the request-cycle, I think that this cycle will only start once the whole of the data is received. At least a quick glance through my copy of "Practical mod_perl" did not show anything about pre-empting the buffer.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

[reply]
      Knowing that I can do it in Apache 2.0 with a CGI, is there reason to even hope I could do it with mod_perl on Apache 1.3? I've tried looking though some mod_perl references online, but haven't found enough relevant information to ascertain even how much more looking is warranted. If there's a decent chance I could this by finally breaking down and getting a solid understanding of mod_perl under my belt, I'd be happy to do so. Right now, though, there are too many time crunches at home and at work to devote a bunch of study to a short-term dead-end.

      The "easy" fix, I guess, is to find a hosting company with a recent Apache build and convince my employer to let me host some of our clients' sites on it. That's great long-term, but for this one project I'd really like to fit my solution to the environment instead of changing business practices to fit my code.


      Christopher E. Stith
[reply]
Re: Can mod_perl get me past a silly buffering in Apache?
by derby (Abbot) on Aug 22, 2006 at 18:50 UTC

    Where's the code?. Does setting $|=1; help? Ugg ... must remember in is in and out is out.

    -derby
[reply]
[d/l]
      I'm asking about whether the concept of leveraging mod_perl's access to Apache's innards to get at the data when it's needed is plausible.

      When asking if an approach as a concept is plausible, I'm not sure having code to debug is important. You can't debug plausibility.

      If there's a relevant doc I could read that clearly tells me what kind of access I have to the body of a POST request in mod_perl vs. the access I have to it in a CGI process, I'd love to read it. I just don't know where it is.


      Christopher E. Stith
[reply]
        Then find an Apache list to ask on. If you had some code someone might be able to run it on a mod_perl server, but without code it's all just specualtion (at least here anyway!).

        jdtoronto

[reply]

Back to Seekers of Perl Wisdom