in reply to Re: Can two separate responses be sent to the client's browser from Perl, such as via fork{}?
in thread Can two separate responses be sent to the client's browser from Perl, such as via fork{}?

Bliako,

Your response looks difficult to implement, but seems to come nearest to understanding the situation. In my case, I've been running an example TeX file on this (via copy/paste into a form field, not file upload--though I'm on an internal LAN), and the script invokes the XeLaTeX command twice in order to produce a proper TOC. The server returns the PDF in about 6-7 seconds for a 330+ page book. This is quite tolerable, and is in no danger of timeouts. This is a fairly straightforward dictionary-style layout, without images, etc.--just text. Because it had returned the results so promptly, I had not even really grasped why people were giving me answers for long-running processes; but perhaps I should not assume that my example would be representative for all use-cases. FWIW, I have created this functionality on a dedicated VM, with full install of LaTeX components, fonts, etc. specifically for this. Perhaps this is why it runs so quickly, though I had not chosen to use a separate VM for purposes of speed, but rather to keep the LaTeX server configuration separate from my other CGI routines.

I'm still puzzled by the thought that it is possible to send two files in a single base64-encoded lump, to be dissected client-side with JavaScript. If I could do this, though, it would definitely solve the problem. I'll have to look into this more. The way the script presently runs, the user receives the PDF and the server no longer needs to keep it. No need for tracking the time window to keep a particular file in case of a subsequent user request, and, in fact, the files do not need to be uniquely named (though I'm adding a timestamp to the filename as a courtesy to the user for versioning purposes).

Looking at the link you included, I'm left wondering how to indicate a separation between two or more files or segments in the blob data returned. I think I'll have to experiment a little with this, because if it works, it would do what I want. Thank you.

Blessings,

~Polyglot~

  • Comment on Re^2: Can two separate responses be sent to the client's browser from Perl, such as via fork{}?

Replies are listed 'Best First'.
Re^3: Can two separate responses be sent to the client's browser from Perl, such as via fork{}?
by hippo (Archbishop) on Oct 17, 2023 at 09:13 UTC
    The server returns the PDF in about 6-7 seconds for a 330+ page book. This is quite tolerable, and is in no danger of timeouts.

    For requests arriving serially. Try 20 in parallel and see what happens.


    🦛

      I imagine you're quite right. I am unconcerned, however, because I am planning this for a niche group, and the average requests might be fewer than two per day (it might go a week or more between "bursts" of activity, and likely will have only a handful of individuals even accessing the site--fewer than 20, to be sure).

      Now, once put online could it go "viral"? Perhaps. In such a case I would probably have to put it behind a login. But I really do not expect to have much trouble with traffic, at least not for some time.

      Blessings,

      ~Polyglot~

Re^3: Can two separate responses be sent to the client's browser from Perl, such as via fork{}?
by bliako (Abbot) on Oct 18, 2023 at 11:24 UTC

    Your server will be sending a JSON:

    my $PDF = ...; # read PDF binary contents from file my $LOGSTR = ...; # read LOG file contents use MIME::Base64; my $data = { # this is binary that's why we encode it to fit in a JSON string 'pdf' => encode_base64($PDF), # this is text but it does not harm: 'logstr' => encode_base64($LOGSTR) }; # send the data to client: # CGI version: use CGI qw(:standard); use JSON; print header('application/json'); my $json_text = to_json($data); print $json_text; # OR render JSON to go to client with Mojolicious my $c = ... # controller ... $c->render(json => $data); # $data as above

    Here is javascript for the client (more like untested hints):

    function ajaxit( url, // server endpoint method, // e.g. POST data // any data to send to the server for this request? ){ var xmlhttp = new XMLHttpRequest(); xhr.open(method, url, true); // this is the function which handles the xhr data transfer xhr.onreadystatechange = function () { // xhr.readyState can be 0 to 4 and each one comes here // 4 means transaction is all done, // 3 means data is received. // for error or success, state is 4 if( xhr.readyState !== 4 ){ // data is being transfered still, wait ... return; } // xhr.readyState=4, all is done, either on error or success: if( xhr.status === 200 ){ // data was received, make it a JS object (basically a hashtable + aka JSON) var data = JSON.parse(xhr.responseText); // check integrity here // ... // handle the data your html must have a div to put the <a>'s // with this adivID json_data_handler(data, adivID); } else { // we did not get a HTTP code 200 OK, // if we have a callback to handle errors, call it if( onError == null ){ console.log("ajaxre() : error, "+method+"ing to '"+url+"':\n +readyState="+xhr.readyState+", status="+xhr.status); } else { # handle failed request #onError(xhr.readyState, xhr.status); } } }; // now do the XHR request with caller-supplied data xhr.send(data); // this call will be handled by the function above // function is asynchronous and leaves here while request is being m +ade } // this is called when XHR was successful and we got data from server // that data was converted to a JS object (see above ajaxit()) // and this data is given here as dataobj // your html should contain a DIV somewhere (whose id you supply) // to add two <a> tags in there for the user to click function json_data_handler( dataobj, divID ){ // we have already sent an XHR request to server // and it gave us back data (as a xhr.responseText) var obj = JSON.parse(data); // do here some checks if this was successful // ... // atob decodes base64 strings to bytes, // var pdf is now the actual binary pdf content var pdf = atob(dataobj['pdf']); var logstr = atob(dataobj['logstr']); // client-side save-as filename, whatever var saveAsFilename = '...'; // do whatever with pdf data var el = document.getElementById(divID); var a = document.createElement("a"); el.appendChild(a); # edit: set the right headers for pdf data #var blob = new Blob([pdf], {type: "octet/stream"}); var blob = new Blob([pdf], {type: "application/pdf"}); # edit: setting an intermediate variable which should be # revoked when not needed anymore with # window.URL.revokeObjectURL(objurl); var objurl = window.URL.createObjectURL(blob); a.href = objurl; a.download = saveAsFilename+'.pdf'; // do the same for logstr // or, since it is text, display it in the div verbatim }

    above is totally untested but the idea was tested

    Apropos processing latex from Perl, I always use LaTeX::Driver to run latex.

    bw, bliako

      Bliako,

      I'd ++ that post more than once if the system allowed. You're very kind to provide code examples. Not having worked with JSON before, it may help me a lot. I've already been trying to work out how to do things via the information at the links provided earlier--so far have yet to get the download to initiate via AJAX. I know it must be possible somehow, but it seems the entire page almost needs to refresh to get the download http headers. I don't know. In any case, only when the entire page is submitted does it seem to work, whereas with AJAX the page is not refreshed, only updated in key parts.

      Regarding the module you suggested, I was unaware of it, but have already got something working. From a quick glance at the description, though, I'm uncertain the module would help me. It says: "run_latex Runs the formatter (latex or pdflatex)." I'm not able to use pdflatex (I wish I could, as it allows for microjustification) because I'm needing compatibility with Asian-language scripts. For my use case, only XeLaTeX can do the job. Perhaps this is why I'm rolling my own script for this. Unfortunately, there are no microjustification capabilities for any TeX solution on these Asian scripts (Thai, Lao, Karen, Burmese, etc.). For that, people have to rent the very pricey (for this economy) Adobe InDesign--or else use Microsoft Word. Ugh!

      Blessings,

      ~Polyglot~

        Polyglot

        Regarding the first part, AJAX requests do not need refreshing the client page. Just concentrate on a test JS script where you click a button, it does AJAX request to server and presents some results in a div. Start from there. An important part of this is server-side: sending JSON back to client on an AJAX request. Once you get the individual components working all will fit together niecely. A WORD OF WARNING: obvioulsy the proper way is to hava the server queueing the client requests, processing them at its own time and notifying the clients (via an email, or via the client checking on the server, perhaps via a client-page timed-auto-refresh (see Re: Can two separate responses be sent to the client's browser from Perl, such as via fork{}?) etc.) when results are ready. This is what afoken and fletch are telling you.

        regarding LaTeX::Driver, it can run XeLaTeX too if you pass this parameter: 'format' => 'pdf(xelatex)' at construction time. There are options to specify the bin path.

        bw, bliako