Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
I have a system which needs to act as a proxy to another of our servers, mainly for legacy support reasons. With the help of Dancer as a proxy, I've managed to get it mostly working with the following code:
return send_file( \'ignored', streaming => 1, callbacks => { override => sub { eval { my $client_connection = shift; my $ua = LWP::UserAgent->new; my $client; my $status = $ua->get($real_url, ':content_cb' => sub { my ($data, $resp) = @_; unless ($client) { my $headers_in = $resp->headers; my %headers_out = ( 'Content-Disposition' => sprintf('inline; filename="%s"', $file->{fileName}), ); for (qw( Content-Type Content-Length Keep-Alive Last-Mod +ified )) { $headers_out{$_} = $headers_in->header($_) if $headers_in->header($_); } $client = $client_connection->([$resp->code, [%headers_o +ut]]); } $client->write($data); }); if ($status->is_error) { my $headers_in = $status->headers; my %headers_out; for (qw( Content-Type Content-Length Keep-Alive Last-Modif +ied )) { $headers_out{$_} = $headers_in->header($_) if $headers_in->header($_); } my $client = $client_connection->([$status->code, [%header +s_out]]); $client->write($status->error_as_HTML); } 1; } or warn "Proxy failure: $@"; return; }, }, );
I say "mostly working" because, while the files are streamed successfully and browsers will accept them without complaint, wget and curl are less forgiving. Both of these command-line programs issue errors after the file is (successfully) received:
$ curl -sS -o rcvd https://foo/bar.pdf curl: (18) transfer closed with outstanding read data remaining $ wget https://foo/bar.pdf <...> 2017-01-12 14:48:58 (6.10 MB/s) - Read error at byte 3316868 (Success. +).Retrying. <proceeds to loop endlessly>
The byte at which wget reports the read error is always the last byte of the file (i.e., equal to the file size), leading me to suspect that an EOF marker isn't being handled properly. Possibly also relevant is that, while the proxying code copies the Content-Length header from the original source, the original source does not provide that header, so Content-Length is not actually set.

Using curl/wget to download the file directly from the original source works perfectly with no error messages issued.

Does anyone have any insights as to what the cause of the problem might be?

Edit: On further investigation, the problem does not appear to be with Dancer itself. Testing with Dancer's internal mini-server (using bin/app.pl) does not exhibit this problem. It only shows up in the production environment, which has Apache <-> Starman <-> Dancer. So now to work out whether the issue is with one of the other components individually or with the interactions between them.


In reply to EOF problem with Dancer streaming proxy by dsheroh

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (3)
As of 2024-03-29 14:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found