msh210 has asked for the wisdom of the Perl Monks concerning the following question:

O Monks,

I keep getting hit with the die(qq/Unexpected end of stream while looking for line\n/) at line 1126 of HTTP::Tiny (current version, 0.056). It seems the sysread on line 1112 succeeded but returned 0, so the filehandle was at its end (as the die message implies), but I can't seem to figure out (from the module code) the circumstances under which that would occur. I would appreciate any assistance.

Note that this only occurs for a specific HTTP POST request that I'm expecting to take a long time (but not so long that the HTTP::Tiny->new(timeout=>...) is reached). Other POSTs work just fine. I don't know what else you might need to know about my code or Internet connection or… I just don't know what details to supply, but of course will be glad to edit details into this post in reply to requests for clarification.

Replies are listed 'Best First'.
Re: "Unexpected end of stream while looking for line"
by hotchiwawa (Scribe) on Jan 07, 2016 at 19:30 UTC
    Hi msh210 :)

    Can you provide your suspected code? (give only the essential)

    Here is HTTP::Tiny where the code failed:

    sub readline { @_ == 1 || die(q/Usage: $handle->readline()/ . "\n"); my ($self) = @_; while () { if ($self->{rbuf} =~ s/\A ([^\x0D\x0A]* \x0D?\x0A)//x) { return $1; } if (length $self->{rbuf} >= $self->{max_line_size}) { die(qq/Line size exceeds the maximum allowed size of $self +->{max_line_size}\n/); } $self->can_read or die(qq/Timed out while waiting for socket to become ready + for reading\n/); my $r = sysread($self->{fh}, $self->{rbuf}, BUFSIZE, length $s +elf->{rbuf}); if (defined $r) { last unless $r; } elsif ($! != EINTR) { if ($self->{fh}->can('errstr')){ my $err = $self->{fh}->errstr(); die (qq/Could not read from SSL socket: '$err'\n /); } else { die(qq/Could not read from socket: '$!'\n/); } } } #### Line 1126 die(qq/Unexpected end of stream while looking for line\n/); }


    Peace
      Can you provide your suspected code? (give only the essential)

      If I knew what was essential to be provided, I'd have done it in the first place. Here's some of my code:

      #!/usr/bin/perl use strict; use warnings; use HTTP::Tiny; use JSON; use Test::JSON; use utf8::all; use URI::Escape; use Smart::Comments '###'; use List::Util qw(min max); use Statistics::Basic qw(median mode); # ... my $http = HTTP::Tiny->new(timeout => 6000); # ... $response = $http->post($url, {headers=>{Authorization=>"token $access_token +"}, content=>$query}); die $! unless is_valid_json $response->{content}; # ...

        Essential means enough to demonstrate the problem. Sometimes the exercise of producing this reveals the source of the problem.

        But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Romans 5:8 (NASB)

Re: "Unexpected end of stream while looking for line"
by Anonymous Monk on Nov 05, 2024 at 18:35 UTC
    This is a bug in HTTP::Tiny If the server uses "\n\r" for line feeds instead of the more common "\n", it will think it received two line feeds instead of one and throw this error.
      I'm guessing this was supposed to say "\r\n".
      --
      A math joke: r = | |csc(θ)|+|sec(θ)| |-| |csc(θ)|-|sec(θ)| |
Re: "Unexpected end of stream while looking for line"
by hotchiwawa (Scribe) on Jan 07, 2016 at 20:10 UTC
    Constructor of HTTP::Tiny

    sub new { my($class, %args) = @_; my $self = { max_redirect => 5, timeout => 60, # default keep_alive => 1, verify_SSL => $args{verify_SSL} || $args{verify_ssl} || 0, # + no verification by default no_proxy => $ENV{no_proxy}, }; bless $self, $class; $class->_validate_cookie_jar( $args{cookie_jar} ) if $args{cookie_ +jar}; for my $key ( @attributes ) { $self->{$key} = $args{$key} if exists $args{$key} } $self->agent( exists $args{agent} ? $args{agent} : $class->_agent +); $self->_set_proxies; return $self; }

    Default request timeout is 60 seconds, why do you put 6000?

    6000 sec = 100 minutes = 1 hour 40 :)
      I put 6000 because I suspected the timeout caused the 599 error. But that was not the case: the response should have come before 100 minutes.

        The remote server could have a front-end proxy with its own timeout limit exceeded by the process preparing the response.


        The way forward always starts with a minimal test.
        With 100 minutes, there is something wrong in the "flow" or "concept".
        You can't put a timeout of 100 minutes because the server should release the request and don't keep variables in memory or cache. The server performance will drop and you can have "server not responding", with the need to recycle or restart the server!
        Imagine you have 20 connections in 100 minutes, all will be kept during this time but there are other timeouts: session timeout, script timeout, execution timeout... and they are different, less...