in reply to Re^2: How to make an HTTP request with an equivalent of curl's --max-time?
in thread How to make an HTTP request with an equivalent of curl's --max-time?

For the max-time option? It's in the manpage for curl.

No, not for the self-explanatory option, for the magic url that is immune to regular HTTP GET, that it needs a client induced timeout

  • Comment on Re^3: How to make an HTTP request with an equivalent of curl's --max-time?

Replies are listed 'Best First'.
Re^4: How to make an HTTP request with an equivalent of curl's --max-time?
by blindluke (Hermit) on Jan 14, 2015 at 09:16 UTC

    No, not for the self-explanatory option

    It would be self-explanatory if its name was max-seconds, that's the problem with most of the _time / -time variables and options. Very few things these days are self explanatory, and, due to insufficient information provided, one often has to figure out the units, the limits, or even the object of a question. ;)

    for the magic url that is immune to regular HTTP GET, that it needs a client induced timeout

    The OP (probably) refers to the Twitter streaming API - here is the documentation you were (probably) asking for. There are multiple requirements forced upon the client of the stream, like this one below (quote from the docs):

    Set a timer, either a 90 second TCP level socket timeout, or a 90 second application level timer on the receipt of new data. If 90 seconds pass with no data received, including newlines, disconnect and reconnect immediately according to the backoff strategies in the next section. The Streaming API will send a keep-alive newline every 30 seconds to prevent your application from timing out the connection. You should wait at least 3 cycles to prevent spurious reconnects in the event of network congestion, local CPU starvation, local GC pauses, etc.

    - Luke

      LWP timeout option is for seconds of TCP level socket timeout ... default is 120 seconds not 90

        The quote I posted comes from the Twitter documentation (that I linked to in my reply). This has nothing to do with the defaults of the LWP module. The value specified by the streaming API is 90 seconds. I have not written a single word in this thread about LWP and the default values it uses for its options.

        - Luke

Re^4: How to make an HTTP request with an equivalent of curl's --max-time?
by Cody Fendant (Hermit) on Jan 14, 2015 at 22:13 UTC

    Hi, here's the relevant section from the Twitter documentation:

    To connect to the Streaming API, form a HTTP request and consume the resulting stream for as long as is practical. Our servers will hold the connection open indefinitely, barring server-side error, excessive client-side lag, network hiccups, routine server maintenance or duplicate logins.

    Not sure what you mean about it being "immune". The issue is that it doesn't stop sending data until something goes wrong or the client terminates it. Why Twitter decided this was a good way to serve up the data I don't know.