sethviebrock has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,

I'm using Net::Twitter::Stream to connect to the Twitter Streaming API (as Miyagawa's related module, AnyEvent::Twitter::Stream, seems to be out of commission). I'm running this process in the background using nohup. However, the disconnect callback code (shown below) doesn't seem to be reconnecting the socket upon API disconnect (everything works fine for a half hour, or more...but then stops updating). Can anyone see any flaws in the logic of this code? I'm not extremely well-versed at socket programming.

Below is how I'm calling it -- it seems as though the "connection_closed_cb" arg and corresponding callback do not properly handle disconnection from the API:

use constant SEARCH_TERM => "some_search_term"; parse_from_twitter_stream(); sub parse_from_twitter_stream { my $user = 'XXXXX'; my $password = 'YYYYY'; my $stream = Net::Twitter::Stream->new ( user => $user, pass => $pas +sword, callback => \&got_tweet, connection_closed_cb => \&connection_clos +ed, track => SEARCH_TERM); sub connection_closed { sleep 1; warn "Connection to Twitter closed"; parse_from_twitter_stream(); } sub got_tweet { my ( $tweet, $json ) = @_; # a hash containing the tweet # and the original json warn "Got tweet"; #I have more code here but it's snipped } }

This is the code from Net::Twitter::Stream -- see especially the last line of code. Not entirely sure what that does:

sub new { my $class = shift; my %args = @_; die "Usage: Net::Twitter::Stream->new ( user => 'user', pass => 'pas +s', callback => \&got_tweet_cb )" unless $args{user} && $args{pass} && $args{callback}; my $self = bless {}; $self->{user} = $args{user}; $self->{pass} = $args{pass}; $self->{got_tweet} = $args{callback}; $self->{connection_closed} = $args{connection_closed_cb} if $args{connection_closed_cb}; my $content = "follow=$args{follow}" if $args{follow}; $content = "track=$args{track}" if $args{track}; $content = "follow=$args{follow}&track=$args{track}\r\n" if $args{tr +ack} && $args{follow}; my $auth = encode_base64 ( "$args{user}:$args{pass}" ); chomp $auth; my $cl = length $content; my $req = <<EOF; POST /1/statuses/filter.json HTTP/1.1\r Authorization: Basic $auth\r Host: stream.twitter.com\r User-Agent: net-twitter-stream/0.1\r Content-Type: application/x-www-form-urlencoded\r Content-Length: $cl\r \r EOF my $sock = IO::Socket::SSL->new ( PeerAddr => 'stream.twitter.com:ht +tps' ); $sock->print ( "$req$content" ); while ( my $l = $sock->getline ) { last if $l =~ /^\s*$/; } while ( my $l = $sock->getline ) { next if $l =~ /^\s*$/; # skip empty lines $l =~ s/[^a-fA-F0-9]//g; # stop hex from compaining about +\r my $jsonlen = hex ( $l ); last if $jsonlen == 0; eval { my $json; my $len = $sock->read ( $json, $jsonlen ); my $o = from_json ( $json ); $self->{got_tweet} ( $o, $json ); }; } $self->{connection_closed} ( $sock ) if $self->{connection_closed}; }
Any help is much appreciated. Thanks!!

Replies are listed 'Best First'.
Re: Net::Twitter::Stream socket issue
by Steve_p (Priest) on Apr 17, 2012 at 22:28 UTC

    The first thing you need to do is make sure you have IO::Socket::SSL installed. It is a prerequisite for Net::Twitter::Stream, but it isn't listed in the Makefile.PL, so it may not have been installed. There are a couple several year old bugs on this for this module.

    After checking that, it's time to deal with Net::Twitter::Stream. It has some serious issues. Generally, with system calls, you want to be checking return values and $!. If they are undef or 0 where something useful should be returned, checking $! is essential to get the error back. Net::Twitter::Stream doesn't check for errors and return them to the code calling it in a way that can be useful. Basically, you're screwed. If there is a better alternative module, I'd suggest switching to it.


    Steve_p

    Test your modules with bleadperl!

      rsync -avz rsync://public.activestate.com/perl-current/ .
      ./Configure -des -Dusedevel -Dprefix=/path/to/test/perl
      make test
      make install
    

    Now, please test you modules! If you have test failures that don't happen with Perl 5.8.8, send a simplified test case to

    perlbug at perl.org

      Thank you so much! May have to write something custom...