Esteemed Minks

I have several modules which fetch data from various websockets (WS), processes it and returns back results. Fetching and processing for each websocket is different so I guess I need many Modules.

In order to fetch data from WS I use IO::Async::Loop and Net::Async::WebSocket::Client with something like this:

async sub fetch {
my $loop = IO::Async::Loop->new;
my @data;
my $client = Net::Async::WebSocket::Client->new(
  on_text_frame => sub {
    my ( $clientself, $frame ) = @_;
    ...
    push @data, $frame;
    # when I had enough frames
    if( 10 < scalar @data ){
      say "I am ".__PACKAGE__."::fetch() and we had enough data";
      $clientself->close; $loop->stop;
      return;
    }
  } # on_text_frame
);
$loop->add( $client );
await $client->connect(url => $myurl);
# tell the WS that we need data
$client->send_text_frame('send me data');
$loop->run;
my $retdata = Future->done(\@data)->get;
# save $retdata to file
...
# and return
return $retdata;
}
sub give_me_data {
  # collapse the future
  my $data = $self->fetch()->get();
  # process data
  ...
  return $processed_data
}
[download]

I thought that worked OK, until I needed more modules like the above for differrent websockets (which provide different data structures, and need different processing, hence the many modules which blackbox the fetch+process data into a give_me_data())

Now, these modules are loaded like:

# main
use ModuleA;
use ModuleB;

sub give_me_data {
   # no Future here:
   my $dataA = ModuleA->new()->give_me_data();
   my $dataB = ModuleB->new()->give_me_data();
   return ...
}
[download]

But there is a problem: $dataB is *empty* because the Loop in ModuleB thinks that we have received enough frames already and stops receiving (the check when enough frames are received is included above). But the debug message when this happens coming from ModuleB mentions ModuleA::fetch()! Note that the data returned from ModuleB is empty (whereas ModuleA returned lots of data).

1. Does the code-sketch for fetching data look OK to you? It will be difficult to provide running code but I can try if needed.

2. Can there be multiple IO::Async::Loop's each one on its own Module? (EDITAnswer: no, there is a singleton loop object, so don't use loop->run and loop->stop when you have many clients to this loop). All modules loaded but the loops activated *sequentially*. My intention is to dispense each loop when enough data is fetched and then continue to the next Module's loop (instantiate, loop and dispense) and so on. Am I stopping and dispensing the loops OK? Why does it remember ModuleA's fetch() when running ModuleB's loop?

3. If I have hit a wall, is there a simpler, possibly blocking, alternative for receiving data from websocket? (I have tried Mojo::UserAgent with some failures for some of the websockets). I do not really need non-blocking fetches because I need the total data from all modules before I proceed.

many thanks, bw bliako

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.