in reply to Perl Multi Processing.

For scripting languages, Event-driven programming has historically been far more popular for concurrent programming than (heavyweight) threads:

For Perl, the two most popular Event Loops appear to be:

See Also

  • Comment on Re: Perl Multi Processing. (Event-driven Programming References)

Replies are listed 'Best First'.
Re^2: Perl Multi Processing.
by marioroy (Prior) on Dec 07, 2021 at 05:20 UTC

    Chunking complements event-driven programming. For example reaching out to millions of devices.

    use strict; use warnings; use MCE::Flow; use MCE::Queue; use AnyEvent::SNMP; my $work_q = MCE::Queue->new; my $nthreads = 100; my $chunk_size = 300; sub consumer { while ( defined ( my $data = $work_q->dequeue ) ) { MCE->printf("wid %3d, size %d\n", MCE->wid, scalar(@$data)); MCE->yield; # Optional # The delay occurs serially among workers. # It is helpful to not all blast simultaneously. # do event-driven programming to 300 max items # ... } } sub provider { my @input = (1 .. 20_000); while (@input) { my @data = splice @input, 0, $chunk_size; $work_q->enqueue(\@data); } $work_q->end; } MCE::Flow->run( # 100 consumers, 1 provider { max_workers => [ $nthreads, 1 ], interval => {delay => 0.008} }, \&consumer, \&provider, );

    It depends on the app. I tend to set nthreads to about 4x the number of logical cores on the box. The delay is serially behind the scene. Each worker waits it's turn in milliseconds before initiating to the OS eventIO to hundreds of devices.