Dear Perl monks,
may I ask you for your wisdom about the parallel processing in Mojolicios, please?
Here I am trying to understand how to implement parallel processes within a loop, set up arguments to the subprocess and get the result before the render (surely, I am using $c->render_later).
I was trying many different approaches and one seems to work for me, but the only thing with the argument (parameter). If I run the code below without ->($_) the processes work in parallel, the main sub waits for all of them to complete (which takes about 2 seconds) and then renders the page. But once I add ->($_) or any int, for example, the processes run in a blocking way - one by one and it takes about 12 seconds.
use Mojo::Base -strict, -signatures, -async_await;
use Mojo::IOLoop::Subprocess;
use Mojo::Promise;
async sub testMethod{
$c->render_later;
show "BEFORE";
my @promises;
for(0..5){
say "(START) This is $_ loop";
my $subprocess = Mojo::IOLoop::Subprocess->new;
push @promises, $subprocess->run_p(
sub {
my $i = shift || 1;
my $promise = Mojo::Promise->new;
say "Hello, from $i and $$!";
sleep 2;
say "Good bye, from $i and $$!";
$promise->resolve("Done for $i");
return $promise;
#return "Done for $i";
}->($_)
)->then(sub ($_result) {
show $_result;
return $_result;
})->catch(sub {
my $err = shift;
say "Subprocess error: $err";
});
$subprocess->ioloop->start unless $subprocess->ioloop->is_runn
+ing;
say "(END) This is $_ loop";
}
my @results = await Mojo::Promise->all_settled(@promises);
show @results;
show "AFTER";
return $c->render(text => "All done", status => 200);
}
Below is the output with the parameter sent via ->($_)(works in a blocking way, takes about 12 secs):
======( "BEFORE" )===========[ 'Test.pm', line 189 ]======
"BEFORE"
(START) This is 0 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 0 loop
(START) This is 1 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 1 loop
(START) This is 2 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 2 loop
(START) This is 3 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 3 loop
(START) This is 4 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 4 loop
(START) This is 5 loop
Hello, from 5 and 17430!
Good bye, from 5 and 17430!
(END) This is 5 loop
Subprocess error: Can't locate object method "Promise=HASH(0xdc0f198)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc0ac00)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc06310)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdbff708)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc11650)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc0fae0)"
+ via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/
+lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
======( @results )===========[ 'Test.pm', line 238 ]======
[
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
]
======( "AFTER" )============[ 'Test.pm', line 247 ]======
"AFTER"
This is the output without parameters sent (no </code>->($_)</code> construction) (works parallel, takes about 2 secs):
======( "BEFORE" )===========[ 'Test.pm', line 189 ]======
"BEFORE"
(START) This is 0 loop
(END) This is 0 loop
(START) This is 1 loop
(END) This is 1 loop
(START) This is 2 loop
(END) This is 2 loop
(START) This is 3 loop
(END) This is 3 loop
(START) This is 4 loop
(END) This is 4 loop
(START) This is 5 loop
(END) This is 5 loop
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdbfd950) and 20033!
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdc043f8) and 20034!
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdc09c00) and 20036!
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdc09528) and 20035!
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdc0a2c0) and 20037!
Hello, from Mojo::IOLoop::Subprocess=HASH(0xdc0e570) and 20038!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdbfd950) and 20033!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdc043f8) and 20034!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdc09528) and 20035!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdc09c00) and 20036!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdc0a2c0) and 20037!
Good bye, from Mojo::IOLoop::Subprocess=HASH(0xdc0e570) and 20038!
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( $_result )===========[ 'Test.pm', line 213 ]======
undef
======( @results )===========[ 'Test.pm', line 225 ]======
[
{ status => "fulfilled", value => [undef] },
{ status => "fulfilled", value => [undef] },
{ status => "fulfilled", value => [undef] },
{ status => "fulfilled", value => [undef] },
{ status => "fulfilled", value => [undef] },
{ status => "fulfilled", value => [undef] },
]
======( "AFTER" )============[ 'Test.pm', line 234 ]======
"AFTER"
Another option I have tried:
sub {
my $sp = shift; # $subprocess
my $i = shift || 3;
#my $promise = Mojo::Promise->new;
say "Hello, from $i and $$!";
sleep 2;
say "Goodbye, from $i and $$!";
#$promise->resolve("Done for $i");
#return $promise;
return "Done for $i";
#}->($_)
}
This gives the next output (works parallel, takes about 2 secs):
======( "BEFORE" )===========[ 'Test.pm', line 189 ]======
"BEFORE"
(START) This is 0 loop
(END) This is 0 loop
(START) This is 1 loop
(END) This is 1 loop
(START) This is 2 loop
(END) This is 2 loop
(START) This is 3 loop
(END) This is 3 loop
(START) This is 4 loop
(END) This is 4 loop
(START) This is 5 loop
(END) This is 5 loop
Hello, from 3 and 20608!
Hello, from 3 and 20609!
Hello, from 3 and 20610!
Hello, from 3 and 20611!
Hello, from 3 and 20612!
Hello, from 3 and 20613!
Good bye, from 3 and 20608!
Good bye, from 3 and 20611!
Good bye, from 3 and 20609!
Good bye, from 3 and 20610!
Good bye, from 3 and 20612!
Good bye, from 3 and 20613!
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( $_result )===========[ 'Test.pm', line 216 ]======
"Done for 3"
======( @results )===========[ 'Test.pm', line 228 ]======
[
{ status => "fulfilled", value => ["Done for 3"] },
{ status => "fulfilled", value => ["Done for 3"] },
{ status => "fulfilled", value => ["Done for 3"] },
{ status => "fulfilled", value => ["Done for 3"] },
{ status => "fulfilled", value => ["Done for 3"] },
{ status => "fulfilled", value => ["Done for 3"] },
]
======( "AFTER" )============[ 'Test.pm', line 237 ]======
"AFTER"
Dear monks, I spent 3 days on my own trying to get this working, but cannot.
I need to run parallel processes with a custom parameter for each process, all processes run from the loop (or a recursive sub), wait until they all are done, and then have a result from each completed process and finally render the page.
Please, help me with this.
I want to get an idea of what I do wrong and how to make it work, please!
Thank you in advance!
Peace!
|
I get this:
Software error:
Couldn't get it! at Noumena1.pl line 42.
when I try to run the script. It used to work. By 'work' I mean it used attempt to strip a webpage of everything but its punctuation (don't ask) leaving the webpage format unchanged
code here
#!/usr/bin/perl -wT
# /home/sites/www.in-vacua.com/web/cgi-bin/Noumena1.pl -w
use CGI::Carp qw(fatalsToBrowser);
use strict;
use CGI ':standard';
use lib '/.users/27/inv838/Template';
use LWP::Simple;
use HTML::Parser;
use vars qw($html);
my $content;
# Configurable variables for the script
my %templates = (text => "text_output.html",
url => "url_output.html");
# Initialise a new CGI object for parameter handling, etc.
my $q = CGI->new;
# Check to see if we have any input from the user. If so,
# we go to process it. If not, we'll return a blank form
if ($q->param('text')) {
my $text = &process_text($q->param('text'));
&output_template('text',$text);
} elsif ($q->param('url')) {
my $text = &process_url($q->param('url'));
} else {
print $q->redirect("/noumena.html");
}
## Subroutine Definitions
# process_url: strip non-punctuation from html docs (harder)
sub process_url {
my ($url) = @_;
my $content = get($url); #41
die "Couldn't get it!" unless defined $content;
# Slightly ugly kludging to sort out internal document links
# on sites that don't fully qualify (damn them all)
if (!($url =~ m!^http://!)) {
$url ="http://".$url;
}
$url =~ m!(http://(.*))/!;
my $baseurl=$1 || $url;
$content =~ s!href="/(.*)"!href="$baseurl/$1"!ig;
$content =~ s!rel="/(.*)"!rel="$baseurl/$1"!ig;
$content =~ s!src="/(.*)"!src="$baseurl/$1"!ig;
# HTML::Parser is slightly odd - it uses a callback interface which
+throws
# things back into this namespace.
HTML::Parser->new(api_version => 3,
handlers => [start => [\&_html_parser_tag,
+"text"],
end => [\&_html_parser_tag,
+"text"],
text => [\&_html_parser_text,
+ "dtext"]],
marked_sections => 1,)->parse($content);
print $q->header;
print $html;
}
# html_parser_text: handler to tell HTML::Parser what to do with text
+sections
sub _html_parser_text {
my ($text) = @_;
$text =~ s!\w! !g;
$html .= $text;
}
# html_parser_tag: handler to pass html tags unmolested back to HTML::
+Parser
sub _html_parser_tag {
my ($text) = @_;
$html .= $text;
}
# output_template: use Template Toolkit to return data to the user
sub output_template {
my ($type, $text) = @_;
print $q->header;
my $template = Template->new;
$template->process($templates{$type},
{text => $text})
|| die $template->error();
}
|
Mise Wonks,
The problem has been encountered many times: print int(2.26*100) = 225 (although print 2.26*100 = 226).
LanX's Humans have too many fingers warns about Floats are not accurate with decimal fractions because the computer "has only two fingers" and offers this advice: calculate with integers in the desired accuracy and shift the decimal point afterwards.
And this is what I was attempting to do: I was converting the float 2.26 into an integer 226 in order to increment it by 1. (Background: 2.26 is a version number and I wanted to bump it up to version 2.27. Since then I discovered Version::Next which can be a solution ...)
In C I get the correct result it seems to me:
# gcc a.c
#include <stdio.h>
#define ABS(A) ((A)>0?(A):(-(A)))
int main(void){
float version = 2.26;
float newversion = version + 0.01;
printf("version=%f, newversion=%f\n", version, newversion);
printf("2.26*100=%.8f\n", 2.26*100);
float x = 226.0;
if( ABS(x-226.0)>0.00000001 ){ printf("ajaja\n"); }
}
And in bc: echo "2.26*100" | bc -l gives 226.00
I have perl v5.36.0 and a modern linux OS on a modern desktop, 64bit, and :
perl -V:nvtype -V:nvsize
nvtype='double';
nvsize='8';
So, the advice to use decimal cents to do calculations in cents instead of using floating dollars is sound but how do I get to the decimal cents in the first place?
Oh, wonk: a person preoccupied with arcane details or procedures in a specialized field; broadly : nerd. hehe (Edit: I think "nerd" is superfluous and plain wrong).
bw, bliako
|
