Re: Kronecker Product

by vr (Curate)
 on Jun 21, 2022 at 12:44 UTC Need Help??

I have a nitpick and an improvement (well, in my view). The latter: the beauty of "array language" is that it allows one to be succinct. No need to bother with boring details of array shape (i.e. dimensions) explicitly: interpreter ain't stupid, it itself can figure out the info it needs. Size of leading dummy dimensions of \$x, if "1" originally, will be stretched as required to fit shape of second array i.e. multiplicand i.e. \$y. (Shape of which, in its turn, will be padded with dummy dimensions. So then both multiplicands have matching shapes of 4 dimensions.) Further, oh-please spare me the details that 2D shape of result will be pair of multiplications of widths and heights of original matrices. Can't tolerate too much maths for today already! I only know that 4D shape of multiplication result should be reduced: dims 0 and 2 clumped together (note implicit xchg (or mv) here) to produce 3D, then dims 1 and 2 also clumped to produce 2D. Will shape then be w1*w2 x h1*h2? Fine, but it's none of my business. So:

```sub kronecker_product {
my (\$x, \$y) = @_;

return \$x-> dummy( 0 )
-> dummy( 0 )
-> mult( \$y, 0 )
-> clump( 0, 2 )
-> clump( 1, 2 )
}

Or:

```use PDL::NiceSlice;

sub kronecker_product {
my (\$x, \$y) = @_;

( \$x( *1, *1 ) * \$y )
-> clump( 0, 2 )
-> clump( 1, 2 )
}

A nitpick: operation is non-commutative. Function args are \$x, \$y, in that order. There was no reason to write computation as \$y * \$x-> ...etc. Of course scalar multiplication of element by element of 4D arrays is commutative, and so result is correct either way. It's just my preference to keep written expression so it has no unjustified swaps of arguments as \$x-> call() * \$y. Sorry for nitpick.

Replies are listed 'Best First'.
Re^2: Kronecker Product
by choroba (Archbishop) on Jun 21, 2022 at 12:56 UTC
Thanks, that's exactly the reply I was waiting for.

I see no difference between your solution and mine in the benchmark, but I definitely understand your points and see the improvement in the code.

Are you aware of any reading on "array languages" that would be understandable to tyros? I understand your solution but I can't imagine how to get to it (to get mine, I used trial and error repeatedly).

map{substr\$_->[0],\$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

But it's same trial and error on my part. I think if anything can count as systematic approach, as true textbooks, from very basics and up, -- that, for me, have been Learning J and J for C Programmers, books designed to teach "tyros" the "array language", their authors using a bit different perspectives.

A nice intro to the concepts of array programming is https://en.wikipedia.org/wiki/Array_programming.

Don't feel bad about trial and error; my first go at this (and I have the dubious privilege of being current PDL maintainer) assumed (wrongly) that dupN would be suitable for both sides of the multiplication; furthermore, it took me a little while to figure out inflateN, even though, annoyingly, it is nearly identical to dupN except for swapping the two parts of the slice argument.

Re^2: Kronecker Product
by etj (Chaplain) on Jun 21, 2022 at 17:19 UTC
I was inspired to copy the recently (as of 2.077) added dupN to make inflateN, which does the converse of dupN, suitable for Kronecker products.
```sub PDL::inflateN {
my (\$this, @times) = @_;
return \$this->copy if !grep \$_ != 1, @times;
my \$sl = join ',', map "*\$_,:", @times;
\$this = \$this->slice(\$sl);
\$this = \$this->clump(\$_, \$_+1) for 0..\$#times;
\$this;
}

pdl> p \$a = pdl '[[1 2][3 4]]'
[
[1 2]
[3 4]
]
pdl> p \$b = pdl '[[5 6][7 8][9 10]]'
[
[ 5  6]
[ 7  8]
[ 9 10]
]
pdl> p \$a->inflateN(\$b->dims)
[
[1 1 2 2]
[1 1 2 2]
[1 1 2 2]
[3 3 4 4]
[3 3 4 4]
[3 3 4 4]
]
pdl> p \$b->dupN(\$a->dims)
[
[ 5  6  5  6]
[ 7  8  7  8]
[ 9 10  9 10]
[ 5  6  5  6]
[ 7  8  7  8]
[ 9 10  9 10]
]
sub kron { my (\$x,\$y) = @_; \$x->inflateN(\$y->dims) * \$y->dupN(\$x->dims
+) }
pdl> p kron(\$a,\$b)
[
[ 5  6 10 12]
[ 7  8 14 16]
[ 9 10 18 20]
[15 18 20 24]
[21 24 28 32]
[27 30 36 40]
]
The above will broadcast nicely over any number of dimensions, not just 2. inflateN has been added and will be in the next PDL release (2.081).

I think dummy dimensions add only a few bytes to PDL-struct header, but your approach really physicalizes arrays from 2D to 4D (edit: well, it's still same 2D but squared dimensions for this example, sorry), which has grave consequences for performance. If no-one ever needs Kronecker product of 80x80 matrices, then count this comment as another nitpick :). + Note report of memory consumption assumes Win32, so disable/replace otherwise.

```use strict;
use warnings;
use feature 'say';
use PDL;
use Time::HiRes 'time';

use constant DIM => 80;
my \$x = random DIM, DIM;
my \$y = random DIM, DIM;

report( 'choroba, vr', \&kronecker_product, \$x, \$y );
report( 'etj', \&kron, \$x, \$y );

sub report {
my ( \$monk, \$code, \$x, \$y ) = @_;

my \$t = time;
my \$k = \$code-> ( \$x, \$y );
say "\$monk:";
say 'time: ', time - \$t;
say 'memory: ', mem();
}

sub mem {
qx{ typeperf "\\Process(perl)\\Working Set Peak" -sc 1 }
=~ /(\d+)\.\d+\"\$/m;
( my \$s = \$1 ) =~ s/(\d{1,3}?)(?=(\d{3})+\$)/\$1,/g;
return \$s
}

sub kronecker_product {
use PDL::NiceSlice;
my ( \$x, \$y ) = @_;

( \$x( *1, *1 ) * \$y )
-> clump( 0, 2 )
-> clump( 1, 2 )
}

sub PDL::dupN {
my (\$this, @times) = @_;
return \$this->copy if !grep \$_ != 1, @times;
my \$sl = join ',', map ":,*\$_", @times;
\$this = \$this->slice(\$sl);
\$this = \$this->clump(\$_, \$_+1) for 0..\$#times;
\$this;
}

sub PDL::inflateN {
my (\$this, @times) = @_;
return \$this->copy if !grep \$_ != 1, @times;
my \$sl = join ',', map "*\$_,:", @times;
\$this = \$this->slice(\$sl);
\$this = \$this->clump(\$_, \$_+1) for 0..\$#times;
\$this;
}

sub kron { my (\$x,\$y) = @_; \$x->inflateN(\$y->dims) * \$y->dupN(\$x->dims
+) }

__END__

choroba, vr:
time: 0.0958220958709717
memory: 355,741,696
etj:
time: 8.45677018165588
memory: 2,977,234,944

Edit 2. By the way, while we are at this example, the PDL::NiceSlice says

• arguments should not be quoted
• lone '*' inserts a dummy dimension of order 1

but I see that lone unquoted star(s) cause Perl to produce a warning:

Use of uninitialized value in concatenation (.) or string

so I had to insert a pair of very unsightly "1"'s, which of course devalues code nicety by half at least:). Is this a bug in PDL::NiceSlice, documentation flaw, or just me reading it incorrectly?

your approach really physicalizes arrays from 2D to 4D
No, it does not. It uses clump and slice, both of which are affine transformations (as shown by the P2Child=>1 in https://github.com/PDLPorters/pdl/blob/master/Basic/Slices/slices.pd for the relevant operations). Physicalising only happens on output or use in calculations, and when loop fusion comes in, not even then. I'd love to hear of an approach that Kronecker-products m1,n1 and m2,n2 matrices without producing a m1*m2,n1*n2 in-memory object. While you're pondering that, please show some numbers demonstrating these "grave consequences" :-)
Is this a bug in PDL::NiceSlice, documentation flaw, or just me reading it incorrectly?
You're going to need to show some code that you're saying is behaving unexpectedly. This worked as described (with output):
```pdl> p sequence(2)->(*1)
[
[0]
[1]
]
pdl> p sequence(2)->(*)
[
[0]
[1]
]
pdl> p sequence(2)->(*,*)
[
[
[0]
]
[
[1]
]
]
Naturally, you need to make sure a use PDL::NiceSlice is in scope. You will have noticed that generally I avoid using it in core PDL code, because it both slows down module-loading, and has limitations; see PDL::LinearAlgebra, where I had to remove a lot of it in favour of building arguments to slice that took account of adding optional parts for PDL::Complex.

Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11144901]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2022-08-14 09:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?

No recent polls found

Notices?