Re: Dataflow programming on CPU and GPU using AI::MXNet

Do you think you will be able to make a series of posts about how to use AI::MXNet? It's something that I would appreciate reading.

I've been meaning to dip my toes into PDL as a way of working my way into Data Science-y things without needing to do everything according to Python recipes. I think Ovid's post has stirred up some interest in exploring and promoting tools in Perl. There's a lot out there in CPAN already. It just needs some love and blogging, by which I mean taking the time to explain how to do fun things with numbers, etc.

Which is just what we do.

such as 100 PDL exercises and 101 PDL exercises

Ea

Sometimes I can think of 6 impossible attributes before breakfast.

Comment on Re: Dataflow programming on CPU and GPU using AI::MXNet

Replies are listed 'Best First'.
Re^2: Dataflow programming on CPU and GPU using AI::MXNet by bliako (Abbot) on Dec 20, 2019 at 08:02 UTC
Do you think you will be able to make a series of posts about how to use AI::MXNet? Yes, that's my intention. I will post here when I make more progress. The above was my first visit to MXNet-land. And I wanted to stress the fact that 1) you get GPU access on the side and 2) The underlying Dataflow "engine" by MXNet does parallelisation and lazy evaluation as it sees fit without the user being concerned, so this is quite higher up than more traditional ways to do such computations. And applications using these features do not have to be within neural networks... If anyone has any real applications outside neural networks, involving tensor/matrix/vector operations then we can work out a demo of the above 2 features. This (i mean non-neural-network apps) will expand the user-base of AI::MXNet Next stop neural networks. Possibly sooner than a month. bw, bliako	[reply]
Re^3: Dataflow programming on CPU and GPU using AI::MXNet by etj (Priest) on Apr 25, 2022 at 10:43 UTC
PDL natively has forward dataflow (if you opt in by using `$pdl->doflow` - see the next PDL CPAN version for an updated PDL::Dataflow doc that lays all this out, or check out the GitHub version in `Basic/Pod`), and has for decades. Lazy evaluation currently only happens with slice-like operations, but there are plans to switch to lazy evaluation in order to have not only loop-fusion by creating custom operations that would e.g. do the `ab + cd` with only one round-trip from RAM through the CPU to RAM, but also GPU processing. See https://github.com/PDLPorters/pdl/issues/349 for discussion and/or to participate!	[reply] [d/l] [select]
Re^4: Dataflow programming on CPU and GPU using AI::MXNet by etj (Priest) on Aug 04, 2024 at 23:58 UTC
Update from the future: PDL now has better, working dataflow that you can use better, with the `inplace`-like operation `flowing`. It replaces `doflow`. See the latest PDL's PDL::Dataflow and also Example of PDL dataflow to implement 3D space calculations. EDIT: Having read a bit more about "dataflow programming", that's not entirely the same thing I was thinking of. PDL can (at this writing) have automatically-updated ndarrays that depend on other ndarrays. "Dataflow programming" appears to be more stream-orientated, where operations update when all their (smaller, more granular) inputs become valid. PDL can do that a little bit (see the molecule/graph-theory bit in `demo 3d`), but being more event-driven isn't fantastically well-supported yet.	[reply] [d/l] [select]