in reply to Re^2: A mini-language for sequences (part 1) in thread A mini-language for sequences (part 1)
The following are some (lightly edited :) initial reactions to your follow post.
Note: Initial.
There are some positive reactions lower down, but I thought that I would get these out of the way first in the hope that the later reactions would erase any bitter taste these might leave.
- I never said "this crazy FP stuff".
- I get the FP -v- FP-inspired difference.
- Phrases like "spectrum of problems", "shape of the problems", "shape-flexing capability", "code at a higher level" & "think at a higher level" all leave me...um...suspicious.
What is he trying to conceal with flowery language?
Put that down to having suffered the hyperbole of too many "new programming paradyms". (I realise FP isn't new.)
- "The reason you perceive the fp-inspired code as hard to follow is because your brain hasn't "internalized" the meaning of products and zips and the rest of our new vocabulary. To you, these words probably seem like gratuitous functional jargon."
My question is: "Do I need this new vocabulary?".
- "But the words have meaning. When I saw the example problem as first posted by the Seeker of Perl Wisdom, I immediately thought, "Ah, he wants the count of each value from the zipped products." And the code immediately followed."
Fair enough, but then you had to construct a fairly complicated library of intrastructure in order to be able to code the solution in terms of your vocabulary.
When I saw the example problem, I thought: "He want's to iterate over the two arrays of strings and count the unique pairs of aligned characters".
- "...iterate over the two arrays..."
Nested for loops.
- "...count the unique pairs...".
Increment a hash.
- "...pairs of aligned characters."
Two substrs and another for loop.
The code I posted fell straight out of that. No need for any auxillary vocabulary. No need to code a bunch of extra infrastructure. Perl already has all the vocabulary and in-built infrastructure required.
- "That's the reward. To have the right words to describe the problem. To have more ways of stringing together ideas. To reduce the gulfs between problem and understanding, understanding and code." & "If this sounds like worthless babble, that's OK."
{nods} Yes. It does a bit :)
- I didn't appreciate functional programming until I had used it to solve real-world sized problems for over a year. Only then could I "feel" fp in my bones and appreciate its potential.
I've been trying to get into FP since I first starting reading articles on Lisp in Byte magazine circa. 1979/80. I currently have Common Lisp, Scheme, and most recently Haskell.
- Have you read Why Functional Programming Matters? If not, read it. Now."
I pulled the PDF and tried to read it. After a little while it began to seem familiar. Maybe it appeared on Byte, or CoACM or something else I've read a long time ago? Maybe it was referenced from such an article and I tracked it down and tried to read it? Maybe it's just the code examples that seem familiar from some other FP tutorial I've read down the years? Whatever, the PDF sits open on my desktop and I will endevour to try and read it to the end, but I doubt that I made it that far last time either.
The problem is that in common with many other FP (and, notoriously OO) tutorials, it selects very specifically limited and condusive (to their aims) examples. It then spends an inordinate amount of time concentrating on how to construct the solution--in it's own, very esoteric, terms. Your real world examples are infinitely better (IMO).
But the single line that sums up my feelings of what I've seen of FP tutorials and advocacy comes from the Abstract on the page you linked to above.
Since modularity is the key to successful programming, functional languages are vitally important to the real world.
The problem with that statement is that it implies that the only way to achieve modularity, is through FP--which is clearly not the case.
- I love Perl Monks, but don't look at the fp examples on Perl Monks as your window into the fp world. Take a look at the stuff going on in the Haskell Communities Report. Read some of the papers from the ICFP proceedings. That's where you'll find the good stuff.
As I mentioned, my FP exploration goes back a long way, and my research is definitly not confined to what I see on PM. My exploration of Haskell is very new (and did come about because of articles mentioning it here at PM), so thankyou for those links--they will give me many hours of good reading.
I hope that doesn't all seem negative? It isn't meant to be. I'm continuing to enjoy reading your article and pursue various angles that lead from it. Thankyou for writing it and I look forward to Part 2.
For me, the most significant thing that came out of your response to my initial post is:
Instead of passing in \(@site1,@site2) you can pass in any number of input arrays. And the same code works, as is.
Now that is a benefit that I can understand and internalise :).
I did try to download your code and try this out, but there seem to be some bits missing to allow the examples to work? Perhaps in part 2. This is what I extracted from your OP: {**Code moved to bottom of post**} but I couldn't work out how to fix it up so that I could try out the example with 3 or more lists?
Anyway, the concept of writing a generic solution to the nested loops problem is one that I've been trying to get to grips with for some time. The upshot of your article (for me) is that I've finally gotten around to putting some concerted effort into it, and I came up with a solution. It doesn't look much like FP in it's construction, but it does rely heavily of constructing anonymous subroutine iterators, and constructing lists of these. And combining these iterator functions to contruct a higher order iterator. So, having (probably) butchered the FP vocabulary to death :), this is what my solution looks like:
#! perl -slw
use strict;
sub loops {
my @iters = map{
my @list = ( @$_, undef );
sub { $list[ @list ] = shift @list || () };
} @_;
my @rv = map{ $iters[ $_ ]() } 0 .. $#iters;
return sub {
my $rv = [ @rv ];
for my $i ( reverse 0 .. $#iters ) {
$rv[ $i ] = $iters[ $i ]() and return $rv;
$rv[ $i ] = $iters[ $i ]();
}
return;
};
}
my $iter = loops [ 'a' .. 'd' ], [ 1 .. 4 ], [ 'me', 'you' ];
print "@$_" while $_ = $iter->();
__END__
[13:51:21.14] P:\test>loops
a 1 me
a 1 you
a 2 me
a 2 you
a 3 me
a 3 you
b 1 me
b 1 you
b 2 me
b 2 you
b 3 me
b 3 you
c 1 me
c 1 you
c 2 me
c 2 you
c 3 me
I'm pretty pleased with that. Your article has been invaluable to me because it has prompted me to look at this problem in a completely different way to that in which I have been looking at it in the past. And it has triggered a lot of thoughts about using similar techniques to generalise several other pieces of code I have kicking around.
So thankyou again for posting your very thought provoking article. I really look forward to part 2.
I think the conclusion I'm drawing is that it's possible to learn and adopt techniques from other languages and styles of coding, without having to adopt the vocabularies and working practices wholesale. Once you understand (if I have?), the underlying techniques, it becomes possible to utilise them whilst retaining the flavour of the language in which your writing.
The code I extracted from your OP. Moved to the bottom to avoid interupting the main post. (Wouldn;t it be nice if readmore tags only expanded when asked? Even if viewing the containing post directly).
#! perl -w
use strict;
use List::Util qw( reduce );
use Data::Dumper;
#package Sequence;
#sub new {
# my ($proto, $seq) = @_;
# bless $seq, $proto;
#}
#sub seqsub(&) {
# Sequences->new(@_);
#}
sub seq {
my ($i, $elems) = (0, \@_);
seqsub {
$i < @$elems
? ( $elems->[ $i++ ] )
: do { $i = 0; () };
}
}
sub enumerate {
local $Data::Dumper::Terse = 1;
local $Data::Dumper::Indent = 0;
my ($i, $seq) = (0, $_[0]);
while (my @val = $seq->()) {
@val = map { ref ($_) ? Dumper($_) : $_ } @val;
printf "%2d => %s\n", $i++, "@val";
}
$seq;
}
sub seq_prod2 {
my ($s, $t) = @_;
my @sval;
seqsub {
@sval = $s->() unless @sval;
my @tval = $t->();
@tval ? ( @sval, @tval )
: do { @sval = $s->();
@sval ? ( @sval, $t->() ) : () };
}
};
sub seq_prod {
reduce { seq_prod2($a,$b) } @_ ;
}
sub seqs {
map seq(@$_), @_;
}
sub seq_from_spec {
seq_prod( seqs(@_) );
}
sub seq_foreach {
my ($seq, $fn) = @_;
while (my @val = $seq->()) {
$fn->(@val);
}
$seq;
}
sub seq_foreach_from_spec {
my ($spec, $fn) = @_;
seq_foreach( seq_from_spec( @$spec ), $fn );
}
sub seq_filter {
my ($seq, $filter_fn) = @_;
seqsub {
my @val;
1 while @val = $seq->() and !$filter_fn->(@val);
return @val;
}
}
sub seq_map {
my ($seq, $fn) = @_;
seqsub {
my @val = $seq->();
@val ? $fn->(@val) : ();
}
}
sub seq_reset {
my $seq = shift;
if ($seq) {
1 while $seq->();
}
$seq;
}
sub seq_zip {
my $seqs = seq( @_ ); # seq of seqs (!)
my $seq_count = @_;
seqsub {
my @outvals;
while (my $seq = $seqs->()) {
if (my @val = $seq->()) {
push @outvals, @val;
}
else {
seq_reset( $seqs->() ) for 1 .. $seq_count;
seq_reset( $seqs );
return ();
}
}
return @outvals;
}
}
my @site1 = qw( AATKKM aatkkm );
my @site2 = qw( GGGGGG gggggg );
my %counts;
seq_foreach_from_spec( [ \(@site1, @site2) ],
sub {
seq_foreach(
seq_zip( ( map seq(split//), @_ ) ),
sub { $counts{"@_"}++ }
)
}
);
print Dumper(\%counts), "\n";
Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
"Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
Re^4: A mini-language for sequences (part 1)
by tmoertel (Chaplain) on Nov 07, 2004 at 23:50 UTC
|
First, thank you for your most-recent response. There are many
interesting ideas within it, and I want to get to them. But before we
do that, let's put our discussion within proper context.
I must make clear that we have wandered onto dangerous ground. We
are using my meditation as the testing ground for your conclusions
about the merits functional programming, and it was never indented to
be anything other than a meditation.
To be blunt: If you're looking at my mediation as anything
other than a meditation, you're making a mistake. My meditation
is not meant to be practical. It is not meant to be advocacy for
functional programming. It is a meditation: a discourse intended to
express my focusing on one idea – sequences – and going
with it as deep as I can.
The point of the meditation is not to say that "this way is
better." The point is to ignore earthly concerns that normally hold
us back (such as practicality) and to keep moving forward with an idea,
just to see where it takes us. Maybe we'll end up at a really cool
place. Maybe not. Regardless, we hope to learn something from the
journey, if not from the destination.
At certain points along our journey you have asked, "Why did we
walk this strange path to get to this spot? I know a road that most
people would think is better."
But we're not trying to find the best path just now. We're trying
to walk a different path to see if we can learn anything. Instead of
comparing our path to the road and dismissing it as inferior after the
first few miles, can we not keep on walking, just to walk, and to see
if we encounter anything new or valuable?
In the end, maybe we will still prefer the road, but then again,
maybe knowing the path will come in handy. Maybe some destinations
will best be reached by traveling both road and path. Maybe
the path will be a valuable addition to our options for future travel.
To come at it practically, don't judge my examples. Judge the
ideas the examples represent. But only later, after due
consideration.
Now, on to specific items.
My question is: "Do I need this new
vocabulary?"
Whether you need any new vocabulary depends on the vocabulary and on
the problems you are trying to solve. If the cost of learning the
vocabulary and then using the vocabulary to solve your problems is
less than the cost of solving your problems otherwise, then you
probably ought to use the vocabulary.
Do you need my meditation's sequence vocabulary? Most likely, no.
It exists only for this meditation. If I were to create a CPAN module
from it, as some have suggested, the resulting library would look and
be organized much differently than presented here. The goals of
meditation and of practical programming are far apart.
I would, however, argue that much of the vocabulary of ideas and
techniques within the meditation are useful and could earn its place
in most personal programming toolboxes.
But the single line that sums up my feelings of what
I've seen of FP tutorials and advocacy comes from the Abstract on the
page you linked to above.
Since modularity is the key to successful programming, functional languages are vitally important to the real world.
The problem with that statement is that it implies that the only way
to achieve modularity, is through FP--which is clearly not the case.
I don't think that Hughes implies that FP is the only way to achieve
modularity. Rather, he argues that FP (especially modern FP) reduces
the cost of modularity to the point where it can be used, and its
benefits reaped, almost ubiquitously – not just at object or
function boundaries but at much finer granularities.
In most languages, the smallest unit of modularity is the object or
function. But in Haskell, for example, I can go much finer. I can
slice function calls in half, turn operators into functions and
functions into operators, flip arguments around, and do a whole lot
more to reshape code until it fits my needs. The potential for code
reuse goes up dramatically because I can modularize on a much finer
granularity than before; the artificial distinctions that other
languages impose between language and library cease to exist.
Libraries aren't merely collections of functions but real vocabularies
that extend the language.
I've been trying to get into FP since I first starting
reading articles on Lisp in Byte magazine circa. 1979/80. I currently
have Common Lisp, Scheme, and most recently Haskell.
To help me understand and quantify where you're coming from, how many
hours would you conservatively estimate that you have spent writing
functional code in the last year?
The problem is that in common with many other FP (and,
notoriously OO) tutorials, ("Why Functional Programming Matters")
selects very specifically limited and condusive (to their aims)
examples. It then spends an inordinate amount of time concentrating on
how to construct the solution--in it's own, very esoteric, terms. Your
real world examples are infinitely better (IMO).
The problem for people who write about FP to non-FP audiences is that
the introductory slant of their writing prevents the use of complex
example problems, where FP's merits are readily visible. Anybody can
solve simple problems in any language, and so readers understandably
wonder why FP is so great when their "normal" solutions to the same
example problems would arguably be better.
For this reason, some FP authors will pick simple example problems
that just happen to be hard to solve in non-FP languages. While the
intent is to keep the problems simple enough that readers can follow
them and yet "hard" enough to show the merits of FP, many readers
conclude that the examples are contrived or even rigged to put FP in
an artificially glowing light.
Sadly, this is a case when the limitations of the medium undo the
message.
FP languages let you manipulate pieces of programming logic as
easily as most languages let you manipulate data. As a result, it is
easy to create FP programs that morph their shape to match the
structure of the problems they solve. Thus the straightforward FP
solution is often general enough to scale from the simplest to the
most complex problems within a wide spectrum of related problems.
This is deeply cool magic, but you won't appreciate it until you
experience it. And you won't experience it in introduction-to-FP
papers. You can't put examples like that into a paper. The best way,
then, to read the papers is to study the examples carefully,
understand the thinking behind them, and then meditate on how far that
thinking will take you. Often, it will take you surprisingly far.
I hope that doesn't all seem negative? It isn't meant
to be. I'm continuing to enjoy reading your article and pursue various
angles that lead from it.
No, I don't take it as negative. But I do see a slight danger of your
drawing conclusions about FP without due consideration. Be deliberate
about forcing your mind to remain wide open while you invest serious
time in studying and coding functional programs.
The best advice I can give you is to force yourself to write
functional code, the more the better. If you're already looking at
Haskell, that's your best bet. (There is no finer modern FP language,
IMHO.) Get a copy of Paul Hudak's The Haskell School of
Expression: Learning Functional Programming through Multimedia
and work through it. Solve the problems. Don't skip the proofs.
For me, the most significant thing that came out of your response to my initial post is:
Instead of passing in \(@site1,@site2) you can pass in any number of input arrays. And the same code works, as is.
Now that is a benefit that I can understand and internalise :).
I did try to download your code and try this out, but there seem to be some bits missing to allow the examples to work?
I think the problem is that you commented out the OO portion of my
code but forgot to recode seqsub accordingly. Add the
following to your code (or uncomment the OO portion that you commented
out), and you should be good to go:
sub seqsub(&) { $_[0] }
Here's our example solution put into a subroutine so that we can call it repeatedly with differently "shaped" inputs:
sub count_zipped_combo_vals {
my %counts;
seq_foreach_from_spec( \@_, sub {
seq_foreach(
seq_zip( ( map seq(split//), @_ ) ),
sub { $counts{"@_"}++ }
)
});
\%counts;
}
Now we can see how will it handles problems of different shapes.
We'll try 1-, 2-, and 3-array cases, but the same code should work
for any number of arrays greater than zero.
# a few problem cases to try
my @site1 = qw( AATKKM aatkkm );
my @site2 = qw( GGGGGG gggggg );
my @site3 = qw( XXXXXX ++++++ yyyyyy );
# ... and so on ...
my @sites = \(
@site1, @site2, @site3, # ... and so on ...
);
# try 1-, 2-, and 3- array cases
print Dumper( count_zipped_combo_vals( @sites[0..$_] ) )
for 0 .. $#sites;
# $VAR1 = {
# 'A' => 2,
# 'k' => 2,
# 'a' => 2,
# 'M' => 1,
# 'T' => 1,
# 'K' => 2,
# 'm' => 1,
# 't' => 1
# };
# $VAR1 = {
# 'K G' => 2,
# 'A G' => 2,
# 'm g' => 1,
# 'a g' => 2,
# 'A g' => 2,
# 'M G' => 1,
# 'k g' => 2,
# 'k G' => 2,
# 'T G' => 1,
# 'a G' => 2,
# 'm G' => 1,
# 't G' => 1,
# 'K g' => 2,
# 'M g' => 1,
# 't g' => 1,
# 'T g' => 1
# };
# $VAR1 = {
# 't G y' => 1,
# 'T G +' => 1,
# 'K G X' => 2,
# 'a G X' => 2,
# 'm G +' => 1,
# 'a G +' => 2,
# 'T g y' => 1,
# 'm g +' => 1,
# 'a g +' => 2,
# 'k G X' => 2,
# 'a g X' => 2,
# 'm g X' => 1,
# 'T g X' => 1,
# 't G +' => 1,
# 'M G y' => 1,
# 'k g y' => 2,
# 't g y' => 1,
# 'm G X' => 1,
# 't G X' => 1,
# 'K G +' => 2,
# 't g +' => 1,
# 'a g y' => 2,
# 'T G X' => 1,
# 'm g y' => 1,
# 'm G y' => 1,
# 'A G X' => 2,
# 'M g +' => 1,
# 'k g +' => 2,
# 'k g X' => 2,
# 'A g X' => 2,
# 'A g +' => 2,
# 'M g y' => 1,
# 'k G +' => 2,
# 'a G y' => 2,
# 'M G +' => 1,
# 'A g y' => 2,
# 'K g X' => 2,
# 'K G y' => 2,
# 'K g y' => 2,
# 'K g +' => 2,
# 'k G y' => 2,
# 'A G y' => 2,
# 'T g +' => 1,
# 't g X' => 1,
# 'M G X' => 1,
# 'M g X' => 1,
# 'T G y' => 1,
# 'A G +' => 2
# };
Thanks again for taking the time to consider our discussion. I hope
that in the end your will find the journey to be the reward.
Cheers, Tom
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
I hope you can appreciate my skepticism when I respond to:
FP languages let you manipulate pieces of programming logic as easily as most languages let you manipulate data. As a result, it is easy to create FP programs that morph their shape to match the structure of the problems they solve. Thus the straightforward FP solution is often general enough to scale from the simplest to the most complex problems within a wide spectrum of related problems.
with the simple phrase "you have drank from the FP Koolaide".
Fortran solved everything. Then Pascal solved everything. Then Smalltalk solved everything. Then Prolog solved everything. Now you're saying FP solves everything.
Sure.
Having been around the block a few times, lemme say that I can certainly see FP being useful as yet another approach to a problem. But your unrelentless praise for the latest new thing should be taken in context of the history of discovering just another interesting programming technique.
FP will be good for some problems, horrible for others. Just like every other style discovered before it. Understand that, and you'll understand why everyone's not jumping on your bandwagon.
| [reply] [Watch: Dir/Any] |
|
(Updated Mon Nov 8 14:36 EST 2004: Clarified wording.)
Randal,
Thanks for jumping into the conversation. Your viewpoint is
always welcome.
I am concerned, however, that you have quoted my most flowery
praise for FP and used it to dismiss much of my views as mere Koolaide
visions without giving the whole of my views due consideration.
To be clear, I never claimed that FP was the solution for
"everything" or that it was the one, true way. If you genuinely
believe that this was my claim, I am sorry for not having presented my
beliefs more clearly.
Let me take the opportunity to do that now:
- FP is not a panacea.
- FP is another tool for the toolbox (but a very powerful one).
- FP, like all tools, reduces the cost of some things; increases the cost of others.
- Modern FP languages (like Haskell) are not your grandfather's FP language. If you haven't looked at FP in the last 5 years, look again.
- Modern FP languages have reduced the cost of function manipulation to the point where small-scale reuse becomes practical.
- Small-scale reuse provides significant cost reductions.
- Much like Perl, modern FP languages require a serious investment of time before they become understood (and appreciated).
- Most people who dismiss FP haven't spent much time writing code in modern FP languages.
Regarding the following:
Having been around the block a few times, lemme say
that I can certainly see FP being useful as yet another approach to a
problem. But your unrelentless praise for the latest new thing should
be taken in context of the history of discovering just another
interesting programming technique.
Having also been around the block a few times, let me say that my
appreciation of FP comes not from having been smitten by the latest
new thing but rather from the investment of much time and effort. Since the late
1980s I have been coding in and evaluating FP
languages alongside the more common languages such as C, C++, and Perl
that I use to earn a living in industry. However, it is only recently
that I have watched FP flower and come into its own.
While most of industry has turned its back on FP, having long ago
made up its mind, the academic world has been working. Steady, slow,
and ceaseless, they have been working. And they have made much
progress. Please don't dismiss that progress without due
consideration. If you haven't spent serious time coding in a modern
FP language lately, please consider the possibility that
you're drawing conclusions from outdated first-hand knowledge.
FP will be good for some problems, horrible for
others. Just like every other style discovered before it. Understand
that, and you'll understand why everyone's not jumping on your
bandwagon.
To help me put this comment in perspective, how much coding have
you done in modern FP languages lately?
The reason I ask is because if a man came up to me and summed up Perl by saying that
Perl was good for some problems, horrible for others, just like every
other language discovered before it, and that's why not everybody is
jumping on the Perl bandwagon, I would want to know how much he had
used Perl. It's not that I disagree with his statement, which is
clearly true for just about everything, but that I don't find the
statement as useful as knowing why some people are on the
bandwagon. I would want to know whether the man could tell me. I
would want to know whether the man understood Perl's strengths and
weaknesses and could provide useful information that would help me make sensible
choices about when to use Perl.
Cheers, Tom
| [reply] [Watch: Dir/Any] |
|
|
|
To help me understand and quantify where you're coming from, how many hours would you conservatively estimate that you have spent writing functional code in the last year?
Methinks you're asking, where do you sit in The Evolution of a Haskell Programmer.
To wit I reply, currently indeterminate, but I'm aiming for "Tenured professor" :)
Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
"Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
| [reply] [Watch: Dir/Any] |
|
|