"... so by-golly ... (Heck, maybe we're just crazy)."

Watch a lot of Scrubs by any chance? :)

First. I completely agree. Each to their own and no harm no foul. That said, there is a lot of Haskell I like, and despite billing myself upfront as a failed Haskell programmer, I still hold out hope of making progress in using it. Though I may need to transition to it through one or two other languages first. Clean is at the top of my list at the moment. The biggest problems I have with clean are the IDE and the library documentation/tutorials which concentrate too much on the graphical IO capabilities.

If you are the same anonymonk who wrote this, then the paper you cite is far and away the best example of the type of criticism I was levelling against many of the Haskell tutorials. Of course, it isn't a tutorial and isn't aimed at a non-math audience, so it can be forgiven for its use of notation and abstract theory without explanation. None the less, it serves as an example.

To quote, selectively from that document:

The above functional program is thus both a mathematical definition of fib and at the same time an algorithm for computing it. One of the enduring myths about functional programming languages is that they are somehow non-algorithmic. On the contrary, the idea of functional programming is to present algorithms in a more transparent form, uncluttered by housekeeping details.

The seemingly close fit between program text and mathematical reasoning accounts for a large part of the appeal of functional languages (together with their conciseness and expressive power) especially in a pedagogical context.

One of the things we say about functional programming is that it’s easy to prove things, because there are no side effects.

Much of which is summed up by another quote:

"The seemingly close fit between program text and mathematical reasoning accounts for a large part of the appeal of functional languages...."

This is my primary angst with Haskell, and the whole FP movement in general. The theory goes that by making a language resemble existing math notations, and have the compiler use reduction to produce the code, one can go directly from mathematical proof to bug-free code and achieve programming utopia. (Yes. I've exaggerated for effect.)

But let's think about mathematical proofs for a few moments.

In the summer of 1993, after seven years of dedicated work, and more than 30 of increasingly serious and adept casual interest, Andrew Wiles announced that he had climbed the Mount Everest of mathematics and solved the problem that had alluded all of the best minds in his field for the preceding 200 or more years. He had proved Fermat's Last Theorem.

It was hailed by the press, and within the mathematics community as a triumph. But, over the next few months, it was Wiles himself that discovered the flaw in the proof. Of course, he then went on to correct the proof, and get it verified by his peers, and will go down in history as the man that climbed that mountain.

But, how many peers has he? How many people are there that can actually say they understand his proof enough to verify it? Another way of asking that question is how many people could, if presented with the original and the final proofs, could work out which was which, unless they were already familiar with them? 1000? 100? 10?

Again, that is an extreme example, but the point is that 'proofs' can be wrong. And it is much harder to verify a proof than a program. You can run a program and subject its results to tests. Something you cannot easily do with formal mathematical notation. Of course, a big part of the desire for FP is the ability to have a compiler that takes standard mathematical notation and converts it to a running program. But then, you not only have to verify the notation, you also have to verify the compiler that does the conversion, and the results it produces, and the results that what it produces, produce.

And that's where I was coming from when I wrote in my post above:

Chicken & egg

So, it's a chicken and egg situation. If you had a provable implementation of a compiler that was built from provable descriptions of its algorithms, then you could use it to build (implement) programs from provable descriptions of provable algorithms.

Until then, you will need to test programs--statistically. And as long that is true, there will never be a 100% guaranteed, bug-free program.

But it is stated more formally at the end of the Total FP paper you cited:

Theorem: For any language in which all programs terminate, there are always-terminating programs which cannot be written in it - among these are the interpreter for the language itself.

Going on to conclude:

We can draw an analogy with the (closely related) issue of compile-time type systems. If we consider a complete computing system written in a typed high level language, including operating system, compilers, editors, loaders and so on, it seems that there will always be at least one place – in the loader for example – where we are obliged to escape from the type discipline. Nevertheless many of us are happy to do almost all of our programming in languages with compile time type systems. One the rare occasions when we need to we can open an escape hatch, such as Haskell’s UnsafePerformIO.

There is a dichotomy in language design, because of the halting problem. For our programming discipline we are forced to choose between

A) Security - a language in which all programs are known to terminate.

B) Universality - a language in which we can write

(i) all terminating programs

(ii) silly programs which fail to terminate

and, given an arbitrary program we cannot in general say if it is (i) or (ii).

Five decades ago, at the beginning of electronic computing, we chose (B). If it is the case, as seems likely, that we can have languages of type (A) which accommodate all the programs we need to write, bar a few special situations, it may be time to reconsider this decision.

And that's my problem with much of the hyperbole that surrounds and infuses FP. This paper is saying that "we don't need to deal with errors, exceptions, dirty data etc.", or "need a language that is Turing complete" (elsewhere in the paper) except on "rare occasions", but that just doesn't make sense to me.

Eg. There is that co-recursive proof that a number is odd (or even). And that converts nicely into a pure Haskell program. But, even if we assume that the compiler will convert that program into correct code, as soon as we need to fetch that number into the program, rather than embed it, all bets are off. The program may not terminate, because the user never enters a number and hits enter; or the file make contain text, or be empty; or the socket connection may have dropped; or...

In a practical programming language, as opposed to an academic research language, you cannot ignore the inconvenient truth of reality.

I remember Pascal in its early days where 'files' were just internal data structures and you didn't have to deal with the outside world. great for teaching,but totally impractical for anything commercial. Inevitably, along came Borland with Turbo-Pascal and there were some amazing and important programs written using Pascal. Later (I think), that became Delphi, and (I think) that is still being used.

Haskell is already a powerful, elegant, practical programming language. It doesn't need to sell itself on the basis of lofty, theoretical(*) goals. It is already "condemned to be successful". Like Quantum::Superpositions, once the hype has faded and gone, what you are left with are some very important, and very practical, useful ideas and code that handle the real world with aplomb and stand up along side other real world languages on that basis.

(* And I would say, unobtainable--but there are a lot of very bright minds, much brighter than me, that are pursuing them, so I'm probably going to be proved wrong!)

What is missing, to my mind and in my experience of looking for on-line Haskell tutorials, is a description of a moderately complex, real-world, warts-and-all problem, and a worked solution. Forget all the cutesy, convenient (even if theoretically interesting and important) examples of Fibonacci series, and quick-sorts, and NDA language parsers, and tree structures. These are just a bad as corporate/genetic hierarchies examples in the OO world. Work through something useful.

In one or two of SPJ papers there is reference to a Haskell programmed HTTP server. 1500 lines long, with near Apache like performance but less features. Now that would make a fine basis for a tutorial with fully worked example. The source code is probably around somewhere for download, but that's much less than half of what's required.

What is needed is insight into the mind of the expert Haskell programmer on how they approach tackling such a project. Given the HTTP/1.1 spec as a starting point and the standard library, where do they start? How do they proceed? What mistakes did they make; how did the compiler inform them of those mistakes; how did they interpret those error messages and how did they resolve them? Now that would be a tutorial that might allow me to make the transition to the mode of thinking required.

F*** all the theory behind monads, or the type system, or strict -v- lazy -v- pure. Show me the code. But more importantly, show me how you arrived at it.

That said, there is a lot of Haskell I like, and despite billing myself upfront as a failed Haskell programmer, I still hold out hope of making progress in using it. Though I may need to transition to it through one or two other languages first.

Have you taken a look at Prolog? It requires a very different mindset, yet without static typing or monads.

And it is much harder to verify a proof than a program. You can run a program and subject its results to tests. Something you cannot easily do with formal mathematical notation. Of course, a big part of the desire for FP is the ability to have a compiler that takes standard mathematical notation and converts it to a running program. But then, you not only have to verify the notation, you also have to verify the compiler that does the conversion, and the results it produces, and the results that what it produces, produce.

Hmm. You don't have to prove anything. You can still run your functional program and test it like you normally would (also take a look at: QuickCheck). But now you *get* the option to prove (informally) and reason about your programs if you so desire. It's an extra bonus feature that you don't get with an imperative program (New and Improved! Now with 20% more features!)

Although I'm probably getting *really* OT, here's an excerpt I like from "The Way of Z: Practical Programming with Formal Methods" by Jonathan Jacky:

Many programmer believe that fomal specification are not useful. They believe that the program text -- the code itself -- can be the only really complete and unambifuous description of what a program does. This vew holds that a formal specification is nothing more that the program written over again in another language. It misinterprets Z to be some kind of very high-level programming language.

This example shows they are wrong. See for yourself; Here is the code in C.

int f(int a)
{
  int i, term, sum;
  
  term=1; sum=1;
  for (i=0; sum <= a; i++) {
       term=term+2;
       sum=sum+term;
  }
  return i;
}
[download]

The code couldn't be simpler. It is well structured and very brief -- in fact it looks trivial. But what does it do? Is seems to be adding up a series of number -- but why? And it returns the counter, rather than the sum -- is that a mistake? Try to answer before you turn the page.

You can find the answer on page 34 by searching for the book on books.google.com.

And that's my problem with much of the hyperbole that surrounds and infuses FP.

I think our biases must be pretty different. Oh, sure, there are going to be some enthusiastic advocates of any language, but other than a few fly-by-night blog posts, I have a hard time seeing the hyperbole that surrounds and infuses FP.

This paper is saying that "we don't need to deal with errors, exceptions, dirty data etc.", or "need a language that is Turing complete" (elsewhere in the paper) except on "rare occasions", but that just doesn't make sense to me.

Hmm. Maybe experience comes to play here also. I 100% agree with the paper (incidently it one of my favorite CS papers, my number one favorite probably being Can Programming be Liberated from the von Neumann Style?). Most of the programs I write (in any language, and I use plenty of Perl) are about (guestimating) 90+% purely functional in nature (engineering analysis mostly). In fact, I don't think I've probably ever professionally written a program where I didn't think I had at least a rough idea complexity of the algorithm.

Show me the code.

Maybe something like xmonad, is real world, yet small enough to get your feet wet?

For your test, I ran this:

#! perl -slw
use strict;

sub f {
    my $a = shift;
    my $term = 1;
    my $sum = 1;
    my $i;

    for( $i=0; $sum <= $a; $i++) {
        $term = $term + 2;
        $sum = $sum + $term;
#        print "i:$i term:$term sum:$sum";
    }
    return $i;
}

print "f($_) = ", f( $_ ) for -4 .. 36;
[download]

The terminating value for the test loop may indicate to you that I had a good idea of what it was doing before I ran it. I still haven't looked up the book. I'd term f as 'the integer root', but there is probably some proper term for that.

Now I'll turn the page...back.

So, not far wrong, but did I pass the test? I read on a few pages and it makes great play of how the Z axiomatic definition carries more information--such as the possible ranges of inputs and outputs from f()---but that ignores that the C definition could also have captured that information.

unsigned int f( unsigned int a ) {
  unsigned int i, term, sum;
  
  term=1; sum=1;
  for (i=0; sum <= a; i++) {
       term=term+2;
       sum=sum+term;
  }
  return i;
}
[download]

And actually, that carries more information. For one thing, on practical computer systems, variables are of finite size and can therefore only deal with finite ranges of values. It maybe mathematically convenient to reason about functions in terms of the infinite range of natural numbers, but in practice, that reasoning falls down because real hardware overflows, underflows and otherwise does not behave as mathematical theory would have it behave.

The Z definition won't tell you that for inputs above 2**31, (using the original C formulation on a 32-bit processor) that you are going to get bizarre outputs because the intermediate term sum will overflow.

In the Total FP paper, the author says:

RULE 1) All case analysis must be complete. So where a function is defined by pattern matching, every constructor of the argument type must be covered and in a set of guarded alternatives, the terminating ‘otherwise’ case must be present. In the same spirit any built in operations must be total. This will involve some non-standard decisions - for example we will have

0 / 0 = 0

Yeah, right! Good luck with that.

Now how hard do I have to think to come up with a scenario where the undetected, unreported, silent conversion of erroneous input into a mathematically convenient lie causes the reactor to go critical or the patient to receive a massive dose of something lethal? But the mathematicians are happy, so what the hey! Again, just a dramatisation.

Have you taken a look at Prolog?

Yes. I did a Prolog course at college back before the dawn of time. And more recently, about 10 years ago, I had to do some real work with an inferencing engine that used a dialect of Prolog. It does take a very different mindset. And the last time I frequently wrote short brute force C programs to verify the results I was getting from the inferencing engine. They usually ran longer, but they were much faster to produce and I had far more confidence in the results.

Oh, sure, there are going to be some enthusiastic advocates of any language,

It's not "enthusiastic advocates" that disturb me.

• It's the claim that Haskell's purity allows provability of its programs which results in more reliable software.

  This claim is not restricted to appearing in fly-by-night blogs of enthusiastic advocates. I think this claim is bogus. I think I proved it was bogus using evidence from one of your favorite papers above.

• It's the claim that the absence of side effects increases provability (or "the ability to reason about") of Haskell programs.

  It may increase either or both, for those parts of Haskell programs that are purely functional, and for mathematicians and those that are used to thinking in that way.

  But, as I've alluded to elsewhere in the thread, Haskell programs--every useful Haskell program--does have side effects. It may deal with them better than many other FP languages, and possibly other non-FP languages, but they are still there.

  But is there any hard evidence that useful Haskell programs are more bug free than other languages?

• It's the claim that Haskell increases programmer productivity.

  It may do so, for expert Haskell programmers but if the World Programming Council banned all programming languages except pure lazy functional languages tomorrow, how productive would the worlds (1 million?) programmers be the day after? Or the week after? Or 1 year from now?

  How many of the worlds programmers would successfully make the transition from imperative to functional programming?

  Even when they had, they would still have to program in a world where data gets corrupted; where strings have to mutate into integers and reals; where disks fail; communications channels go away; where files are bigger than memory and have the wrong line endings; where DoS attacks abound; where input are accidentally supplied in inches instead of millimetres.

  Will their conversion to Haskell make them any more capable of dealing with these situations?

  Even within the field of Maths itself, most of the best math packages are still written in Fortran.

IMO Haskell should be downplaying the arguments about whether the rules underlying Monads are correct, or whether MonadPlus is better. And whether Lazy is a good or a bad thing.

I didn't get to go through the Xmonad stuff, but there doesn't appear, at first glance at the page you linked, to be any tutorial associated with it? I did say show me the code, but went on to say "But more importantly, show me how you arrived at it."

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

I dearly hope that you will come back and see this.

I cannot thankyou enough for the link to Backus. I understand why this is your favorite. I think it may well become one of mine. Actually I think it already is, and I haven't finished reading it yet.

What a refreshing change from most of the other papers I have been reading. Clear points, made with clear explanations and clear examples. Where he uses algebraic notation, each symbol is briefly described, in-situ.

By the time I finished section 11, I understood why I so want to like FP. Simplicity, brevity, clarity. Thankyou.

30 years on, his presentation style shows why the greats are.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."