Fellow monks,

As I'm wading through "Mastering algorithms with Perl" and interesting idea was born in my mind. It's been long since I first noticed it - implementation of algorithms is inherently procedural.

Think of a known algorithms book like Cormen et. al. ("Introduction to algorithms"). The pseudo code in it is procedural. Now, it's understandable that all the books with C implementations are procedural.

When we get to Perl we might expect to see other styles - but... we also see procedural (structural, or however you want to call it) code. No OO (OO has little meaning in algorithms, only in interfaces of data structures), no functional.

Even Lisp code of algorithms is usually procedural. See the common implementations of A*, for example.

So, I wonder... algorithm is "description of an operation step-by-step to solve some problem". Does this definition render it naturally to structural code ?

  • Comment on coding of algorithms is inherently procedural

Replies are listed 'Best First'.
Re: coding of algorithms is inherently procedural
by kvale (Monsignor) on May 18, 2005 at 05:20 UTC
    I think that algorithm descriptions are often procedural because a procedural approach represents the lowest common denominator of understanding among students. Everyone has followed step-by-step procedures in their life and good pedagogy strives to provide an impedance match between new material and the students' previous experience.

    Not all books approachs algorithms in a procedural manner, however. Higher Order Perl, for instance treats a number of algorithms from a functional point of view, as do most books on Haskell. Books on SQL algorithms (like my favorite, SQL for Smarties) are mostly declarative in style.

    -Mark

      Giving it some more thought and after some online research, I think I have a clearer view on this:

      We should separate two things: algorithms and programming techniques. Algorithms like "to solve ax = c for x, divide c by a" have nothing to do with programming techniques. It's a way humans are used to think about "stuff to do" in "steps". Our computers are designed this way - no matter what language / paradigm you use, down bottom it compiles to some sort of machine code, which is inherently procedural and linear. Turing machines, the universal model of computation, are as linear as you can get.

      OTOH, programming techniques like functional, logical and OO are just different ways to *implement* algorithms. Some of the repliers raised the issue of different (non procedural) representation of algorithms. That's curious, but still probably less useful to people, who are "designed" to think in a procedural way about solving problems.

Re: coding of algorithms is inherently procedural
by toma (Vicar) on May 18, 2005 at 05:41 UTC
    I recently referred to the Mastering Algorithms book, and as a result used the Graph module. When I started, it seemed that I needed to write a lot of difficult code, and I dreaded the bugs. Instead, my OO graph representation using this module solved the whole problem in about 10 lines of code. No bugs found so far!

    I think the graph module shows an example where the algorithms are naturally object-oriented. They match at least these three hints paraphrased from Damian Conway's ten rules for when to use OO:

    • Data is aggregated into obvious structures.
    • Data forms a natural inheritance hierarchy.
    • Many different operations are applied to one piece of data.

    Other good examples for OO would be matrix and graphics algorithms.

    It should work perfectly the first time! - toma
Re: coding of algorithms is inherently procedural
by Zaxo (Archbishop) on May 18, 2005 at 05:31 UTC

      This is obviously some strange new usage of the word "interesting" I've never been made aware of. *shudder*

      With apologies to D. Adams and A. Dent . . . :)

Re: coding of algorithms is inherently procedural
by demerphq (Chancellor) on May 18, 2005 at 14:00 UTC

    Algorithms are pretty well language agnostic. Knuth makes this point in "Art Of Programming" by designing an imaginary computer (with a number of strange properties, like it could be binary or it could be decimal) and then presenting all of his examples in assembler code targeted for this imaginary computer/cpu enviornment. He says that this is important as programmers have a tendency to use constructs that are as brief as possible in the language they are using, and not necessarily as efficient as possible and that a CPU level understanding is necessary to truely understanding and analysing algorithm efficiency.

    Its important to remember that FP, OO and Imperative programming are all just different tools, none of them really is any better than the others, just more suited to different types of problems, and with their own particular quirks. OO makes it really easy to write bizarre and inefficient solutions, FP has been known to drive folks mad, and Imperative code often is just horribly difficult to understand and maintain, but underneath they all break down to the same set of CPU instructions, and the CPU really doesnt care where those instructions came from.

    ---
    $world=~s/war/peace/g

Re: coding of algorithms is inherently procedural
by dragonchild (Archbishop) on May 18, 2005 at 13:11 UTC
    I think you're conflating "algorithm" with "paradigm". Every algorithm can be expressed in a procedural fashion. It can also be expressed in recursive fashion, functional fashion, or declarative fashion. Every algorithm also has a fashion that it is expressed very simply and one that is torturous to express it in.

    • In general, if you think something isn't in Perl, try it out, because it usually is. :-)
    • "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"
Re: coding of algorithms is inherently procedural
by blokhead (Monsignor) on May 18, 2005 at 13:32 UTC
    When writing a dynamic programming algorithm, a problem's solution is formulated in terms of its smaller subproblem solution. The first thing you do when developing a dynamic programming algorithm is to formulate the recursive optimality constraint among subproblems. For example, optimal matrix chain multiplication:
    m[i,i] = 0 m[i,j] = min { m[i,k] + m[k+1,j] + p[i-1]*p[k]*p[j] } i<=k<j
    Even the Cormen et al book does dynamic programming this way (starting with the recursive optimality). In general, this is enough to describe a memoized dynamic programming algorithm, since the details are always the same for these kinds of problems. I consider this more of a logic programming formulation. It doesn't say "do this, do that", it only gives a production rule for constructing optimal subproblems and some base cases.

    blokhead

Re: coding of algorithms is inherently procedural
by mstone (Deacon) on May 18, 2005 at 18:37 UTC

    What you're seeing is a trick of the light.

    There are infinitely many ways to express algorithms, some of which look nothing like structured code. You can implement any given algorithm with a network of logic gates, for instance, or a cellular automaton. They're both Turing-complete, but neither their definitions nor their operation look much like procedural/structured code.

    It's true that algorithms do support the concept of state, and with it, a primitive concept of time. Any statement that follows the general pattern: "do this, then use the results to do that," defines boundaries between the inputs and outputs of an operation, and establishes the basic idea that one thing needs to happen before another. Yes, that's 'procedural', but it isn't inherently 'structured'

    Writers explain algorithms with structured code because that happens to be what people are used to. Structured programming.. specifically the notion that all programs can be written with:

    • nested functions with a single entry point and well-defined exit points
    • while() statements with some simple variants (for() loops, next, last, and redo modifiers, continue blocks and so on)
    • and if-then-else statements

    is the common denominator of every programming language made since the 1970s.

    It's surprisingly easy to convert structured code to OO code, though.

    An Object is bascially a Random Access Machine, which can be summarized as a block of addressed memory with a set of functions which operate on that memory. To refactor a piece of structural code as OO code, you start by turning all your variables into object attributes, and all your functions into object methods. At that point, you have compatible, if somewhat trivial, OO code. It won't necessarily be good OO code, but it will follow the OO paradigm.

    Basically, though, the fact that you're seeing structured code everywhere is a self-selecting convention. Writers use it because they assume it will be familar to any programmer trained in the last 30 years. If OO becomes the dominant mindset for programming (or FP, or some declarative style, or some formal-methods typesystem), the writers will shift over to a pidgin version of whatever is most popular.

Re: coding of algorithms is inherently procedural
by etcshadow (Priest) on May 18, 2005 at 18:14 UTC
    No. I wouldn't do a good enough job of explaining it, and don't have enough time to try, but the short story is: there are many means of expressing an algorithm that are equivalent in their expressiveness. That's the meat of Church's Thesis (one of the most important things to understand in computer science).

    And that's not just pedantry, either. There are definitely fields of algorithms wherein the most natural way of expressing the algorithm is not procedurally. Take proofs by induction, for example: those are much more naturally represented functionally.

    I think that maybe what's throwing you off is the definition of algorithm you've got there. That's simply not a good definition of the term, as it says "step by step". A better way of describing an algorithm would center more on the fact that it is a complete and unambiguous description of a way to solve a problem, not that it is "step by step". "Step by step" is sort of begging the question, if you will, of how that algorithm is represented (as "steps", in this case)... and that's really imaterial to what an algorithm is.

    ------------ :Wq Not an editor command: Wq
Re: coding of algorithms is inherently procedural
by ady (Deacon) on May 18, 2005 at 18:19 UTC
    ...implementation of algorithms is inherently procedural
    ...algorithm is "description of an operation step-by-step to solve some problem".
    Does this definition render it naturally to structural code ?


    Not so in the strict sense. Consider this common definition (from wikipedia.org)

    ... So far, this discussion of the formalisation of an
    algorithm has assumed the premises of imperative
    programming. This is the most common conception, and it
    attempts to describe a task in discrete, 'mechanical'
    means. Unique to this conception of formalized algorithms
    is the assignment operation, setting the value of a
    variable. It derives from the intuition of 'memory' as a
    scratchpad...
    See functional programming and logic programming for
    alternate conceptions of what constitutes an algorithm.

    The lack of mathematical rigor in the "well-defined
    procedure" definition of algorithms posed some difficulties
    for mathematicians and logicians of the 19th and early 20th
    centuries. This problem was largely solved with the
    description of the Turing machine, an abstract model
    of a computer formulated by Alan Turing, and the demonstration
    that every method yet found for describing "well-defined
    procedures" advanced by other mathematicians could be
    emulated on a Turing machine (a statement known as the
    Church-Turing thesis).
    Nowadays, a formal criterion for an algorithm is that it
    is a procedure that can be implemented on a completely-
    specified Turing machine or one of the equivalent
    formalisms.

    This is also the point demerphq makes with Knuth's imaginary computer and that dragonchild states as different programming paradigms being equivalent in algorithmic power (all "mappable" to a Turing machine). The wikipedia goes on along this line of reasoning:

    One way of classifying algorithms is by their design
    methodology or paradigm. There is a certain number of
    paradigms, each different from the other. Furthermore,
    each of these categories will include many different types
    of algorithms. Some commonly found paradigms include:

    ... etc, read for yourself
    -- allan

    ===========================================================
    As the eternal tranquility of Truth reveals itself to us, this very place is the Land of Lotuses
    -- Hakuin Ekaku Zenji