RE: Sorting a list of IP addresses (aka Why I hate Big O)

in reply to Sorting a list of IP addresses (aka Why I hate Big O)

Alright. What did big O teach you... did it tell you how fast something was going to run? no. Big O was a measure of asymptotic complexity... the entire theory of Big O was: for N data items, proportionally, how many steps M will I have to take. What it was designed to tell you was that: If an algorithm of Big O of N took 10 seconds to do 10 items, when we increased the number if items to 20, it would probably take somewhere near 20 seconds.

Comparing two algorithms that have a similar Big O, although it might seem like an intuitive thing to do, is really comparing apples and oranges. Remember, and this should be your mantra: "we are not comparing runtimes, but asymptotic complexity." Let us say that you have two algorithms with a same Big O, let us say of N, and data for them to utilize for size M. Let us say that algorithm 1 takes X time to accomplish it's task, and algorithm 2 takes Y time to accomplish it's task. All that Big O is telling you is that for M*2 datum, algorithm 1 will take X*2 and algorithm 2 will take Y*2! Their rate of CHANGE is what you are looking for (god, is that the 1st derivative?) not their actual values.

Eek... i think that's right... anyone wanna jump in on this one with me?

Comment on RE: Sorting a list of IP addresses (aka Why I hate Big O)

Replies are listed 'Best First'.
RE: RE: Sorting a list of IP addresses (aka Why I hate Big O) by young perlhopper (Scribe) on Aug 03, 2000 at 02:58 UTC
Yes, i will jump in with you, enthusiastically. Thank you for the cogent explanation. However, I wouldn't necessarily say the big O notation is a measure of the rate of change. (it is, sort of, but i don't think that's the best way to think of it). Another way of thinking of it is this: Take two algorithms, algorithm A, which is O(Nlog N), and algorithm B, which is O(N^2). Now let's say that the execution time for Algorithm A is 100 seconds when N=1000, and that the execution time for Algorithm B is 1 second when N=1000. At first glance, B might appear faster than A, and it certainly is for N=1000. However, what Big O tells us is that no matter what* execution times we measure for A and B at a given value of N, A will be faster than B for some arbitrarily large N. This, is essentially what the theorem behind big O states: That an algorithm with a 'smaller' big O will run faster than an algorithm with a 'larger' big O, for all N > X, X finite but arbitrarily large. B may run 10^100 times faster than A for N = 10^1000, but since A has a 'smaller' big O, there is some N such that A will run faster than B. Okay, i'll stop rambling. -Mark	[reply]

Replies are listed 'Best First'.

RE: RE: Sorting a list of IP addresses (aka Why I hate Big O)
by young perlhopper (Scribe) on Aug 03, 2000 at 02:58 UTC

However, I wouldn't necessarily say the big O notation is a measure of the rate of change. (it is, sort of, but i don't think that's the best way to think of it).

Another way of thinking of it is this: Take two algorithms, algorithm A, which is O(N*log N), and algorithm B, which is O(N^2). Now let's say that the execution time for Algorithm A is 100 seconds when N=1000, and that the execution time for Algorithm B is 1 second when N=1000. At first glance, B might appear faster than A, and it certainly is for N=1000. However, what Big O tells us is that no matter what execution times we measure for A and B at a given value of N, A will be faster than B for some arbitrarily large N.

This, is essentially what the theorem behind big O states: That an algorithm with a 'smaller' big O will run faster than an algorithm with a 'larger' big O, for all N > X, X finite but arbitrarily large.

B may run 10^100 times faster than A for N = 10^1000, but since A has a 'smaller' big O, there is some N such that A will run faster than B.

Okay, i'll stop rambling.
-Mark

[reply]

In Section Meditations