Re: An informal introduction to O(N) notation

One thing that you didn't actualy give is a defintion for "order of growth", which would explain why O(1)==O(2), and why O(2N+3) == O(N).

I'd try to give a defintion, but I'd most likely get it wrong... I almost quoted Knuth here, but then I realized that I didn't understand what he meant. (For those that care, see pg 107 of vol 1, 3rd ed, of TAoCP.)

Update: Everything below this point. (I kept reading that section, and didn't want to "waste" another node.)

In purticular, note that O(n) gives an /upper/ bound on the order of growth of a purticular algo, and is by no means exact. There's also Ω(N), which gives the lower-bound.

Additionaly, saying that one algo is O(1), whereas another is O(2^N) does mean that, for a large enough dataset, the first algo will be faster then the second. It does /not/ mean that it will be for any reasonable dataset. For example, if the first algo takes 1 year to complete, but the second takes 2^N nanoseconds, it will take... well, a huge dataset before the first becomes faster.

Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

Comment on Re: An informal introduction to O(N) notation

Replies are listed 'Best First'.
Re: Re: An informal introduction to O(N) notation by dws (Chancellor) on Jan 18, 2003 at 05:15 UTC
One thing that you didn't actualy give is a defintion for "order of growth", which would explain why O(1)==O(2), and why O(2N+3) == O(N). From a casual perspective, "order of growth" can be thought of as the shape of the worst-case growth curve, regardless of the slope of that shape. Even it a line indicating linear (i.e., O(N)) growth has a high slope, eventually--when the data set is big enough--it will be overtaken by an O(N²) curve. That's one of the pitfalls. People get seduced by the behavior of their code on small data sets, then get blindsided when they try a much larger data set and performance sucks.	[reply]
Re: An informal introduction to O(N) notation by Abigail-II (Bishop) on Jan 18, 2003 at 12:37 UTC
Actually, a year is less than 2^55 nanoseconds. And I wouldn't call a set with 55 elements "huge". Beware of the power of exponentation. Your computer needs to speed up with a factor of 1000 to be able to increase your dataset with no more than 10 so that it will run in the same time.... Abigail	[reply]
Re: Re: An informal introduction to O(N) notation by theorbtwo (Prior) on Jan 18, 2003 at 21:40 UTC
Sigh. That's what I get for not checking my arithmetic. You're right, of course. I stand corrected on this example, but if you change the example to be O(1) at 1 year vs. O(N^2) at 1ns*N^2, the size of the dataset for the second to become slower becomes a lot larger. (Specificly, around 178 million items.) Also, there's the consideration of O() notation being the worst-case senerio. For example, even though bubble-sort is O(N^2), for nearly-sorted input, it can be quit efficent. Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).	[reply]