There is a field of mathematics called "Combinatorics". You may not know what that is, but it isn't as scary as Calculus because it is not nearly as complicated. This field is about studying the ways of combining a finite number of things, and typically what you study is the number of ways of counting things. So from the point of view of combinatorics, what are the operations +, *, **, and factorial?

Well m + n is the operation of taking two groups of m and n things, putting them together, and then counting the combined group. So 0+0 is putting nothing with nothing and counting nothing, so you get 0 again.

As we all know, m * n is the operation of taking m things along one axis, n things along another axis, creating a rectangle, and then counting up unit squares in the rectangle. So 0*0 is making a rectangle with no height and no width then realizing that its area is 0, so you get 0 back.

Analogies with these two familiar operations are probably why many people have incorrect expectations of what happens with powers and factorial.

When we write mⁿ what we are counting are the number of ways there are to pick out n things from a set of m in order with repetition allowed. Since each choice you make gives you m times as many things, we are used to calculating it with m*m*...*m where the number of m's is n. But that calculation leaves many of us confused over what 0⁰ is supposed to be. Well forget the formula, go back to basics. If I want to pick nothing from no things, I can do nothing! There is no other thing I can do, but I do have one way to accomplish the task. And that is to look back, grin, and say I have no work to do, the task is accomplished!

So 0⁰ is 1 since there is one way to pick nothing from no things.

Similarly with factorial. n! is the number of ways to arrange n things in order. Again we have a well-known formula for it. But again the well-known formula does not give most people any insight as to what 0! should be. But right now take a look at a collection of no things. Well they are in an arrangement already, can we rearrange them? Not that I can see! So there is exactly one way to arrange 0 things in order, and that is to leave that space empty! Therefore 0! is 1.

Still not convinced? Well there is a more fundamental mathematical principle at work here. In many places in mathematics you have formulas where you have to add two sets of things together. So you can add them up separately and then add together the sums. In others you want to multiply two sets of things together. So you multiply them separately and then multiply together the products. ariels offers examples of both involving the binomial theorem and Taylor series. But the concept is clear.

But the special case that arises is what you do when one of the sets has no things in it? How do you add together no things? How do you multiply no things? What answer makes sense?

Well the overriding principle is that how you divide a set up first should make no difference to the final sum or product. So if I take a set of numbers and break it into that set and an empty set, add them separately, then add them together, I have to come to the actual sum. Which is only going to work if adding the sum of no things doesn't change anything. Ditto for multiplication. Multiplying a set of numbers together should be the same as breaking it into the entire set and an empty set, multiplying them separately, then multiplying them together.

As we all know, the only thing you can add and not change anything is 0, for which reason 0 is called the additive identity. Therefore the empty sum is always 0 in mathematics. Likewise 1 serves the same role in multiplication that 0 serves in addition. Multiplying by 1 changes nothing. So the only reasonable answer to get from multiplying nothing is 1. And since powers and factorial both work out to be formulas that involve a series of multiplications, not additions, the answer in the degenerate case (ie 0! and n⁰) is an empty product which is 1.

As for division by 0, there is a simple reason for making that a trappable error. The vast majority of the time when a programmer winds up dividing by 0, that wasn't what was intended and is a sign of a fairly serious bug. So you don't want to make it undef and sweep it under the rug. However there are times when the programmer needs to check the error so you make it trappable.

You might not believe me, but if you do a lot of programming with numbers, just pay attention. Most of the time when you get that error, was it a sign of something that was really wrong? (It is for me.)

UPDATE
jynx pointed out an obvious typo.