Unskilled and Unaware of It has an intersting take on that. Specifically, it defines metacognition (coupled with metamemory, metacomprehension, and self-monitoring) as a set of skills allowing one to assess one's performance and accuracy. Such as, a novice programmer is ill equipped to gauge himself. Only further experience (or criticism from the journeymen and/or master programmers... or even from PM) will raise his self awareness concerning his skills and abilities. The article doesn't state definitively, however, that inflated self assessment comes from incompetence.

... people who actually are good ... generally are painfully aware of their own shortcomings

Experience (and exposure to outside opinion as touched on later in your writeup) raises the ability for self assessment. It's often said and heard that "the more one learns, the more one learns how much there is yet to be learned".

This confuses us, because we think, "A causes B, so more A will result in more B." But more B causes a reaction that gets rid of B and our change shows up in some entirely unanticipated way.

Like when I optimize my path to work, shaving off 5-7 minutes from my commute. I should be that much earlier to arrive, giving myself more time to check the news, email, and browse PM. However, that time is actually realized as I hit the snooze one more time every morning. =]