As with all the legends listed above, it depends on which storyline you choose. :-)

In one popular story, Musashi was defeated exactly once: by an unknown farmer who fought with a wooden staff. Musashi learned never to underestimate an opponent, and extended the lesson by deliberately fighting other swordmasters with weapons that they were unprepared to fight against; staves, wooden swords, oars.... one famous story has him knock a challenger unconcious with a wood sculpture after the foolish contender wouldn't leave Musashi alone to finish his woodcarving, and kept pestering him for a duel...