In my opinion if it looks like a number it should act like a number. There's already a well-established way to increment 1.9 without going all the way to 2.0, just increment to 1.95. If anyone is unclear as to where 1.95 falls on the number line you can direct them back to junior high.

This isn't a completely abstract choice - it's often necessary to do actual math on version numbers. Imagine you've decided you support version 1.3 and higher of a particular browser. In a magical world where version numbers are treated like floats your job is easy - $version >= 1.3. But in our world you have to somehow guess the intent of number - will 1.10 come along someday and expect to be treated higher than 1.9? There's really no way to know! Either way you code it could turn out to be very wrong.

-sam