It all depends on what you're using "time" for and what you're using as your ground truth.

As far as I understand, Google uses synchronized time for lots stuff like of query ordering and vector clocks etc and it is important for them to have one ground truth up to the point that they install their own GPS clocks in datacenters. As these are mostly for the use by Google, it's up to them to decide how they will handle the additional second, and I can understand from a risk assessment point of view that it's likely less risk to have slightly longer seconds instead of having one additional second with the number 60 and auditing your code for the parts where that becomes relevant.

The situation becomes interesting when the private use/decision of Google leaks out into the real world for (say) Google Cloud Engine users or whoever else relies on Google infrastructure and timekeeping.

Personally I can't imagine situations where the exact and synchronized duration of a second is important to you but you don't have your own synchronized clock(s), but you'll have to be prepared for an apparent one second gap when comparing the timestamps of Google infrastructure with the timestamps of your own infrastructure, and over time, the two kinds of timestamps will diverge until at 2017-01-01 00:00:00Z where they will suddenly converge again.

If this is your first time dealing with diverging clocks, it will be an interesting learning experience, especially if you did to GPS-based time exactly to avoid this situation.

In the aftermath of this leap second, Cloudflare experienced an outage and blogged about it. It seems the root cause was code that expected a monotonically ascending value for seconds, but the additional second was handled (by the Go library used) by letting time go backwards one second, which led to negative durations for some events, which finally were not handled gracefully.

I think this would not have been a problem for Cloudflare if they too had stretched the duration of a second, at least for their machines running RRDNS. Of course, this is literally Monday quarterbacking as I wasn't part of the decision process there. Also, knowing and understanding how time and durations are used within your code is not an easy thing if you don't explicitly analyze your code for the usage of both.

Very interesting, and quite surprising that they experienced such a problem. Speaks to the immaturity of Go I suppose.

We are sorry that our customers were affected by this bug and are insp
+ecting all our
code to ensure that there are no other leap second sensitive uses of t
+ime intervals.
[download]

... might have been nice to do that beforehand. Doesn't seem like it would have been too hard to spot:

- if rttMax == 0 {
+ if rttMax <= 0 {
    rttMax = DefaultTimeout
}
[download]

... if only they were coding in Perl and could use $rtt_max ||= $default_timeout;, LOL


The way forward always starts with a minimal test.

I don't know if it's relevant, but it occurred to me that RTC crystals used in embedded systems are tuned by manufacturers to operate at the very specific frequency of 32.768kHz. If I recall the original reasoning was that the MSB of a 16-bit counter value toggles every 1 sec for that frequency, and now I'm guessing it's just kind of legacy that it continues to be done that way instead of going to some faster multiple of that value (which would mean physically smaller crystals, less material, board space, etc.). Kind of a moot point since modern technology tends to move away from traditional RTC crystals anyway, but I digress.

I mention the above because the period for the 32.768kHz frequency is about 30.5 microseconds, so the choice of 20 hours for the smear could have been as arbitrary as the number of hours necessary to reach less than half that value? Technically 19 hours would have been enough, but who likes odd numbers? Besides, us silly engineers have this always add in a margin of error habit even in the digital/discrete realms where we know darn well it doesn't matter. Granted, I'm completely speculating on the reason for the 20 hour smear, I could be totally wrong, I have no inside knowledge or anything like that.