I only have a few votes each day, and I usually don't spend any of them. Getting double the number of votes the next day wouldn't be very useful to me.

I personally believe that it wouldn't be annoying either: you would have the double number of votes and you still wouldn't spend any of them.