No, the Swiss system will actually optimize that people of equal strength play against each other as soon as possible - where equal strength is determined mostly by the performance in the tournament. That is, if I win the first three rounds, I will play against someone else who has won in the first three rounds (unless I'm the odd one out, in which I most likely play against someone with 2.5/3).