Maybe what you want is positive look-behind instead of negative?

$ perl -lane 'print if /(?<=10\.20\.11\.99\s)kvm-test-v06.*/' your-hos
+ts-file
10.20.11.99 kvm-test-v06.example.com    kvm-test-v06
[download]