Shouldn't the ^S be enough to say that this line does not match?

Yes, I believe it should - and I'm afraid that part of it also has me stumped. Perhaps some other monk can explain why that is so.

Update: ahh, of course - as others have pointed out below - it's because you haven't used parentheses to define the boundaries of your alternation :)

Anyhow, getting back to your requirements, here is how I would do it:

Update: oops, I just realised that I posted the wrong pattern. It can be simplified somewhat by grouping the part that matches the floats and using the {2} quantifier. I've updated it (output remains the same)

use strict;
use warnings;

while (<DATA>) {
    if (/^S\s+(?:\-?\d+(?:\.\d+)?\s+){2}\(\s?(?:(INOUT|OUTPUT))\s?\)/)
+ {
        print "Matched:$_";
    }
    else {
        print "Did NOT match:$_";
    }
}


__DATA__
  onn (bbcreccsnnl_output) !OUTPUT
S 0.0 0.0 (OUTPUT)
A 0.0 0.0 (OUTPUT)
S 1 4 5 (OUTPUT)
S 35 -27 ( INOUT )
S -26.95 32.73 (OUTPUT )
[download]

The above outputs:

Did NOT match:  onn (bbcreccsnnl_output) !OUTPUT
Matched:S 0.0 0.0 (OUTPUT)
Did NOT match:A 0.0 0.0 (OUTPUT)
Did NOT match:S 1 4 5 (OUTPUT)
Matched:S 35 -27 ( INOUT )
Matched:S -26.95 32.73 (OUTPUT )
[download]

Which I believe meets your requirements, yes?