flock seems to return true/false rather than a system return code (which is what I'm assuming you're expecting - and also what I'd expect).

I made a test where I flocked, printed the rc, slept for 1000. Ran this in the background and then ran again. First process got 1, second got 0. This was on AIX, BTW. I'm assuming you're just concerned about why it's failing?