exec() replaces the current program by another one. Unless the exec fails to start the mentioned program, exec will not return..

system() calls another program, and then returns a status code (which contains the exit value of the program, and possibly information about signals). system() can be called in such a way that no shell will be used.

qx calls another program, and returns the output of that program. It's not under the programmers control whether a shell can be bypassed or not.

Both system and qx are implemented using fork and exec (at least, under Unix).

Abigail