Is the "java" command actually in /users/sophix/Desktop/prottest3/prottest3/ ? Sounds like it's not. If you are running it from that directory so you can use relative path names, you should chdir to that directory first and then do not include it as part of the command path.
The ">" shows that you are calling java at the command line with no path specified, yet in your system call you give an absolute path, incorrectly as saberworks pointed out. You seem to have that part resolved.
In the system call there are missing spaces between filenames and command line switches. For example between "\.aa" . "-t ".
I suggest two things to straighten this out. First, don't use concatenation here until you get it working. Then only use concatenation to split the long line to make it more readable. Embed your variables into a long string.
Second, save the string into a variable, print it so you can see what is going to be called. Then after it looks like expected, pass this varible to the system command.
One more thing I noticed, you don't need to backslash escape periods in a string. You aren't using these in regexes.