This kind of thing cannot be specified by regular expressions except in the trivial case where you only have one pair of brackets. In order to properly specify and parse complex expressions you could use Damian Conway's excellent Parse::RecDescent.

Using this you can specify your input as a context free grammar. Your grammar would probably be something like

start: expression

expression: expression m/[+-/*]/ expression
            '(' expression ')'
            function '(' expression ')'
            identifier
            number
[download]

There is an excellent FAQ on Parse::RecDescent available on the web.

1. dude, what does mine say?
2. "sweet", what about mine?
3. "dude", what does mine say?
4. GOTO 2