For some simple plots, I have used the modules GD and GD::Graph, but for more complex things, I use R extensively (sometimes directly, sometimes with a perl wrapper, sometimes with perl to set up the input files and then R to do the work). I find it to be a great package, with a great community. There are multitudinous user-submitted packages, and a strong culture of users giving back. I don't think CRAN rivals CPAN in terms of size (probably nothing does), but it takes after it in concept. I have also used the Statistics::R module a bit, which helps link perl to R.

I usually use the UNIX version writing scripts or performing ad-hoc analysis/testing things out via the well-integrated emacs mode, but my colleagues tend to use the windows GUI front end for their ad-hoc use.

I haven't used Octave, so can't comment on it or give you a feature comparison with R. Octave and R are free replicas of Matlab and S, so presumably appeal to slightly different constituencies.

If you have specific analytic needs, before you take the plunge with one platform or another, it might be worth checking out CRAN (one of the many mirrors is here) and the Octave user-submitted packages (OctaveForge), and see if there are already implementations of the sorts of analyses you need -- that alone could save you lots of time!

Another R resource is this page of interesting graphics examples.

Good luck with your quest!