Niner710 has asked for the wisdom of the Perl Monks concerning the following question:

I have a question on what graphing module would be best suited to use to graph hundreds of thousands(>100,000) of data points. I originally used Excel but found that each graph has a 255 series limitation. I need to make simple line charts to display all the data and would like to display it in a png/jpg format. I am using windows xp as well. Any suggestions? TK/GdGraph?

Replies are listed 'Best First'.
Re: GDGraph for many data points?
by zentara (Cardinal) on Feb 17, 2009 at 12:52 UTC
    I havn't tried that many data points on a Tk or Gtk2 canvas, but I will guess that it will start to bog down, unless you break the datapoints into smaller sets, of say 1000 points, then graph them as separate bezier curve segments. See Tk Realtime data aquisition if you are interested. You also should look at PDL, but I have no idea how well that works on win32, especially getting all the extra graphics output modules compiled right. The PDL maillist is full of guys that plot millions of points for space exploration, so you might want to look at it, and join in.

    I'm not really a human, but I play one on earth My Petition to the Great Cosmic Conciousness
      You also should look at PDL, but I have no idea how well that works on win32, especially getting all the extra graphics output modules compiled right.

      I can provide some details in that regard. The PGPLOT and GD graphics capabilities are certainly available, and come with the PDL ppm package available from the Uwinnipeg repository.

      OpenGL access is also available to PDL via PDL::Graphics::OpenGL::Perl::OpenGL. It builds fine for me on Win32, though it's still in the development stage. I don't personally use any of this stuff, so I don't know what sort of milage is provided. All I know is that 'dmake test' (or 'nmake test', as the case may be) works fine.

      Cheers,
      Rob
Re: GDGraph for many data points?
by hda (Chaplain) on Feb 17, 2009 at 13:01 UTC
    I will also recommend PDL as the tool for you in this case. PDL usually works with the graphical interfaces PLPlot or PGPLOT, but you might might also want to try others, such as Gnuplot. Fortunately, Perl has modules for all that!

    To finish, just a comment: are you sure that you want to make line plots of >100000 points? These might become a bit messy ...

Re: GDGraph for many data points?
by BrowserUk (Patriarch) on Feb 17, 2009 at 14:17 UTC

    This draws a million point line graph. It takes maybe 3 seconds, requires around 600MB and isn't very useful, but it has no problem handling it.

    #! perl -slw use strict; use GD::Graph::lines; our $N ||= 1e6; my( @x, @y ); push( @x, int rand 1024 ), push( @y, int rand 768 ) for 1 .. $N; my @sortOrder = sort{ $x[ $a ] <=> $x[ $b ] } 0 .. $#x; @x = @x[ @sortOrder ]; @y = @y[ @sortOrder ]; my $g = GD::Graph::lines->new( 1024, 768 ) or die $!; $g->set( no_axes => 1 ); my $gd = $g->plot( [ \@x, \@y ] ) or die $g->error; open PNG, '>:raw', 'graph.png' or die $!; print PNG $gd->png; close PNG; system 'graph.png';

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: GDGraph for many data points?
by Anonymous Monk on Feb 17, 2009 at 02:11 UTC
    Graphviz
      Could you please elaborate on how to use Graphviz to generate a lines plot?

      It can generate charts that display graphs and trees, but I don't see how it can generate line plots...