Certainly the human eye can process a 1 megapixel photo, so it's the way you represent it. First why are you doing it? (besides that it's fun).

You might be interested in googling for: visualizing large graphs

There are two free tools, H3Viewer which is in C++ and Walrus which is in Java3D. These both use fisheye style views.

Here's a few links to get you started. GINY (scroll toward bottom), Munzer paper, LGL (gallery has a 10^6 edge image), Large RDF graphs.

So these different approaches tend to use interactive zoomable charts, and show details that can be made sense of when seen even from far away.

FWIW I remember once I was involved with a company doing Y2K remediation and I saw the visualizations IBM had cooked up. They may not have been as complex as yours but resembled your data, it was to describe safety of code in different programs. Each program was represented by a circle of dots and each circle was drawn like a pie chart in a way, with colorings of segments indicating their safety. It was just a bunch of these pie charts in a big table but from far away you could tell which programs were most complex or dangerous (more red). So maybe you need to think about what is most important to represent.

That, and also to realize that 100M edges is extremely complex. Probably if you really want to grasp what is going on you should try some different ways, some which shrink groups down to a small number of pixels and other ways that allow you to zoom in more and forget people outside the group.

For example look at the picture on the large-scale rdf graph visualization page. The Software Options section half-way down explains that GraphViz is aimed at making nice pictures of reasonably sized graphs. Since yours isn't reasonable you might want to consider using another tool or at least not drawing straight graphs but doing something else. It says Walrus is good for around 100K nodes and Tulip for about 1M nodes/edges (they have a social network graph sample).

Sorry I couldn't be of more help.

In reply to Re: Graphing HUGE Social Networks by mattr
in thread Graphing HUGE Social Networks by spx2

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.