Re: Managing a graph with large number of nodes

I've made a graph using the Graph module that has 0.5 million nodes and 100 thousand links. It takes > 300M of memory, so tieing it to disk or increasing the amount of swap space might work. The other possibility, depending on the nature of the graph, is to try PDL which is implemented in C and use a matrix representation for the weighted graph with other data structures to handle the various attributes.

If you get desperate enough to write it yourself, take a gander at the C++ code under Optimising "generalised modularities". For my graph, it runs very fast.

perl -e 'print qq(Just another Perl Hacker\n)' # where's the irony switch?

Comment on Re: Managing a graph with large number of nodes

Replies are listed 'Best First'.
Re^2: Managing a graph with large number of nodes by etj (Priest) on May 29, 2022 at 22:55 UTC
I'd agree with this; the matrix approach I'd go with is an incidence one, which would be O(E), here 2100k (col1=startpoint,col2=endpoint). The adjacency approach would be 500k500k = 2.5e11 = 250 billion entities = minimum 30GB even if you used bitfields. Naturally, I think PDL would be a good platform for this.	[reply]