13

I'm looking to visualize the data, hopefully make it interactive. Right now I'm using NetworkX and Matplotlib, which maxes out my 8gb when I attempt to 'draw' the graph. I don't know what options and techniques exist for handling such a large cluster** of data. If someone could point me in the right direction, that'd be great. I also have a CUDA enabled GFX card if that could be of use.

Right now I'm thinking of drawing only the most connected nodes, say top 5% of vertices with the most edges, then filling in less connected nodes as the user zooms or clicks.

6 Answers 6

4

I don't have any experience with it, but tulip seems to be made for that.

4
  • The link isn't working for me.
    – Jay Askren
    Apr 13, 2011 at 13:28
  • Now it works. Maybe the server was down temporarily.
    – Jay Askren
    Apr 13, 2011 at 14:30
  • 2
    My friend told me Tulip, while perfect for this sort of job, has a steep learning curve. He suggested Cytoscape, but now I'm looking at web app friendly solutions.
    – wnewport
    Apr 13, 2011 at 23:56
  • I don't think any browser is fast enough to handle that kind of load, but you can try :) Apr 14, 2011 at 9:31
3

Maybe PyOpenGL? It can be used together with wxPython.

Edit: Just tried the performance without any optimization, it takes 0.2s to draw 100k vertices and 4s to draw 1M edges.

2

You should ask on the official wxPython mailing list. There are people there that can probably help you. I am surprised that matplotlib isn't able to do this though. It may just require you to restructure your code in some way. Right now, the main ways to draw in wxPython are via the various DCs, one of the FloatCanvas widgets or for graphing, wx.Plot or matplotlib.

1

Have you considered graphviz? Not interactive although it was designed from the outset to handle very large graphs (although 1M edges may be beyond even it's capabilities).

There's a python module (pydot) that makes interacting with graphviz simple. Again, can't say for sure it'll scale to your levels. However, it should be easy to find out: installation of both is simple.

hth.

2
  • Dot is nice, but I'm pretty sure this is beyond its capabilities. Fortunately, there are better open source solutions now, such as tulip (there are others as well). Apr 14, 2011 at 9:30
  • yeah, tulip was new to me - looks interesting. Good luck finding something.
    – sfinnie
    Apr 14, 2011 at 10:58
1

Have you considered using ParaView or VisIt? These are two interactive plotting programs which are designed to deal with and plot (very!) large data sets. They both also have a Python scripting interface, so you can automate/control your visualizations from within the Python interpreter.

0

Have you tried Gephi ?

I believe it scales very well.

Not the answer you're looking for? Browse other questions tagged or ask your own question.