visualizing networks
TRANSCRIPT
Visualizing Networks
Lynn ChernyVisiting Knight Chair, UMiami, School of Comm
@arnicas / [email protected]
23 Oct 2015
The Hairball: A Metaphor for Complexity
h6p://www.nd.edu/~networks/PublicaBon%20Categories/01%20Review%20ArBcles/ScaleFree_ScienBfic%20Ameri%20288,%2060-‐69%20(2003).pdf
h6p://www.linkedin.com/today/post/arBcle/20121016185655-‐10842349-‐the-‐hidden-‐power-‐of-‐networks
Release by Joint Economic Committee minority Republicans led by Boehner (2010), Iliinsky & Steele fig 4-14
Neither Exploratory Nor Explanatory:Political.
4
Redesign by Robert Palmer(Iliisnky & Steele fig 4-15)
“By releasing your chart, instead of meaningfully educating the public, you willfully obfuscated an already complicated proposal. There is no simple proposal to
solve this problem. You instead chose to shout ‘12! 16! 37! 9! 24!’ while we were trying to count
something.”
5
Why Care About Networks?
• Epidemiology • Trees/networks in biology • Clustering / recommendaBon systems • “Social network” so^ware • Community design: online and off • OrganizaBonal Design • Computer networks, phone networks, banks… • Another vis method -‐ for relaBonal data.
6
WHAT IS A NETWORK?
It’s not a visualizaBon. Think of it as a data structure.
Data relaBonship: enBBes + relaBonships to other objects (node/edge, vertex/link)
Nodes and Edges may have a6ributes, eg. gender, age, weight, tv prefs connecBon date, frequency of contact, type of exchange, direcBonality of relaBonship a6ributes may be calculated from network relaBons too
A quick look at some data…
h6ps://marketplace.gephi.org/plugin/excel-‐csv-‐converter-‐to-‐network/
Lane Harrison: h6p://blog.visual.ly/network-‐visualizaBons/
11h6p://blog.visual.ly/network-‐visualizaBons/
Best!
A User Study on Visualizing Directed Edges in Graphs” Danny Holten and Jarke J. van Wijk, 27th SIGCHI Conference on Human Factors in Computing Systems (Proceedings of CHI 2009),
12
It’s a natural human trait to see visual similarity and proximity as meaningful.
Be very careful about your display choices and layout methods!
Reading a network visualizaBon
Look at this outlier case!
There’s obviously something important going on here, structurally....
A ménage à
trois ove
r here
MIS?
Using a “random” Gephi layout on the dolphins
Random!
SIMPLE CALCULATIONS ON NETWORKS CAN TELL YOU LOADSO^en you need to visualize the structure/role of the graph elements as part of the visualizaBon: So, do some simple math.
Degree (In, Out)
“Degree” is a measure of the edges in (directed), out (directed), or total (in directed or undirected graphs) to a node
“Hub” nodes have high in-‐degree. In scale-‐free networks, we see preferenBal a6achment to the popular kids.
h6p://mlg.ucd.ie/files/summer/tutorial.pdf
The Threat of Hub-‐Loss
Albert-‐László Barabási and Eric Bonabeau, Scale-‐Free Networks, 2003.h6p://www.scienBficamerican.com/arBcle.cfm?id=scale-‐free-‐networks
VisualizaBon Aside: If Some Names are Huge, the Others are Invisible-‐?
17
Gephi Panel
CorrecBng for text size by degree display issue
BetweennessA measure of connectedness between
(sub)components of the graph
“Betweenness centrality thus tends to pick out boundary individuals who play the role of brokers between communiBes.”
h6p://en.wikipedia.org/wiki/Centrality#Betweenness_centralityLusseau and Newman. h6p://www.ncbi.nlm.nih.gov/pmc/arBcles/PMC1810112/pdf/15801609.pdf
19
? This one?
Sized by Betweenness
Judging By Eye Will Probably Be Wrong...
Even be6er, download the stats and look at them yourself.
Eigenvector Centrality
IntuiBon: A node is important if it is connected to other important nodes A node with a small number of influenBal contacts may outrank one with a larger number of mediocre contacts
h6p://mlg.ucd.ie/files/summer/tutorial.pdf h6p://demonstraBons.wolfram.com/NetworkCentralityUsingEigenvectors/
Pagerank
Wikipedia image
Community DetecBon Algorithms
E.g., the Louvain method, in Gephi as “Modularity.” Many layout algorithms help you intuit these structures, but don’t rely on percepBon of layout!
h6p://en.wikipedia.org/wiki/File:Network_Community_Structure.png
You can even do it in your browser now…
23http://bl.ocks.org/john-guerra/ecdde32ab4ad91a1a240
CitaBon: Lusseau D (2007) Why Are Male Social RelaBonships Complex in the Doubxul Sound Bo6lenose Dolphin
PopulaBon?. PLoS ONE 2(4): e348. doi:10.1371/journal.pone.0000348
IdenBfying th
e role th
at animals p
lay in th
eir social networks (2
004)
D Lusseau, M
EJ New
man
Proceedings o
f the
Royal Society of Lon
don. Series B
: Biological Scien
ces
Eduarda Mendes Rodrigues, Natasa Milic-‐Frayling, Marc Smith, Ben Shneiderman, Derek Hansen, Group-‐in-‐a-‐box Layout for MulB-‐faceted Analysis of CommuniBes. IEEE Third InternaBonal Conference on Social CompuBng, October 9-‐11, 2011. Boston, MA
NodeXL excel plugin
h6p://mbostock.github.com/d3/talk/20111116/force-‐collapsible.html Also see Ger Hobbelt D3: h6p://bl.ocks.org/3616279
USING THOSE STATSExample
28
29http://blogger.ghostweather.com/2011/09/combing-through-infovis-twitter-network.html
NetworkX is a python lib, Gephi is so^ware we’ll see
in a minute.
30http://blogger.ghostweather.com/2011/09/combing-through-infovis-twitter-network.html
31
MATRIX LAYOUTS / REPRESENTATIONSLet’s do other representaBons now…
h6p://barabasilab.neu.edu/networksciencebook/download/network_science_October_2012.pdf
Real social networks are generally quite sparse.
h6p://www.cise.ufl.edu/research/sparse/matrices/Newman/dolphins.html
35
Tamara Munzner’s book Visualization Analysis and Design (2014)
D3 demo by me h6p://www.ghostweather.com/essays/talks/networkx/adjacency.html
37
Readability: Matrix Be6er (except for path finding)
Jean-Daniel Fekete http://lamut.informatik.uni-wuerzburg.de/gd2014/data/uploads/gd2014-fekete-keynote.pdf
ARC / LINEAR LAYOUTS
Philipp Steinweber and Andreas Koller Similar Diversity, 2007
For a D3 example in another domain: h6p://tradearc.laserdeathstehr.com/
Topics in Da Vinci Code (by me)
40
https://www.dropbox.com/s/090vwj12kh3mu96/Screenshot%202014-11-04%2020.45.02.png?dl=0
Lynn Cherny - http://www.ghostweather.com/essays/talks/openvisconf/topic_arc_diagram/TopicArc.html
CIRCULAR / CHORD LAYOUTS
h6p://circos.ca/images/
Circos“If it's round, Circos can
probably do it”
TIL (shocking): It’s all perl code?!
A Nice UI Improvement -‐ DifferenBate Source/DesBnaBon
44Nikola Sander, http://www.global-migration.info/
Hierarchical Edge Bundling demo by @mbostock
D3: h6p://bl.ocks.org/1044242
"Hierarchical Edge Bundles: VisualizaBon of Adjacency RelaBons in Hierarchical Data”, Danny Holten, IEEE TransacBons on VisualizaBon and Computer Graphics (TVCG; Proceedings of Vis/InfoVis 2006), Vol. 12, No. 5, Pages 741 -‐ 748, 2006.
ALGORITHMIC LAYOUTS
Gephi / D3.js / Other tools
48https://dhs.stanford.edu/visualization/more-networks/
Sample Layout Plugins in Gephi
h6ps://gephi.org/tutorials/gephi-‐tutorial-‐layouts.pdf
Gephi Plugin Layout DetailsLayout Complexity Graph Size Author Comment
Circular O(N) 1 to 1M nodes Ma6 Groeninger Used to show distribuBon, ordered layout
Radial Axis O(N) 1 to 1M nodes Ma6 Groeninger Show ordered groups (homophily)
Force Atlas O(N²) 1 to 10K nodes Mathieu Jacomy Slow, but uses edge weight and few biases
Force Atlas 2 O(N*log(N)) 1 to 1M nodes Mathieu Jacomy
(does use weight)
OpenOrd O(N*log(N)) 100 to 1M nodes S. MarBn, W. M. Brown, R. Klavans, and K. Boyack
Focus on clustering (uses edge weight)
Yifan Hu O(N*log(N)) 100 to 100K nodes Yifan Hu (no edge weight)
Fruchterman-‐Rheingold O(N²) 1 to 1K nodes Fruchterman & Rheingold!
ParBcle system, slow (no edge weight)
GeoLayout O(N) 1 to 1M nodes Alexis Jacomy Uses Lat/Long for layout
h6ps://gephi.org/2011/new-‐tutorial-‐layouts-‐in-‐gephi/
Dolphins Social Network
51
OpenOrd + “No Overlap” ForceAtlas2
52
Weight 2: Force Atlas Weight 4: Force Atlas Weight 4: Yifan Hu
Unweighted dolphins, Force Atlas
Effect of An Artificial Weight on the Layout
Simple Orderings of Nodes in Circular Layout
53Dolphins colored by modularity class (community) in Gephi
“Dual Circle” layout with most popular dolphin in center
Circular Layout ordered by Degree
Sorted by Modularity
Gephi Workflowsà Sigma.js / D3…
Gephi.org: Open source, runs on Mac, Linux, PC Output high quality images that are staBc (from Preview) or…
Sigma.js : Will display a gexf gephi layout file with minimal work, using a plugin interpreter for sigma site export Also offers a force-‐directed layout plugin for graphs without x&y coords Does CANVAS drawing, not SVG
D3: Export JSON from Gephi and load into D3 network layout — can calculate/layout your x/y coords (and other stats) in Gephi and then use them in a staBc layout in D3! Also see cola.js.
h6p://www.barabasilab.com/pubs/CCNR-‐ALB_PublicaBons/200705-‐14_PNAS-‐HumanDisease/Suppl/Goh_etal_poster.pdf
Sigma.js version of the Gephi export h6p://exploringdata.github.com/vis/human-‐disease-‐network/
Movie:
57By me: h6p://www.ghostweather.com/fun/sigma_topic_network/#
58
FINAL DESIGN THOUGHTS… BY HAND? BY ALGORITHM-‐?
Conspiracy Theorist Mark Lombardi
Learning from Lombardi: h6p://benfry.com/exd09/
Jeff Heer: h6p://hci.stanford.edu/courses/cs448b/f11/lectures/CS448B-‐20111110-‐GraphsAndTrees.pdf
Ger Hobbelt in D3: h6p://bl.ocks.org/3637711
Hybrid Method: Use algorithmic layout, and then adjust nodes by hand. (Can be done in Gephi or D3.)
Also read http://bost.ocks.org/mike/example/#2
62http://www.nytimes.com/interactive/2013/02/20/movies/among-the-oscar-contenders-a-host-of-connections.html
http://bost.ocks.org/mike/example/#2
Minimize Info: Less is More
63Color by important data value, sized by degree, no edges
64h6ps://dhs.stanford.edu/visualizaBon/more-‐networks/
Eigenvector Centrality
Betw
eenn
ess
Cen
tral
ity
Check the Stats.
A Nice D3 tutorial with some good UI
65
Jim Vallandingham h6p://flowingdata.com/2012/08/02/how-‐to-‐make-‐an-‐interacBve-‐network-‐visualizaBon/ Also see his OpenVis talk: h6p://vallandingham.me/abusing_the_force.html
Wrap it up on design...
Design Reminders (1/2)
Do data analysis / reducBon -‐ why would you want to show 1T of network data? Calculate and reveal important facts about node relaBonships. Make it interacBve -‐ details on demand.
Help people find things in your network! (Search?) Show more on hover/click Too m
uch data
=Not always good
data science!
Design Reminders (2/2)
Different layouts communicate different things to your viewer -‐ choose wisely.
Take care: people will infer things from proximity/similarity even if it was not intended!
Consider if it has to be a node-‐edge display at all: Is it the stats you care about? The important nodes who branch sub-‐communiBes? The ones with the most in/out links? The “top influencers”?
69
Thanks! @arnicas
Slides will be on Slideshare:http://www.slideshare.net/arnicas/presentations
A Few More ReferencesJeff Heer class slides: h6p://hci.stanford.edu/courses/cs448b/w09/lectures/20090204-‐GraphsAndTrees.pdf A great in-‐progress book on networks: h6p://barabasilab.neu.edu/networksciencebook/ Mark Newman’s new book-‐ NetWorks: An IntroducBon Tutorial on Gephi: h6ps://t.co/ZsMekjkfMt
Journal of Graph Algorithms and ApplicaBons: h6p://jgaa.info/home.html
Jim Vallandingham’s D3 network tutorials: h6p://flowingdata.com/2012/08/02/how-‐to-‐make-‐an-‐interacBve-‐network-‐visualizaBon/, h6p://vallandingham.me/bubble_charts_in_d3.html
Brian Keegan’s post about GamerGate tweets and the dangers of network vis ww.brianckeegan.com/2014/10/my-‐15-‐
Robert Kosara’s post: h6p://eagereyes.org/techniques/graphs-‐hairball
Lane Harrison’s post: h6p://blog.visual.ly/network-‐visualizaBons/ MS Lima’s book Visual Complexity and site
A couple arBcles on community structure:
Overlapping Community DetecBon in Networks: State of the Art and ComparaBve Study by Jierui Xie, Stephen Kelley, Boleslaw K. Szymanski
Empirical Comparison of Algorithms for Network Community DetecBon by Leskovec, Lang, Mahoney
My posts on NetworkX with D3 and Twi6er network vis with Gephi