Spectral Analysis and Spectral Analysis and Optimal Synchronizability Optimal Synchronizability

of Complex Networksof Complex Networks

复杂网络谱分析及最优同步性能研究复杂网络谱分析及最优同步性能研究

Spectral Analysis and Spectral Analysis and Optimal Synchronizability Optimal Synchronizability

of Complex Networksof Complex Networks

复杂网络谱分析及最优同步性能研究复杂网络谱分析及最优同步性能研究

Guanrong (Ron) Chen

City Univesity of Hong Kong

SynchronizationSynchronization

Synchrony can be essential Synchrony can be essential

IEEE Signal Processing Magazine (2012)

“Clock synchronization is a critical component in the operation of wireless sensor networks, as it provides a common time frame to different nodes.”

2009

Spectra of network Laplacian matrices

Spectra and synchronizability of some typical complex networks

Networks with best synchronizability

ContentsContentsContentsContents

A linearly and diffusively coupled network:

f (.) – Lipschiz Coupling strength c > 0

If there is a connection between node i and node j (j ≠ i), then aij = aji = 1 ; otherwise, aij = aji = 0 and aii = 0, i = 1, … , N

A = [aij] – Adjacency matrix H – Coupling matrix function

N

jjijii xHacxfx

1

)()(

Laplacian matrix L = D – A where D = diag{ d1 , … , dN }

ni Rx Ni ,...,2,1

N 210For connected networks:

A General Dynamical Network ModelA General Dynamical Network Model

Linearized at equilibrium s :

N

jjijii xHacxfx

1

)()( Ni ,...,2,1ni Rx

Msf ||)(||

Only

is important

}{or ][ iij cac

f (.) Lipschitz, or assume:

i

N

jsxijsxii xDHacxDfx

ii

1

][)]([

Network SynchronizationNetwork SynchronizationNetwork SynchronizationNetwork Synchronization

Complete state

synchronization: Njitxtx jit

,,2,1,,0||)()(||lim 2

Synchronizing: if or if

N 210

320 10

Master stability equation: (L.M. Pecora and T. Carroll, 1998)

2 :,]][][[ cyDHDfy issi

maxL

Maximum Lyapunov exponent which is a function of

is called a synchronization regionmaxS

i

N

jsijsii xDHacxDfx

1

][)]([

Network Synchronization:Network Synchronization: CriteriaNetwork Synchronization:Network Synchronization: Criteria

A (conservative) sufficient condition:

0max L

Recall: Eigenvalues of

synchronizing if or if

Case III:Sync region

Case II:Sync region

Case I: No sync

N 210

),( 322 S),( 11 S

ADL

32

20

N

bigger is better bigger is better2N

2

Case IV: Union of intervals

210 c

Network Synchronization:Network Synchronization: CriteriaNetwork Synchronization:Network Synchronization: Criteria

Synchronizability characterized by Laplacian eigenvalues:

1. unbounded region (X.F. Wang and G.R. Chen, 2002)

2. bounded region (M. Banahona and L.M. Pecora, 2002)

3. union of several disconnected regions

(A. Stefanski, P. Perlikowski, and T. Kapitaniak, 2007) (Z.S. Duan, C. Liu, G.R. Chen, and L. Huang, 2007 - 2009)

),(0, 11212 SN

),(0,/ 322212 SNN

),(),(),( 4321 mmS

A Brief HistoryA Brief History

Some theoretical results

Relation with network topology

Role in network synchronizability

Spectra of NetworksSpectra of NetworksSpectra of NetworksSpectra of Networks

Spectrum

- maximum, minimum, average degreeavgminmax , , ddd

maxmaxmin2 211

ddN

Nd

N

NN

Graph Theory Textbooks

F.M. Atay, T. Biyikoglu, J. Jost, Network synchronization: Spectral versus statistical properties, Physica D, 224 (2006) 35–41.

Theoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian Eigenvalues

NND N ...1

2

Nd

d

2

max

min

D – Diameter of the graph

)(avg Ad N

Theoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian EigenvaluesTheoretical Bounds of Laplacian Eigenvalues

TN ,...,, 21 (both in increasing order)

C. J. Zhan, G. Chen and L. F. Yeung, Physica A (2010)

Njj ,...,2,1|* idFor any node degree there exists a

such thatiiii dddd *

12

1

2

2

||||||||

||||

||||

||||

d

N

d

d

d

d

TNdddd ),...,,( 21

Distribution: (interlacing property)

Lemma (Hoffman-Wielanelt, 1953)

For matrices B – C = A: 2

1

2 |||| |)()(| F

n

i ii ACB

Frobenius Norm

For Laplacian L = D – A: 2

1

2 |||| |)(| F

n

i ii AdL

n

i i

n

i

n

j ijF daA11 1

22 ||||||

Cauchy inequality

12

2

12

2

2

2

||||/||||

||||

||||

||||

d

N

d

N

Nd

d

d

d

d

d

d

d

ii

i

i

i

TN ,...,, 21

12

1

2

2

||||||||

||||

||||

||||

d

N

d

d

d

d

T

Ndddd ),...,,( 21

ER random-graph networksN=2500, average of 2000 runs

BA scale-free networks

N=2500, average of 2000 runs

Difference between Eigenvalue and Degree SequencesDifference between Eigenvalue and Degree Sequences

C. J. Zhan, G. Chen and L. F. Yeung, Physica A (2010)

4210001000 d

Above: relative variancesbelow: relative errors

Upper Bounds for Largest EigenvaluesUpper Bounds for Largest Eigenvalues

}),(edges|max{ vudd vuN

}max{ uuN dd

ud

}nodes|||max{ GinvuNNdd vuvuN

vu NN

NN

- average degree of all neighbors of node u

- total number of common neighbors of u and v

W.N. Anderson, T.D. Morley, Eigenvalues of the Laplucian of a graph, Linear and Multilinear Algebra, 18 (1985) 141-145.R. Merris, A note on Laplacian graph eigenvalues, Linear Algebra and its Applications, 285 (1998) 33-35.O. Rojo, R. Sojo, H. Rojo, An always nontrivial upper bound for Laplacian graph eigenvalues, Linear Algebra and its Applications, 312 (2000) 155-159.

- for any graph

Lower Bounds for Smallest EigenvaluesLower Bounds for Smallest EigenvaluesLower Bounds for Smallest EigenvaluesLower Bounds for Smallest Eigenvalues

)2/(sin4))/cos(1(2 22 NeNe

max2 ))/cos(1)(/cos(2)]/2cos()/[cos(2 dNNNN

)/(12 NDve

e – edge connectivity

(number of edges whose removal will disconnect the graph

e.g., for a chain, e = 1 ; for a triangle, e = 2)

v – node connectivity

D – diameter

Regularly-connected networks Complete graph, ring, chain, star …

Random-graph networks For every pair of nodes, connect them with a certain probability. Poisson node distribution

Small-world networks Similar to random graph, but with short average distance and large clustering, with near-Poisson node distribution

Scale-free networks Heterogeneous, with power-law degree distribution

Spectra of Typical Complex NetworksSpectra of Typical Complex NetworksSpectra of Typical Complex NetworksSpectra of Typical Complex Networks

Piet Van Mieghem, Graph Spectra for Complex Networks (2011)

Complete graphs: 2K-rings:

K=1:

N - even:

K≠1:

Chains:

Stars:

1

2 ( 1)2 2 cos , 2,3,...,

K

il

i lK i N

N

Regularly-Connected NetworksRegularly-Connected Networks

1,..,.12

,4,12

,...,2,1,sin4,0 2

NNN

iN

i

1,..,2,1,2

sin4,0 2

NiN

i

1... ,0 12 N NN /1/2

NN ... ,0 2

NN

1/2 N

24sin , 1,2,..., 1i

i NN

Rectangular Random NetworksRectangular Random Networks

RRN: N nodes are randomly uniformly and independently distributed in a unit rectangle 1 with ],[ 22 baRba

(It can be generalized to higher-dimensional setting)

Two nodes are connected by an edge if they are inside a ball of radius r > 0

Example: N =250

a = 1r = 0.15

E. Estrada and G. Chen (2015)

Na

ar

NN N

224

22

2log

1

)(8

)1(

1

E. Estrada and G. Chen (2015)

Theorem: For RRN, eigenvalues are bounded by

Lower bound

The worst case: all nodes are located on the diagonal

)1(

112

NNND

NN

and

Na

ar

NN N

224

22

2log

1

)(8

)1(

1

E. Estrada and G. Chen (2015)

Upper bound

Lemma 1: Diameter D = diagonal length / r

Lemma 2: Based on a result of Alon-Milman (1985)

ar

aD

14

ND

d 222

max2 log

8

Spectral Role in Network SynchronizationSpectral Role in Network SynchronizationSpectral Role in Network SynchronizationSpectral Role in Network Synchronization

Implication: small local changes affect eigenratio, hence network synchronizability, which however do not affect much the network

statistical properties in general: degree distribution, clustering,

average distance, …

• Recall: (bigger is better)

Topology affects Eigenvalues affects SyncTopology affects Eigenvalues affects Sync

Nd

d

2

max

min

2 2 S / S 0 < |S| /2N

S - any subgraph, with

Statistical properties depend on H, yet depends on S

2

Si SGj

ijaS

Statistics Determines Synchronizability?Statistics Determines Synchronizability?

Answer - Not necessarily

Example:

• They have same structural characteristics: Graphs and have same degree sequence {3,3,3,3,3,3}

• So, all nodes have degree 3 ; same average distance 7/5 ; and same node betweenness 2 ; same ……

Graph 1G Graph 2G

1G 2G

Z. S. Duan, G. R. Chen and L. Huang, Phys. Rev. E, 2007

But they have different synchronizabilities:

Eigenvalues of are and Eigenvalues of are and

Eigenratio satisfies:

1G

2G

2)(3)( 2212 GG

Nr /2 4.0)(5.0)( 21 GrGr

3,3,3,3,05,3,3,2,0

65

G1 G2

Answer: Not necessarily

• Lemma 1: For any given connected undirected graph G , all its nonzero eigenvalues will not decrease with the number of added edges; namely, by adding any edge e , one has

• Note:NiGeG ii ,..,2,1),()(

Z. S. Duan, G. R. Chen and L. Huang, Phys. Rev. E, 2007

Topology Determines Synchronizability ?Topology Determines Synchronizability ?

N2

N2

2

Example

Given Laplacian:

Q: How to replace 0 and -1 while keeping the connectivity (and all row-sums = 0), such that = maximum ?

2 0 0 0 1 1

0 2 1 1 0 0

0 1 3 0 1 1

0 1 0 3 1 1

1 0 1 1 4 1

1 0 1 1 1 4

L

2 N

What Topology Good Synchronizability ??What Topology Good Synchronizability ??

Answer:

*

3 1 0 1 0 1

1 3 1 0 1 0

0 1 3 1 0 1

1 0 1 3 1 0

0 1 0 1 3 1

1 0 1 0 1 3

L

2 N = maximum

Observation: Homogeneous + Symmetrical Topology

With the same numbers of node and edges, while keeping the connectivity, what kind of network has the best possible synchronizability?

Computationally, this is NP-hard

Objective: Find analytic solutions

02 Such that “row-sum = 0” and

matricesLaplacian ofset - * 2

*NNLMax

NLL

(Laplacian) (connected)

ProblemProblemProblemProblem

• Nishikawa et al., Phys. Rev. Lett. 91, 014101(2003), regular networks with uniform small (node or edge) betweenness [we found: edge betweenness is more important than node betweenness]

• Donetti et al., Phys. Rev. Lett. 95, 188701(2005) Entangled networks have biggest eigenratios [we found: not necessarily]

• Donetti et al., J. Stat. Mech.: Theory and Experiment 8, 1742(2006), algorithm based on algebraic graph theory; big spectral gap big eigenratio [we found: the opposite]

• Zhou et al., Eur. Phys. J. B. 60, 89(2007), algorithm based on smallest clustering coefficient

• Hui, Ann Oper. Res., July (2009), algorithm based on entropy • Xuan et al., Physica A 388, 1257(2009), algorithm based on short

average path length• Mishkovski et al., ISCAS, 681(2010), fast generating algorithm• Shi, Chen, Thong, Yan., IEEE CAS Magazine, 13, 1(2013), 3-

regular optimal graphs

ProgressProgressProgressProgress

• Homogeneity + Symmetry

• Same node degree

• Minimize average path length

• Minimize path-sum

• Maximize girth

1 Nd d

1 Ng g

1 Nl l

i ijj il l

Our ApproachOur ApproachOur ApproachOur Approach

D. Shi, G. Chen, W. W. K. Thong and X. Yan, IEEE Circ. Syst. Magazine, (2013) 13(1) 66-75

Illustration:

Grey ： networks with same numbers of nodes and edges Green ： degree-homogeneous networks Blue ： networks with maximum girths Pink ： possible optimal networks Red ： near homogenous networks

White ： Optimal

solution location

Non-Convex OptimizationNon-Convex OptimizationNon-Convex OptimizationNon-Convex Optimization

Optimal 3-Regular NetworksOptimal 3-Regular NetworksOptimal 3-Regular NetworksOptimal 3-Regular Networks

Optimal 3-Regular NetworksOptimal 3-Regular NetworksOptimal 3-Regular NetworksOptimal 3-Regular Networks

Optimal network topology (in the sense of having best possible synchronizability) should have

o Homogeneity

o Symmetry

o Shortest average path-length

o Shortest path-sum

o Longest girth

o Anything else ?

SummarySummarySummarySummary

Open ProblemOpen ProblemOpen ProblemOpen Problem

Looking for optimal solutions:

Nd

d

d

001

0

0

01

101

2

1

Nd

d

d

001

0

0

01

101

2

1

Given: Move which -1 can maximize ?

N2

Where to add -1 can maximize ?N

2

Nd

d

d

001

01

0

011

101

2

1

Where to delete -1 …………………… can maximize ?N

2

And so on …… ???

Thank You !Thank You !Thank You !Thank You !Homogeneity Symmetry

[1] F.Chung, Spectral Graph Theory, AMS (1992, 1997)[2] T.Nishikawa, A.E.Motter, Y.-C.Lai, F.C.Hoppensteadt, Heterogeneity in oscillator

networks: Are smaller worlds easier to synchronize? PRL 91 (2003) 014101[3] H.Hong, B.J.Kim, M.Y.Choi, H.Park, Factors that predict better synchronizability

on complex networks, PRE 65 (2002) 067105[4] M.diBernardo, F.Garofalo, F.Sorrentino, Effects of degree correlation on the

synchronization of networks of oscillators, IJBC 17 (2007) 3499-3506[5] M.Barahona, L.M.Pecora, Synchronization in small-world systems, PRL 89 (5)

(2002)054101[6] H.Hong, M.Y.Choi, B.J.Kim, Synchronization on small-world networks, PRE 69

(2004) 026139[7] F.M. Atay, T.Biyikoglu, J.Jost, Network synchronization: Spectral versus statistical

properties, Physica D 224 (2006) 35–41[8] A.Arenas, A.Diaz-Guilera, C.J.Perez-Vicente, Synchronization reveals topological

scales in complex networks, PRL 96(11) (2006 )114102 [9] A.Arenas, A.Diaz-Guilerab, C.J.Perez-Vicente, Synchronization processes in

complex networks, Physica D 224 (2006) 27–34[10] B.Mohar, Graph Laplacians, in: Topics in Algebraic Graph Theory, Cambridge

University Press (2004) 113–136[11] F.M.Atay, T.Biyikoglu, Graph operations and synchronization of complex

networks, PRE 72 (2005) 016217[12] T.Nishikawa, A.E.Motter, Maximum performance at minimum cost in network

synchronization, Physica D 224 (2006) 77–89

ReferencesReferencesReferencesReferences

[13] J.Gomez-Gardenes, Y.Moreno, A.Arenas, Paths to synchronization on complex networks, PRL 98 (2007) 034101

[14] C.J.Zhan, G.R.Chen, L.F.Yeung, On the distributions of Laplacian eigenvalues versus node degrees in complex networks, Physica A 389 (2010) 17791788

[15] T.G.Lewis, Network Science –Theory and Applications, Wiley 2009[16] T.M.Fiedler, Algebraic Connectivity of Graphs, Czech Math J 23 (1973) 298-305[17] W.N.Anderson, T.D.Morley, Eigenvalues of the Laplacian of a graph, Linear and

Multilinear Algebra 18 (1985) 141-145[18] R.Merris, A note on Laplacian graph eigenvalues, Linear Algebra and its

Applications 285 (1998) 33-35[19] O.Rojo, R.Sojo, H.Rojo, An always nontrivial upper bound for Laplacian graph

eigenvalues, Linear Algebra and its Applications 312 (2000) 155-159[20] G.Chen, Z.Duan, Network synchronizability analysis: A graph-theoretic

approach, CHAOS 18 (2008) 037102[21] T.Nishikawa, A.E.Motter, Network synchronization landscape reveals

compensatory structures, quantization, and the positive effects of negative interactions, PNAS 8 (2010)10342

[22] P.V.Mieghem, Graph Spectra for Complex Networks (2011) [23] D. Shi, G. Chen, W. W. K. Thong and X. Yan, Searching for optimal network

topology with best possible synchronizability, IEEE Circ. Syst. Magazine, (2013) 13(1) 66-75

AcknowledgementsAcknowledgements AcknowledgementsAcknowledgements

Prof Ernesto Estrada, University of Strathclyde, UK

Prof Xiaofan Wang, Shanghai Jiao Tong University

Prof Zhi-sheng Duan, Peking University

Prof Jun-an Lu, Wuhan University

Prof Dinghua Shi, Shanghai University

Dr Juan Chen, Wuhan University

Dr Choujun Zhan, City University of Hong Kong

Dr Wilson W K Thong, City University of Hong Kong

Dr Xiaoyong Yan, Beijing Normal University