Laplacian matrix explained

In the mathematical field of graph theory, the Laplacian matrix, also called the graph Laplacian, admittance matrix, Kirchhoff matrix or discrete Laplacian, is a matrix representation of a graph. Named after Pierre-Simon Laplace, the graph Laplacian matrix can be viewed as a matrix form of the negative discrete Laplace operator on a graph approximating the negative continuous Laplacian obtained by the finite difference method.

The Laplacian matrix relates to many useful properties of a graph. Together with Kirchhoff's theorem, it can be used to calculate the number of spanning trees for a given graph. The sparsest cut of a graph can be approximated through the Fiedler vector — the eigenvector corresponding to the second smallest eigenvalue of the graph Laplacian — as established by Cheeger's inequality. The spectral decomposition of the Laplacian matrix allows constructing low dimensional embeddings that appear in many machine learning applications and determines a spectral layout in graph drawing. Graph-based signal processing is based on the graph Fourier transform that extends the traditional discrete Fourier transform by substituting the standard basis of complex sinusoids for eigenvectors of the Laplacian matrix of a graph corresponding to the signal.

The Laplacian matrix is the easiest to define for a simple graph, but more common in applications for an edge-weighted graph, i.e., with weights on its edges — the entries of the graph adjacency matrix. Spectral graph theory relates properties of a graph to a spectrum, i.e., eigenvalues, and eigenvectors of matrices associated with the graph, such as its adjacency matrix or Laplacian matrix. Imbalanced weights may undesirably affect the matrix spectrum, leading to the need of normalization — a column/row scaling of the matrix entries — resulting in normalized adjacency and Laplacian matrices.

Definitions for simple graphs

Laplacian matrix

G

with

n

vertices

v1,\ldots,vn

, its Laplacian matrix L_ is defined element-wise as[1]

Li,j:=\begin{cases} \deg(vi)&ifi=j\\ -1&ifijandviisadjacenttovj\\ 0&otherwise, \end{cases}

or equivalently by the matrix

L=D-A,

where D is the degree matrix and A is the adjacency matrix of the graph. Since G is a simple graph, A only contains 1s or 0s and its diagonal elements are all 0s.

Here is a simple example of a labelled, undirected graph and its Laplacian matrix.

We observe for the undirected graph that both the adjacency matrix and the Laplacian matrix are symmetric, and that row- and column-sums of the Laplacian matrix are all zeros (which directly implies that the Laplacian matrix is singular).

For directed graphs, either the indegree or outdegree might be used, depending on the application, as in the following example:

In the directed graph, both the adjacency matrix and the Laplacian matrix are asymmetric. In its Laplacian matrix, column-sums or row-sums are zero, depending on whether the indegree or outdegree has been used.

Laplacian matrix for an undirected graph via the oriented incidence matrix

The |v| \times |e| oriented incidence matrix B with element Bve for the vertex v and the edge e (connecting vertices v_i and v_j, with i ≠ j) is defined by

Bve=\left\{\begin{array}{rl} 1,&ifv=vi\\ -1,&ifv=vj\\ 0,&otherwise. \end{array}\right.

Even though the edges in this definition are technically directed, their directions can be arbitrary, still resulting in the same symmetric Laplacian |v| \times |v| matrix L defined as

L=BBsf{T}

where B^\textsf is the matrix transpose of B.

An alternative product

Bsf{T}B

defines the so-called |e| \times |e| edge-based Laplacian, as opposed to the original commonly used vertex-based Laplacian matrix L.

Symmetric Laplacian for a directed graph

The Laplacian matrix of a directed graph is by definition generally non-symmetric, while, e.g., traditional spectral clustering is primarily developed for undirected graphs with symmetric adjacency and Laplacian matrices. A trivial approach to apply techniques requiring the symmetry is to turn the original directed graph into an undirected graph and build the Laplacian matrix for the latter.

In the matrix notation, the adjacency matrix of the undirected graph could, e.g., be defined as a Boolean sum of the adjacency matrix

A

of the original directed graph and its matrix transpose

AT

, where the zero and one entries of

A

are treated as logical, rather than numerical, values, as in the following example:

Laplacian matrix normalization

A vertex with a large degree, also called a heavy node, results in a large diagonal entry in the Laplacian matrix dominating the matrix properties. Normalization is aimed to make the influence of such vertices more equal to that of other vertices, by dividing the entries of the Laplacian matrix by the vertex degrees. To avoid division by zero, isolated vertices with zero degrees are excluded from the process of the normalization.

Symmetrically normalized Laplacian

The symmetrically normalized Laplacian matrix is defined as:

Lsym:=(D+)1/2L(D+)1/2=I-(D+)1/2A(D+)1/2,

where

D+

is the Moore–Penrose inverse.

The elements of L^\text are thus given by

sym
L
i,j

:=\begin{cases} 1&ifi=jand\deg(vi)0\\ -

1
\sqrt{\deg(vi)\deg(vj)
} & \mbox i \neq j \mbox v_i \mbox v_j \\ 0 & \mbox.\end

The symmetrically normalized Laplacian matrix is symmetric if and only if the adjacency matrix is symmetric.

For a non-symmetric adjacency matrix of a directed graph, either of indegree and outdegree can be used for normalization:

Left (random-walk) and right normalized Laplacians

The left (random-walk) normalized Laplacian matrix is defined as:

Lrw:=D+L=I-D+A,

where

D+

is the Moore–Penrose inverse.The elements of L^\text are given by
rw
L
i,j

:=\begin{cases} 1&ifi=jand\deg(vi)0\\ -

1
\deg(vi)

&ifijandviisadjacenttovj\\ 0&otherwise. \end{cases}

Similarly, the right normalized Laplacian matrix is defined as

LD+=I-AD+

.

The left or right normalized Laplacian matrix is not symmetric if the adjacency matrix is symmetric, except for the trivial case of all isolated vertices. For example,

The example also demonstrates that if

G

has no isolated vertices, then

D+A

right stochastic and hence is the matrix of a random walk, so that the left normalized Laplacian

Lrw:=D+L=I-D+A

has each row summing to zero. Thus we sometimes alternatively call

Lrw

the random-walk normalized Laplacian. In the less uncommonly used right normalized Laplacian

LD+=I-AD+

each column sums to zero since

AD+

is left stochastic.

For a non-symmetric adjacency matrix of a directed graph, one also needs to choose indegree or outdegree for normalization:

The left out-degree normalized Laplacian with row-sums all 0 relates to right stochastic
+A
D
out
, while the right in-degree normalized Laplacian with column-sums all 0 contains left stochastic
+
AD
in
.

Definitions for graphs with weighted edges

Common in applications graphs with weighted edges are conveniently defined by their adjacency matrices where values of the entries are numeric and no longer limited to zeros and ones. In spectral clustering and graph-based signal processing, where graph vertices represent data points, the edge weights can be computed, e.g., as inversely proportional to the distances between pairs of data points, leading to all weights being non-negative with larger values informally corresponding to more similar pairs of data points. Using correlation and anti-correlation between the data points naturally leads to both positive and negative weights. Most definitions for simple graphs are trivially extended to the standard case of non-negative weights, while negative weights require more attention, especially in normalization.

Laplacian matrix

The Laplacian matrix is defined by

L=D-A,

where D is the degree matrix and A is the adjacency matrix of the graph.

For directed graphs, either the indegree or outdegree might be used, depending on the application, as in the following example:

Graph self-loops, manifesting themselves by non-zero entries on the main diagonal of the adjacency matrix, are allowed but do not affect the graph Laplacian values.

Symmetric Laplacian via the incidence matrix

For graphs with weighted edges one can define a weighted incidence matrix B and use it to construct the corresponding symmetric Laplacian as

L=BBsf{T}

. An alternative cleaner approach, described here, is to separate the weights from the connectivity: continue using the incidence matrix as for regular graphs and introduce a matrix just holding the values of the weights. A spring system is an example of this model used in mechanics to describe a system of springs of given stiffnesses and unit length, where the values of the stiffnesses play the role of the weights of the graph edges.

We thus reuse the definition of the weightless |v| \times |e| incidence matrix B with element Bve for the vertex v and the edge e (connecting vertexes v_i and v_j, with i > j) defined by

Bve=\left\{\begin{array}{rl} 1,&ifv=vi\\ -1,&ifv=vj\\ 0,&otherwise. \end{array}\right.

We now also define a diagonal |e| \times |e| matrix W containing the edge weights. Even though the edges in the definition of B are technically directed, their directions can be arbitrary, still resulting in the same symmetric Laplacian |v| \times |v| matrix L defined as

L=BWBsf{T}

where B^\textsf is the matrix transpose of B.

The construction is illustrated in the following example, where every edge e_i is assigned the weight value i, with i=1, 2, 3, 4.

Symmetric Laplacian for a directed graph

Just like for simple graphs, the Laplacian matrix of a directed weighted graph is by definition generally non-symmetric. The symmetry can be enforced by turning the original directed graph into an undirected graph first before constructing the Laplacian. The adjacency matrix of the undirected graph could, e.g., be defined as a sum of the adjacency matrix

A

of the original directed graph and its matrix transpose

AT

as in the following example:where the zero and one entries of

A

are treated as numerical, rather than logical as for simple graphs, values, explaining the difference in the results - for simple graphs, the symmetrized graph still needs to be simple with its symmetrized adjacency matrix having only logical, not numerical values, e.g., the logical sum is 1 v 1 = 1, while the numeric sum is 1 + 1 = 2.

Alternatively, the symmetric Laplacian matrix can be calculated from the two Laplacians using the indegree and outdegree, as in the following example:

The sum of the out-degree Laplacian transposed and the in-degree Laplacian equals to the symmetric Laplacian matrix.

Laplacian matrix normalization

The goal of normalization is, like for simple graphs, to make the diagonal entries of the Laplacian matrix to be all unit, also scaling off-diagonal entries correspondingly. In a weighted graph, a vertex may have a large degree because of a small number of connected edges but with large weights just as well as due to a large number of connected edges with unit weights.

Graph self-loops, i.e., non-zero entries on the main diagonal of the adjacency matrix, do not affect the graph Laplacian values, but may need to be counted for calculation of the normalization factors.

Symmetrically normalized Laplacian

The symmetrically normalized Laplacian is defined as

Lsym:=(D+)1/2L(D+)1/2=I-(D+)1/2A(D+)1/2,

where L is the unnormalized Laplacian, A is the adjacency matrix, D is the degree matrix, and

D+

is the Moore–Penrose inverse. Since the degree matrix D is diagonal, its reciprocal square root (D^+)^ is just the diagonal matrix whose diagonal entries are the reciprocals of the square roots of the diagonal entries of D. If all the edge weights are nonnegative then all the degree values are automatically also nonnegative and so every degree value has a unique positive square root. To avoid the division by zero, vertices with zero degrees are excluded from the process of the normalization, as in the following example:

The symmetrically normalized Laplacian is a symmetric matrix if and only if the adjacency matrix A is symmetric and the diagonal entries of D are nonnegative, in which case we can use the term the symmetric normalized Laplacian.

The symmetric normalized Laplacian matrix can be also written as

Lsym:=(D+)1/2L(D+)1/2=(D+)1/2BWBsf{T}(D+)1/2=SST

using the weightless |v| \times |e| incidence matrix B and the diagonal |e| \times |e| matrix W containing the edge weights and defining the new |v| \times |e| weighted incidence matrix S=(D^+)^B W^ whose rows are indexed by the vertices and whose columns are indexed by the edges of G such that each column corresponding to an edge e = has an entry \frac in the row corresponding to u, an entry -\frac in the row corresponding to v, and has 0 entries elsewhere.

Random walk normalized Laplacian

The random walk normalized Laplacian is defined as

Lrw:=D+L=I-D+A

where D is the degree matrix. Since the degree matrix D is diagonal, its inverse D^+ is simply defined as a diagonal matrix, having diagonal entries which are the reciprocals of the corresponding diagonal entries of D. For the isolated vertices (those with degree 0), a common choice is to set the corresponding element L^\text_ to 0. The matrix elements of L^\text are given by

rw
L
i,j

:=\begin{cases} 1&ifi=jand\deg(vi)0\\ -

1
\deg(vi)

&ifijandviisadjacenttovj\\ 0&otherwise. \end{cases}

The name of the random-walk normalized Laplacian comes from the fact that this matrix is L^\text = I - P, where P = D^+A is simply the transition matrix of a random walker on the graph, assuming non-negative weights. For example, let e_i denote the i-th standard basis vector. Then x = e_i P is a probability vector representing the distribution of a random walker's locations after taking a single step from vertex i; i.e., x_j = \mathbb\left(v_i \to v_j\right). More generally, if the vector x is a probability distribution of the location of a random walker on the vertices of the graph, then x' = x P^t is the probability distribution of the walker after t steps.

The random walk normalized Laplacian can also be called the left normalized Laplacian

Lrw:=D+L

since the normalization is performed by multiplying the Laplacian by the normalization matrix

D+

on the left. It has each row summing to zero since

P=D+A

is right stochastic, assuming all the weights are non-negative.

In the less uncommonly used right normalized Laplacian

LD+=I-AD+

each column sums to zero since

AD+

is left stochastic.

For a non-symmetric adjacency matrix of a directed graph, one also needs to choose indegree or outdegree for normalization:

The left out-degree normalized Laplacian with row-sums all 0 relates to right stochastic
+A
D
out
, while the right in-degree normalized Laplacian with column-sums all 0 contains left stochastic
+
AD
in
.

Negative weights

Negative weights present several challenges for normalisation:

Properties

For an (undirected) graph G and its Laplacian matrix L with eigenvalues \lambda_0 \le \lambda_1 \le \cdots \le \lambda_:

\begin{align} λi&=

sf{T}L
v
i

vi\\ &=

sf{T}M
v
i

sf{T}Mvi\\ &=\left(M

sf{T}\left(M
v
i\right)

vi\right).\\ \end{align}

Because \lambda_i can be written as the inner product of the vector M \mathbf_i with itself, this shows that \lambda_i \ge 0 and so the eigenvalues of L are all non-negative.

Lrw=

-1
2
I-D

\left(I-Lsym\right)

1
2
D
,

i.e., L^\text is similar to the normalized Laplacian L^\text. For this reason, even if L^\text is in general not symmetric, it has real eigenvalues — exactly the same as the eigenvalues of the normalized symmetric Laplacian L^\text.

Interpretation as the discrete Laplace operator approximating the continuous Laplacian

The graph Laplacian matrix can be further viewed as a matrix form of the negative discrete Laplace operator on a graph approximating the negative continuous Laplacian operator obtained by the finite difference method.(See Discrete Poisson equation)[2] In this interpretation, every graph vertex is treated as a grid point; the local connectivity of the vertex determines the finite difference approximation stencil at this grid point, the grid size is always one for every edge, and there are no constraints on any grid points, which corresponds to the case of the homogeneous Neumann boundary condition, i.e., free boundary. Such an interpretation allows one, e.g., generalizing the Laplacian matrix to the case of graphs with an infinite number of vertices and edges, leading to a Laplacian matrix of an infinite size.

Generalizations and extensions of the Laplacian matrix

Generalized Laplacian

The generalized Laplacian

Q

is defined as:[3]

\begin{cases} Qi,j<0&ifijandviisadjacenttovj\\ Qi,j=0&ifijandviisnotadjacenttovj\\ anynumber&otherwise. \end{cases}

Notice the ordinary Laplacian is a generalized Laplacian.

Admittance matrix of an AC circuit

The Laplacian of a graph was first introduced to model electrical networks.In an alternating current (AC) electrical network, real-valued resistances are replaced by complex-valued impedances.In models of such networks, the entries of the adjacency matrix are complex, but the Kirchoff matrix remains symmetric, rather than being Hermitian.Such a matrix is usually called an "admittance matrix", denoted

Y

, rather than a "Laplacian".This is one of the rare applications that give rise to complex symmetric matrices.

Magnetic Laplacian

There are other situations in which entries of the adjacency matrix are complex-valued, and the Laplacian does become a Hermitian matrix. The Magnetic Laplacian for a directed graph with real weights

wij

is constructed as the Hadamard product of the real symmetric matrix of the symmetrized Laplacian and the Hermitian phase matrix with the complex entries

\gammaq(i,j)=

i2\piq(wij-wji)
e
which encode the edge direction into the phase in the complex plane.In the context of quantum physics, the magnetic Laplacian can be interpreted as the operator that describes the phenomenology of a free charged particle on a graph, which is subject to the action of a magnetic field and the parameter

q

is called electric charge.[4] In the following example

q=1/4

:

Deformed Laplacian

The deformed Laplacian is commonly defined as

\Delta(s)=I-sA+s2(D-I)

where I is the identity matrix, A is the adjacency matrix, D is the degree matrix, and s is a (complex-valued) number. [5]
The standard Laplacian is just \Delta(1) and \Delta(-1) = D + A is the signless Laplacian.

Signless Laplacian

The signless Laplacian is defined as

Q=D+A

where

D

is the degree matrix, and

A

is the adjacency matrix.[6] Like the signed Laplacian

L

, the signless Laplacian

Q

also is positive semi-definite as it can be factored as

Q=RRsf{T}

where R is the incidence matrix.

Q

has a 0-eigenvector if and only if it has a bipartite connected component (isolated vertices being bipartite connected components). This can be shown as

xsf{T}Qx=xsf{T}RRsf{T}x\impliesRsf{T}x=0.

This has a solution where

x0

if and only if the graph has a bipartite connected component.

Directed multigraphs

An analogue of the Laplacian matrix can be defined for directed multigraphs.[7] In this case the Laplacian matrix L is defined as

L=D-A

where D is a diagonal matrix with Di,i equal to the outdegree of vertex i and A is a matrix with Ai,j equal to the number of edges from i to j (including loops).

Open source software implementations

Application software

See also

Notes and References

  1. Book: Chung , Fan . Fan Chung. Spectral Graph Theory. 1992. 1997. American Mathematical Society. 978-0821803158.
  2. .
  3. Book: Godsil . C. . Royle . G. . 2001 . Algebraic Graph Theory, Graduate Texts in Mathematics . Springer-Verlag.
  4. Graph Signal Processing for Directed Graphs based on the Hermitian Laplacian . ECML PKDD 2019: Machine Learning and Knowledge Discovery in Databases . 447–463 . 2020. 10.1007/978-3-030-46150-8_27. Satoshi Furutani . Toshiki Shibahara. Mitsuaki Akiyama. Kunio Hato. Masaki Aida .
  5. The Deformed Consensus Protocol . F. . Morbidi . Automatica . 49 . 10 . 3049–3055 . 2013 . 10.1016/j.automatica.2013.07.006. 205767404 .
  6. Cvetković. Dragoš. Simić. Slobodan K.. 2010. Applicable Analysis and Discrete Mathematics. 4. 1. 156–166. 1452-8630. 43671298. Towards a Spectral Theory of Graphs Based on the Signless Laplacian, III. 10.2298/AADM1000001C. free.
  7. Matrix Tree Theorems . Chaiken, S. . Kleitman, D. . Daniel Kleitman . Journal of Combinatorial Theory, Series A . 24 . 3 . 377–381 . 1978 . 0097-3165 . 10.1016/0097-3165(78)90067-5. free .
  8. Web site: SciPy. GitHub. 4 October 2023.
  9. Web site: NetworkX. GitHub. 4 October 2023.
  10. Web site: Julia. GitHub. 4 October 2023.
  11. Web site: 2.3. Clustering.
  12. Web site: PyGSP: Graph Signal Processing in Python. GitHub. 23 March 2022.
  13. Web site: Megaman: Manifold Learning for Millions of Points. GitHub. 14 March 2022.
  14. Web site: SmoothG. GitHub. 17 September 2020.
  15. Web site: Announcing Our Paper at KDD 2020.
  16. Web site: Harshangrjn/LaplacianOpt.jl. GitHub. 2 February 2022.
  17. Web site: LigMG (Large Irregular Graph MultiGrid)-- A distributed memory graph Laplacian solver for large irregular graphs. GitHub. 5 January 2022.
  18. Web site: Laplacians.jl. GitHub. 11 March 2022.