Graph entropy explained

In information theory, the graph entropy is a measure of the information rate achievable by communicating symbols over a channel in which certain pairs of values may be confused.[1] This measure, first introduced by Körner in the 1970s,[2] [3] has since also proven itself useful in other settings, including combinatorics.[4]

Definition

Let

G=(V,E)

be an undirected graph. The graph entropy of

G

, denoted

H(G)

is defined as

H(G)=minX,YI(X;Y)

where

X

is chosen uniformly from

V

,

Y

ranges over independent sets of G, the joint distribution of

X

and

Y

is such that

X\inY

with probability one, and

I(X;Y)

is the mutual information of

X

and

Y

.[5]

That is, if we let

l{I}

denote the independent vertex sets in

G

, we wish to find the joint distribution

X,Y

on

V x l{I}

with the lowest mutual information such that (i) the marginal distribution of the first term is uniform and (ii) in samples from the distribution, the second term contains the first term almost surely. The mutual information of

X

and

Y

is then called the entropy of

G

.

Properties

G1

is a subgraph of

G2

on the same vertex set, then

H(G1)\leqH(G2)

.

G1=(V,E1)

and

G2=(V,E2)

on the same set of vertices, the graph union

G1\cupG2=(V,E1\cupE2)

satisfies

H(G1\cupG2)\leqH(G1)+H(G2)

.

G1,G2,,Gk

be a sequence of graphs on disjoint sets of vertices, with

n1,n2,,nk

vertices, respectively. Then

H(G1\cupG2\cupGk)=

k
\tfrac{1}{\sum
i=1

ni}\sum

k
i=1

{niH(Gi)}

.

Additionally, simple formulas exist for certain families classes of graphs.

log2k

. In particular,

0

.

n

vertices have entropy

log2n

.

1

.

n

vertices in one partition and

m

in the other have entropy
H\left(n
m+n

\right)

, where

H

is the binary entropy function.

Example

Here, we use properties of graph entropy to provide a simple proof that a complete graph

G

on

n

vertices cannot be expressed as the union of fewer than

log2n

bipartite graphs.

Proof By monotonicity, no bipartite graph can have graph entropy greater than that of a complete bipartite graph, which is bounded by

1

. Thus, by sub-additivity, the union of

k

bipartite graphs cannot have entropy greater than

k

. Now let

G=(V,E)

be a complete graph on

n

vertices. By the properties listed above,

H(G)=log2n

. Therefore, the union of fewer than

log2n

bipartite graphs cannot have the same entropy as

G

, so

G

cannot be expressed as such a union.

\blacksquare

General References

Notes and References

  1. Book: Matthias Dehmer. Abbe Mowshowitz. Frank Emmert-Streib. Advances in Network Complexity. 21 June 2013. John Wiley & Sons. 978-3-527-67048-2. 186–.
  2. Körner. János. 1973. Coding of an information source having ambiguous alphabet and the entropy of graphs.. 6th Prague Conference on Information Theory. 411–425.
  3. Book: Niels da Vitoria Lobo. Takis Kasparis. Michael Georgiopoulos. Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshop, SSPR & SPR 2008, Orlando, USA, December 4-6, 2008. Proceedings. 24 November 2008. Springer Science & Business Media. 978-3-540-89688-3. 237–.
  4. Book: Bernadette Bouchon. Bernadette Bouchon-Meunier . Lorenza Saitta. Lorenza Saitta . Ronald R. Yager. Uncertainty and Intelligent Systems: 2nd International Conference on Information Processing and Management of Uncertainty in Knowledge Based Systems IPMU '88. Urbino, Italy, July 4-7, 1988. Proceedings. 8 June 1988. Springer Science & Business Media. 978-3-540-19402-6. 112–.
  5. G. Simonyi, "Perfect graphs and graph entropy. An updated survey," Perfect Graphs, John Wiley and Sons (2001) pp. 293-328, Definition 2”