Hadamard matrix explained

See also: Walsh matrix. In mathematics, a Hadamard matrix, named after the French mathematician Jacques Hadamard, is a square matrix whose entries are either +1 or −1 and whose rows are mutually orthogonal. In geometric terms, this means that each pair of rows in a Hadamard matrix represents two perpendicular vectors, while in combinatorial terms, it means that each pair of rows has matching entries in exactly half of their columns and mismatched entries in the remaining columns. It is a consequence of this definition that the corresponding properties hold for columns as well as rows.

The n-dimensional parallelotope spanned by the rows of an n × n Hadamard matrix has the maximum possible volume among parallelotopes spanned by vectors whose entries are bounded in absolute value by 1. Equivalently, a Hadamard matrix has maximal determinant among matrices with entries of absolute value less than or equal to 1 and so is an extremal solution of Hadamard's maximal determinant problem.

Certain Hadamard matrices can almost directly be used as an error-correcting code using a Hadamard code (generalized in Reed–Muller codes), and are also used in balanced repeated replication (BRR), used by statisticians to estimate the variance of a parameter estimator.

Properties

Let H be a Hadamard matrix of order n. The transpose of H is closely related to its inverse. In fact:

H H ^{sf{T}=} n I _{n}

where I_n is the n × n identity matrix and H^T is the transpose of H. To see that this is true, notice that the rows of H are all orthogonal vectors over the field of real numbers and each have length

\sqrt{n}. Dividing H through by this length gives an orthogonal matrix whose transpose is thus its inverse. Multiplying by the length again gives the equality above. As a result,\operatorname{det}(H) = \pm n ^{n/2} ,

where det(H) is the determinant of H.

Suppose that M is a complex matrix of order n, whose entries are bounded by |M_ij | ≤ 1, for each i, j between 1 and n. Then Hadamard's determinant bound states that

|\operatorname{det}(M)| \leq n ^{n/2} .

Equality in this bound is attained for a real matrix M if and only if M is a Hadamard matrix.

The order of a Hadamard matrix must be 1, 2, or a multiple of 4.^[1] The proof of the nonexistence of Hadamard matrices with dimensions other than 1, 2, or a multiple of 4 follows:

If

n> 1 , then there is at least one scalar product of 2 rows which has to be 0. The scalar product is a sum of n values each of which is either 1 or −1, therefore the sum is odd for odd n, so n must be even.

If

n = 4 m + 2 with m \geq 1 , and there exists an n x n Hadamard matrix H = (h _{i,j} ) _{i,j} }, then it has the property that for any k ≠ l :\sum ^{n -1} _{i =0} h _{k,i} h _{l,i} = 0

Now we define the matrix

A = (a _{i,j} ) _{i,j} } by setting a _{i,j} = h _{0,j} h _{i,j} .Note that A has all 1s in row 0.We check that A is also a Hadamard matrix:\sum ^{n -1} _{i =0} a _{k,i} a _{l,i} = \sum ^{n -1} _{i =0} h _{0,j} h _{k,i} h _{0,j} h _{l,i} = \sum ^{n -1} _{i =0} h ^{2} _{0 ,j} h _{k,i} h _{l,i} = \sum ^{n -1} _{i =0} h _{k,i} h _{l,i} = 0 .

Row 1 and row 2, like all other rows except row 0, must have

n/ 2 entries of 1 and n/ 2 entries of −1 each. (*)

Let

\alpha denote the number of 1s of row 2 beneath 1s in row 1.Let \beta denote the number of −1s of row 2 beneath 1s in row 1.Let \gamma denote the number of 1s of row 2 beneath −1s in row 1.Let \delta denote the number of −1s of row 2 beneath −1s in row 1.

Row 2 has to be orthogonal to row 1, so the number of products of entries of the rows resulting in 1,

\alpha + \delta , has to match those resulting in −1, \beta + \gamma .Due to (*), we also have n/ 2 = \alpha + \gamma = \beta + \delta , from which we can express \gamma = n/ 2 - \alpha and \delta = n/ 2 - \beta and substitute:\alpha + \delta = \beta + \gamma \alpha + (n)/(2) - \beta = \beta + (n)/(2) - \alpha \alpha - \beta = \beta - \alpha \alpha = \beta

But we have as the number of 1s in row 1 the odd number

n/ 2 = \alpha + \beta , contradiction.

Sylvester's construction

Examples of Hadamard matrices were actually first constructed by James Joseph Sylvester in 1867. Let H be a Hadamard matrix of order n. Then the partitioned matrix

\begin{bmatrix} H & H\\ H & - H \end{bmatrix}

is a Hadamard matrix of order 2n. This observation can be applied repeatedly and leads to the following sequence of matrices, also called Walsh matrices.

\begin{align} H _{1} & = \begin{bmatrix} 1 \end{bmatrix}, \\ H _{2} & = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}, \\ H _{4} & = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & -1 & 1 & -1 \\ 1 & 1 & -1 & -1 \\ 1 & -1 & -1 & 1 \end{bmatrix}, \end{align}

and

H _{2 ^{k}} = \begin{bmatrix} H _{2 ^{k-1}} & H _{2 ^{k-1}} \\ H _{2 ^{k-1}} & - H _{2 ^{k-1}} \end{bmatrix} = H _{2} ⊗ H _{2 ^{k-1}} ,

for

2 \le k \in N , where ⊗ denotes the Kronecker product.

In this manner, Sylvester constructed Hadamard matrices of order 2^k for every non-negative integer k.^[2]

Sylvester's matrices have a number of special properties. They are symmetric and, when k ≥ 1 (2^k  > 1), have trace zero. The elements in the first column and the first row are all positive. The elements in all the other rows and columns are evenly divided between positive and negative. Sylvester matrices are closely connected with Walsh functions.

Alternative construction

(\{ 1 , -1 \}, x ) \mapsto (\{ 0 , 1 \}), ⊕ \} , we can describe an alternative construction of Sylvester's Hadamard matrix. First consider the matrix F _{n} , the n x 2 ^{n} matrix whose columns consist of all n-bit numbers arranged in ascending counting order. We may define F _{n} recursively by\begin{align} F _{1} & = \begin{bmatrix} 0 & 1 \end{bmatrix} \\ F _{n} & = \begin{bmatrix} 0 _{1 x 2 ^{n-1}} & 1 _{1 x 2 ^{n-1}} \\ F _{n-1} & F _{n-1} \end{bmatrix}. \end{align}

It can be shown by induction that the image of the Hadamard matrix under the above homomorphism is given by

H _{2 ^{n}} = F ^{sf{T} F} _{n.}

This construction demonstrates that the rows of the Hadamard matrix

H _{2 ^{n}} can be viewed as a length 2 ^{n} linear error-correcting code of rank n, and minimum distance 2 ^{n-1} with generating matrix F _{n.}

This code is also referred to as a Walsh code. The Hadamard code, by contrast, is constructed from the Hadamard matrix

H _{2 ^{n}} by a slightly different procedure.

Hadamard conjecture

The most important open question in the theory of Hadamard matrices is one of existence. Specifically, the Hadamard conjecture proposes that a Hadamard matrix of order 4k exists for every positive integer k. The Hadamard conjecture has also been attributed to Paley, although it was considered implicitly by others prior to Paley's work.^[3]

A generalization of Sylvester's construction proves that if

H _{n} and H _{m} are Hadamard matrices of orders n and m respectively, then H _{n} ⊗ H _{m} is a Hadamard matrix of order nm. This result is used to produce Hadamard matrices of higher order once those of smaller orders are known.

Sylvester's 1867 construction yields Hadamard matrices of order 1, 2, 4, 8, 16, 32, etc. Hadamard matrices of orders 12 and 20 were subsequently constructed by Hadamard (in 1893).^[4] In 1933, Raymond Paley discovered the Paley construction, which produces a Hadamard matrix of order q + 1 when q is any prime power that is congruent to 3 modulo 4 and that produces a Hadamard matrix of order 2(q + 1) when q is a prime power that is congruent to 1 modulo 4.^[5] His method uses finite fields.

The smallest order that cannot be constructed by a combination of Sylvester's and Paley's methods is 92. A Hadamard matrix of this order was found using a computer by Baumert, Golomb, and Hall in 1962 at JPL.^[6] They used a construction, due to Williamson,^[7] that has yielded many additional orders. Many other methods for constructing Hadamard matrices are now known.

In 2005, Hadi Kharaghani and Behruz Tayfeh-Rezaie published their construction of a Hadamard matrix of order 428.^[8] As a result, the smallest order for which no Hadamard matrix is presently known is 668.

By 2014, there were 12 multiples of 4 less than 2000 for which no Hadamard matrix of that order was known.^[9] They are:668, 716, 892, 1132, 1244, 1388, 1436, 1676, 1772, 1916, 1948, and 1964.

Equivalence and uniqueness

Two Hadamard matrices are considered equivalent if one can be obtained from the other by negating rows or columns, or by interchanging rows or columns. Up to equivalence, there is a unique Hadamard matrix of orders 1, 2, 4, 8, and 12. There are 5 inequivalent matrices of order 16, 3 of order 20, 60 of order 24, and 487 of order 28. Millions of inequivalent matrices are known for orders 32, 36, and 40. Using a coarser notion of equivalence that also allows transposition, there are 4 inequivalent matrices of order 16, 3 of order 20, 36 of order 24, and 294 of order 28.^[10]

Hadamard matrices are also uniquely recoverable, in the following sense: If an Hadamard matrix

H of order n has O(n ^{2/log} n) entries randomly deleted, then with overwhelming likelihood, one can perfectly recover the original matrix H from the damaged one. The algorithm of recovery has the same computational cost as matrix inversion.^[11]

Special cases

Many special cases of Hadamard matrices have been investigated in the mathematical literature.

Skew Hadamard matrices

A Hadamard matrix H is skew if

H ^{sf{T}+} H = 2 I. A skew Hadamard matrix remains a skew Hadamard matrix after multiplication of any row and its corresponding column by −1. This makes it possible, for example, to normalize a skew Hadamard matrix so that all elements in the first row equal 1.

Reid and Brown in 1972 showed that there exists a doubly regular tournament of order n if and only if there exists a skew Hadamard matrix of order n + 1. In a mathematical tournament of order n, each of n players plays one match against each of the other players, each match resulting in a win for one of the players and a loss for the other. A tournament is regular if each player wins the same number of matches. A regular tournament is doubly regular if the number of opponents beaten by both of two distinct players is the same for all pairs of distinct players. Since each of the n(n − 1)/2 matches played results in a win for one of the players, each player wins (n − 1)/2 matches (and loses the same number). Since each of the (n − 1)/2 players defeated by a given player also loses to (n − 3)/2 other players, the number of player pairs (i, j ) such that j loses both to i and to the given player is (n − 1)(n − 3)/4. The same result should be obtained if the pairs are counted differently: the given player and any of the n − 1 other players together defeat the same number of common opponents. This common number of defeated opponents must therefore be (n − 3)/4. A skew Hadamard matrix is obtained by introducing an additional player who defeats all of the original players and then forming a matrix with rows and columns labeled by players according to the rule that row i, column j contains 1 if i = j or i defeats j and −1 if j defeats i. This correspondence in reverse produces a doubly regular tournament from a skew Hadamard matrix, assuming the skew Hadamard matrix is normalized so that all elements of the first row equal 1.^[12]

Regular Hadamard matrices

Regular Hadamard matrices are real Hadamard matrices whose row and column sums are all equal. A necessary condition on the existence of a regular n × n Hadamard matrix is that n be a square number. A circulant matrix is manifestly regular, and therefore a circulant Hadamard matrix would have to be of square order. Moreover, if an n × n circulant Hadamardmatrix existed with n > 1 then n would necessarily have to be of the form 4u^ 2 with u odd.^{[13] [14]}

Circulant Hadamard matrices

The circulant Hadamard matrix conjecture, however, asserts that, apart from the known 1 × 1 and 4 × 4 examples, no such matrices exist. This was verified for all but 26 values of u less than 10⁴.^[15]

Generalizations

One basic generalization is a weighing matrix. A weighing matrix is a square matrix in which entries may also be zero and which satisfies

WW ^{sf{T}=} wI for some w, its weight. A weighing matrix with its weight equal to its order is a Hadamard matrix.^[16]

Another generalization defines a complex Hadamard matrix to be a matrix in which the entries are complex numbers of unit modulus and which satisfies H H^* = n I_n where H^* is the conjugate transpose of H. Complex Hadamard matrices arise in the study of operator algebras and the theory of quantum computation.Butson-type Hadamard matrices are complex Hadamard matrices in which the entries are taken to be q^th roots of unity. The term complex Hadamard matrix has been used by some authors to refer specifically to the case q = 4.

Practical applications

• Olivia MFSK – an amateur-radio digital protocol designed to work in difficult (low signal-to-noise ratio plus multipath propagation) conditions on shortwave bands.
• Balanced repeated replication (BRR) – a technique used by statisticians to estimate the variance of a statistical estimator.
• Coded aperture spectrometry – an instrument for measuring the spectrum of light. The mask element used in coded aperture spectrometers is often a variant of a Hadamard matrix.
• Feedback delay networks – Digital reverberation devices which use Hadamard matrices to blend sample values
• Plackett–Burman design of experiments for investigating the dependence of some measured quantity on a number of independent variables.
• Robust parameter designs for investigating noise factor impacts on responses
• Compressed sensing for signal processing and under-determined linear systems (inverse problems)
• Quantum Hadamard gate for quantum computing and the Hadamard transform for quantum algorithms.

See also

• Combinatorial design
• Hadamard transform
• Quincunx matrix
• Walsh matrix
• Weighing matrix
• Quantum logic gate

Further reading

• L. D. . Baumert . Marshall . Hall. Hadamard matrices of the Williamson type. Math. Comp. . 1965 . 19 . 91 . 442–447. 10.1090/S0025-5718-1965-0179093-2 . 0179093. free .
• Book: S. . Georgiou . C. . Koukouvinos . J. . Seberry . Hadamard matrices, orthogonal designs and construction algorithms . 133–205 . Designs 2002: Further computational and constructive design theory . Boston . Kluwer . 2003 . 978-1-4020-7599-5 .
• J. M. . Goethals . J. J. . Seidel. A skew Hadamard matrix of order 36 . J. Austral. Math. Soc.. 1970 . 11 . 3 . 343–344. 10.1017/S144678870000673X . 14193297 .
• Hiroshi . Kimura. New Hadamard matrix of order 24. . 1989 . 5 . 235–242 . 1. 10.1007/BF01788676. 39169723.
• 10.1214/aoms/1177730883 . Alexander M.. Mood. Annals of Mathematical Statistics. 1964. 17 . 4 . On Hotelling's Weighing Problem. 432–446. free.
• K. B. . Reid . E. . Brown . Doubly regular tournaments are equivalent to skew Hadamard matrices . J. Combin. Theory Ser. A . 12 . 1972 . 3 . 332–338 . 10.1016/0097-3165(72)90098-2 . free .
• Jennifer . Seberry Wallis. On the existence of Hadamard matrices. 1976 . J. Comb. Theory A . 21 . 2 . 10.1016/0097-3165(76)90062-5. 188–195. free .
• Jennifer . Seberry. A construction for generalized hadamard matrices. 1980 . J. Statist. Plann. Infer. . 4 . 4 . 10.1016/0378-3758(80)90021-X . 365–368.
• J. . Seberry . B. . Wysocki . T. . Wysocki . On some applications of Hadamard matrices . Metrika . 62 . 2005 . 2–3 . 221–239 . 10.1007/s00184-005-0415-y . 40646 .
• Edward . Spence. Classification of hadamard matrices of order 24 and 28. Discrete Math.. 1995 . 140 . 1–3 . 185–242 . 10.1016/0012-365X(93)E0169-5 . free .
• Book: R. K. . Yarlagadda . J. E. . Hershey . Hadamard Matrix Analysis and Synthesis . 1997 . Boston . Kluwer . 978-0-7923-9826-4 .

External links

• Skew Hadamard matrices of all orders up to 100, including every type with order up to 28;
• Web site: Hadamard Matrix. in OEIS
• Web site: N. J. A. Sloane. Library of Hadamard Matrices. N. J. A. Sloane.
• On-line utility to obtain all orders up to 1000, except 668, 716, 876 & 892.
  • R-Package to generate Hadamard Matrices using R JPL: In 1961, mathematicians from NASA’s Jet Propulsion Laboratory and Caltech worked together to construct a Hadamard Matrix containing 92 rows and columns

Notes and References

1. Web site: UC Denver. 11 February 2023. Hadamard Matrices and Designs.
2. J.J. Sylvester. Thoughts on inverse orthogonal matrices, simultaneous sign successions, and tessellated pavements in two or more colours, with applications to Newton's rule, ornamental tile-work, and the theory of numbers. Philosophical Magazine, 34:461–475, 1867
3. Hedayat . A. . Wallis . W. D. . 6 . . 2958712 . 523759 . 1184–1238 . Hadamard matrices and their applications . 6 . 1978 . 10.1214/aos/1176344370. free . .
4. J. . Hadamard . Résolution d'une question relative aux déterminants . . 17 . 240–246 . 1893 .
5. R. E. A. C. . Paley . On orthogonal matrices . . 12 . 1–4. 311–320 . 1933 . 10.1002/sapm1933121311.
6. L. . Baumert . S. W. . Golomb . M. Jr. . Hall . Discovery of an Hadamard Matrix of Order 92 . . 68 . 3 . 237–238 . 1962 . 10.1090/S0002-9904-1962-10761-7 . 0148686 . free .
7. J. . Williamson . Hadamard's determinant theorem and the sum of four squares . . 11 . 1 . 65–81 . 1944 . 10.1215/S0012-7094-44-01108-7 . 0009590 .
8. H. . Kharaghani . B. . Tayfeh-Rezaie . A Hadamard matrix of order 428 . Journal of Combinatorial Designs . 13 . 2005 . 6 . 435–440 . 10.1002/jcd.20043 . 17206302 .
9. 10.1002/jcd.21358. Đoković. Dragomir Ž. Golubitsky. Oleg . Kotsireas . Ilias S. . Some new orders of Hadamard and Skew-Hadamard matrices. Journal of Combinatorial Designs. 2014. 22. 6. 270–277. 1301.3671. 26598685.
10. Wanless. I.M.. Permanents of matrices of signed ones. Linear and Multilinear Algebra . 2005 . 53 . 6. 427–433 . 10.1080/03081080500093990. 121547091.
11. Kline. J.. Geometric search for Hadamard matrices. Theoretical Computer Science. 2019 . 778 . 33–46. 10.1016/j.tcs.2019.01.025. 126730552. free.
12. Reid. K.B.. Brown. Ezra. Doubly regular tournaments are equivalent to skew hadamard matrices. Journal of Combinatorial Theory, Series A . 1972 . 12 . 3. 332–338 . 10.1016/0097-3165(72)90098-2. free.
13. R. J. . Turyn . Character sums and difference sets . . 15 . 1 . 319–346 . 1965 . 0179098 . 10.2140/pjm.1965.15.319. free .
14. Book: Turyn, R. J. . Sequences with small correlation . H. B. . Mann . Error Correcting Codes . Wiley . New York . 1969 . 195–228 .
15. B. . Schmidt . Cyclotomic integers and finite geometry . . 12 . 4 . 929–952 . 1999 . 10.1090/S0894-0347-99-00298-2 . 2646093 . free . 10356/92085 . free .
16. Geramita . Anthony V. . Pullman . Norman J. . Wallis . Jennifer S. . Families of weighing matrices . Bulletin of the Australian Mathematical Society . Cambridge University Press (CUP) . 10 . 1 . 1974 . 0004-9727 . 10.1017/s0004972700040703 . 119–122. 122560830 .