Semi-orthogonal matrix explained

In linear algebra, a semi-orthogonal matrix is a non-square matrix with real entries where: if the number of columns exceeds the number of rows, then the rows are orthonormal vectors; but if the number of rows exceeds the number of columns, then the columns are orthonormal vectors.

Equivalently, a non-square matrix A is semi-orthogonal if either

A ^{\operatorname{T} } A = I \text A A^ = I. \,^{[1] [2] [3]}

In the following, consider the case where A is an m × n matrix for m > n.Then

A ^{\operatorname{T} } A = I_n, \textA A ^{\operatorname{T} } = \text A.

The fact that $A^\; A\; =\; I\_n$ implies the isometry property

\|A x\| _{2} = \|x\| _{2} for all x in Rⁿ.

For example,

\begin{bmatrix} 1 \ 0 \end{bmatrix} is a semi-orthogonal matrix.

A semi-orthogonal matrix A is semi-unitary (either A^†A = I or AA^† = I) and either left-invertible or right-invertible (left-invertible if it has more rows than columns, otherwise right invertible). As a linear transformation applied from the left, a semi-orthogonal matrix with more rows than columns preserves the dot product of vectors, and therefore acts as an isometry of Euclidean space, such as a rotation or reflection.

Notes and References

1. Abadir, K.M., Magnus, J.R. (2005). Matrix Algebra. Cambridge University Press.
2. Zhang, Xian-Da. (2017). Matrix analysis and applications. Cambridge University Press.
3. Povey, Daniel, et al. (2018). "Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks." Interspeech.