Algebraic signal processing explained

Algebraic signal processing (ASP) is an emerging area of theoretical signal processing (SP). In the algebraic theory of signal processing, a set of filters is treated as an (abstract) algebra, a set of signals is treated as a module or vector space, and convolution is treated as an algebra representation. The advantage of algebraic signal processing is its generality and portability.

History

In the original formulation of algebraic signal processing by Puschel and Moura, the signals are collected in an

l{A} -module for some algebra l{A} of filters, and filtering is given by the action of l{A} on the l{A} -module.^[1]

Definitions

Let

K be a field, for instance the complex numbers, and l{A} be a K -algebra (i.e. a vector space over K with a binary operation \ast: l{A} ⊗ l{A} \to l{A} that is linear in both arguments) treated as a set of filters. Suppose l{M} is a vector space representing a set signals. A representation of l{A} consists of an algebra homomorphism \rho: l{A} \to End ( l{M}) where End ( l{M}) is the algebra of linear transformations T: l{M} \to l{M} with composition (equivalent, in the finite-dimensional case, to matrix multiplication). For convenience, we write \rho _{a} for the endomorphism \rho(a) . To be an algebra homomorphism, \rho must not only be a linear transformation, but also satisfy the property$$\backslash rho\_\; =\; \backslash rho\_\backslash circ\backslash rho\_a\; \backslash quad\; \backslash forall\; a,\; b\; \backslash in\; \backslash mathcal$$Given a signal x \in l{M} , convolution of the signal by a filter a \in l{A} yields a new signal \rho _{a(x)} . Some additional terminology is needed from the representation theory of algebras. A subset l{G} \subseteq l{A} is said to generate the algebra if every element of l{A} can be represented as polynomials in the elements of l{A} . The image of a generator g \in l{G} is called a shift operator. In all practically all examples, convolutions are formed as polynomials in End ( l{M}) generated by shift operators. However, this is not necessarily the case for a representation of an arbitrary algebra.

Examples

Discrete Signal Processing

In discrete signal processing (DSP), the signal space is the set of complex-valued functions

l{M} = l{L} ^{2(Z)} with bounded energy (i.e. square-integrable functions). This means the infinite series \sum ^{inf ty} _{n =-inf ty} |(x) _{n|} < inf ty where | ⋅ | is the modulus of a complex number. The shift operator is given by the linear endomorphism (S x) _{n} = (x) _{n-1} . The filter space is the algebra of polynomials with complex coefficients l{A} = C [z ^{-1} ,z] and convolution is given by \rho _{h} = \sum ^{inf ty} _{k =-inf ty} h _{k} S ^{k} where h(t) = \sum ^{inf ty} _{k =-inf ty} h _{k} z ^{k} is an element of the algebra. Filtering a signal by h , then yields (y) _{n} = \sum ^{inf ty} _{k =-inf ty} h _{k} x _{n-k} because (S ^{k} x) _{n} = (x) _{n-k} .

Graph Signal Processing

l{G} = ( l{V}, l{E}) with pseudometric on the node set l{V} written a _{ij} . A graph signal is simply a real-valued function on the set of nodes of the graph. In graph neural networks, graph signals are sometimes called features. The signal space is the set of all graph signals l{M} = \R ^{l{V}} where l{V} is a set of n = | l{V}| nodes in l{G} = ( l{V}, l{E}) . The filter algebra is the algebra of polynomials in one indeterminate l{A} = R [t] . There a few possible choices for a graph shift operator (GSO). The (un)normalized weighted adjacency matrix of [A] _{ij} = a _{ij} is a popular choice, as well as the (un)normalized graph Laplacian [L] _{ij} = \begin{cases} \sum ^{n} _{j =1} a _{ij} & i = j \\ - a _{ij} & i ≠ j \end{cases} . The choice is dependent on performance and design considerations. If S is the GSO, then a graph convolution is the linear transformation \rho _{h} = \sum ^{inf ty} _{k =0} h _{k} S ^{k} for some h(t) = \sum ^{inf ty} _{k =0} h _{k} z ^{k} , and convolution of a graph signal x : l{V} \to R by a filter h(t) yields a new graph signal y = \left(\sum ^{inf ty} _{k =0} h _{k} S ^{k} \right) ⋅ x .

Other Examples

Other mathematical objects with their own proposed signal-processing frameworks are algebraic signal models. These objects include including quivers,^[2] graphons,^[3] semilattices,^[4] finite groups, and Lie groups,^[5] and others.

Intertwining Maps

In the framework of representation theory, relationships between two representations of the same algebra are described with intertwining maps which in the context of signal processing translates to transformations of signals that respect the algebra structure. Suppose

\rho: l{A} \to End ( l{M}) and \rho': l{A} \to End ( l{M}') are two different representations of l{A} . An intertwining map is a linear transformation \alpha: l{M} \to l{M}' such that\alpha \circ \rho _{a} = \rho' _{a} \circ \alpha \forall a \in l{A}

Intuitively, this means that filtering a signal by

a then transforming it with \alpha is equivalent to first transforming a signal with \alpha , then filtering by a . The z transform is a prototypical example of an intertwining map.

Algebraic Neural Networks

Inspired by a recent perspective that popular graph neural networks (GNNs) architectures are in fact convolutional neural networks (CNNs),^[6] recent work has been focused on developing novel neural network architectures from the algebraic point-of-view.^{[7] [8]} An algebraic neural network is a composition of algebraic convolutions, possibly with multiple features and feature aggregations, and nonlinearities.

References

1. Puschel . M. . Moura . J. . 2008 . Algebraic Signal Processing Theory: Foundation and 1-D Time . IEEE Transactions on Signal Processing . 56 . 8 . 3572–3585 . 10.1109/TSP.2008.925261 . 2008ITSP...56.3572P . 206797175 . 1053-587X.
2. Parada-Mayorga . Alejandro . Riess . Hans . Ribeiro . Alejandro . Ghrist . Robert . 2020-10-22 . Quiver Signal Processing (QSP) . eess.SP . 2010.11525.
3. Ruiz . Luana . Chamon . Luiz F. O. . Ribeiro . Alejandro . 2021 . Graphon Signal Processing . IEEE Transactions on Signal Processing . 69 . 4961–4976 . 10.1109/TSP.2021.3106857 . 2003.05030 . 2021ITSP...69.4961R . 212657497 . 1053-587X.
4. Puschel . Markus . Seifert . Bastian . Wendler . Chris . 2021 . Discrete Signal Processing on Meet/Join Lattices . IEEE Transactions on Signal Processing . 69 . 3571–3584 . 10.1109/TSP.2021.3081036 . 2012.04358 . 2021ITSP...69.3571P . 227736440 . 1053-587X.
5. Bernardini . Riccardo . Rinaldo . Roberto . 2021 . Demystifying Lie Group Methods for Signal Processing: A Tutorial . IEEE Signal Processing Magazine . 38 . 2 . 45–64 . 10.1109/MSP.2020.3023540 . 2021ISPM...38b..45B . 232071730 . 1053-5888.
6. Gama . Fernando . Isufi . Elvin . Leus . Geert . Ribeiro . Alejandro . 2020 . Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks . IEEE Signal Processing Magazine . 37 . 6 . 128–138 . 10.1109/MSP.2020.3016143 . 2003.03777 . 2020ISPM...37f.128G . 226292855 . 1053-5888.
7. Parada-Mayorga . Alejandro . Ribeiro . Alejandro . 2021 . Algebraic Neural Networks: Stability to Deformations . IEEE Transactions on Signal Processing . 69 . 3351–3366 . 10.1109/TSP.2021.3084537 . 2009.01433 . 2021ITSP...69.3351P . 221517145 . 1053-587X.
8. Parada-Mayorga . Alejandro . Butler . Landon . Ribeiro . Alejandro . 2023 . Convolutional Filtering and Neural Networks with Non Commutative Algebras . IEEE Transactions on Signal Processing . 71 . 2683 . 10.1109/TSP.2023.3293716 . 2108.09923. 2023ITSP...71.2683P .

External links

• Smart Project: Algebraic Theory of Signal Processing at the Department of Electrical and Computer Engineering at Carnegie Mellon University.
• Lecture 12: "Algebraic Neural Networks," University of Pennsylvania (ESE 514).