Ridge function explained

In mathematics, a ridge function is any function

f:\R ^{d → \R} that can be written as the composition of a univariate function with an affine transformation, that is: f(\boldsymbol{x}) = g(\boldsymbol{x} ⋅ \boldsymbol{a}) for some g:\R → \R and \boldsymbol{a}\in\R ^{d} .Coinage of the term 'ridge function' is often attributed to B.F. Logan and L.A. Shepp.^[1]

Relevance

A ridge function is not susceptible to the curse of dimensionality, making it an instrumental tool in various estimation problems. This is a direct result of the fact that ridge functions are constant in

d -1 directions:Let a _{1,...,a} _{d-1} be d -1 independent vectors that are orthogonal to a , such that these vectors span d -1 dimensions.Then f\left(\boldsymbol{x} + \sum ^{d -1} _{k =1} c _{k\boldsymbol{a}} _{k\right)=g\left(\boldsymbol{x} ⋅ \boldsymbol{a}} + \sum ^{d -1} _{k =1} c _{k\boldsymbol{a}} _{k ⋅ \boldsymbol{a}\right)=g\left(\boldsymbol{x} ⋅ \boldsymbol{a}} + \sum ^{d -1} _{k =1} c _{k0\right)} = g(\boldsymbol{x} ⋅ \boldsymbol{a}) = f(\boldsymbol{x})

for all

c _{i\in\R,1\le} i<d .In other words, any shift of \boldsymbol{x} in a direction perpendicular to \boldsymbol{a} does not change the value of f .

Ridge functions play an essential role in amongst others projection pursuit, generalized linear models, and as activation functions in neural networks. For a survey on ridge functions, see.^[2] For books on ridge functions, see.^{[3] [4]}

Notes and References

1. Logan . B.F. . Shepp . L.A. . Optimal reconstruction of a function from its projections . Duke Mathematical Journal . 1975 . 42 . 4 . 645–659 . 10.1215/S0012-7094-75-04256-8.
2. Konyagin . S.V. . Kuleshov . A.A. . Maiorov . V.E. . Some Problems in the Theory of Ridge Functions . Proc. Steklov Inst. Math. . 2018 . 301 . 144–169 . 10.1134/S0081543818040120. 126211876 .
3. Book: Pinkus. Allan. August 2015. Ridge functions. Cambridge . Cambridge Tracts in Mathematics 205. Cambridge University Press. 215 pp. . 9781316408124 .
4. Book: Ismailov. Vugar. December 2021. Ridge functions and applications in neural networks. Providence, RI . Mathematical Surveys and Monographs 263. American Mathematical Society. 186 pp. . 978-1-4704-6765-4.