Shrinkage Fields (image restoration) explained

Shrinkage fields is a random field-based machine learning technique that aims to perform high quality image restoration (denoising and deblurring) using low computational overhead.

Method

The restored image

x is predicted from a corrupted observation y after training on a set of sample images S .

A shrinkage (mapping) function

{f} _{{\pi} _{i} }\left(v\right)=_^_\exp \left(-\frac^\right) is directly modeled as a linear combination of radial basis function kernels, where \gamma is the shared precision parameter, \mu denotes the (equidistant) kernel positions, and M is the number of Gaussian kernels.

Because the shrinkage function is directly modeled, the optimization procedure is reduced to a single quadratic minimization per iteration, denoted as the prediction of a shrinkage field

{g} _{\Theta} \left( x \right) = { l{F}} ^{-1} \left\lbrack (l{F)/(\left( λ) {K} ^{T} y + {\sum } ^{N} _{i =1} {F} ^{T} _{i} {f} _{{\pi} _{i} }\left(_x\right)\right)}\right\rbrack =^\eta where l{F} denotes the discrete Fourier transform and F _{x} is the 2D convolution f ⊗ x with point spread function filter, \breve{F} is an optical transfer function defined as the discrete Fourier transform of f , and {\breve{F}} ^{*} is the complex conjugate of \breve{F} .{\hat{x}} _{t} is learned as {\hat{x}} _{t} = {g} _{{\Theta} _{t} }\left(_\right) for each iteration t with the initial case {\hat{x}} _{0} = y , this forms a cascade of Gaussian conditional random fields (or cascade of shrinkage fields (CSF)). Loss-minimization is used to learn the model parameters { \Theta }_=_^.

The learning objective function is defined as

J\left({ \Theta }_\right)=_^l\left(_^;_^\right), where l is a differentiable loss function which is greedily minimized using training data {\left\lbrace {x} ^{\left(s\right)} _{gt} ,{y} ^{\left(s\right)} ,{k} ^{\left(s\right)} \right\rbrace } ^{S} _{s =1} and {\hat{x}} ^{\left(s\right)} _{t} .

Performance

Preliminary tests by the author suggest that RTF₅^[1] obtains slightly better denoising performance than

{ CSF }_^, followed by { CSF }_^, { CSF }_^, { CSF }_^, and BM3D.

BM3D denoising speed falls between that of

{ CSF }_^ and { CSF }_^, RTF being an order of magnitude slower.

Advantages

• Results are comparable to those obtained by BM3D (reference in state of the art denoising since its inception in 2007)
• Minimal runtime compared to other high-performance methods (potentially applicable within embedded devices)
• Parallelizable (e.g.: possible GPU implementation)
• Predictability:

O(D log D) runtime where D is the number of pixels

• Fast training even with CPU

Implementations

• A reference implementation has been written in MATLAB and released under the BSD 2-Clause license: shrinkage-fields

See also

• Random field
• Discrete Fourier transform
• Convolution
• Noise reduction
• Deblurring

References

• Shrinkage Fields for Effective Image Restoration . Uwe . Schmidt . Stefan . Roth . 2014 . Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on . IEEE . 978-1-4799-5118-5 . 10.1109/CVPR.2014.349 . Columbus, OH, USA.

Notes and References

1. Regression Tree Fields – An Efficient, Non-parametric Approach to Image Labeling Problems . Jancsary . Jeremy. Nowozin . Sebastian . Sharp. Toby. Rother. Carsten . 10 April 2012 . IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE Computer Society . Providence, RI, USA . 10.1109/CVPR.2012.6247950 .