Loss-specific training of non-parametric image restoration models: A new state of the art

Jeremy Jancsary, Sebastian Nowozin and Carsten Rother
12th European Conference on Computer Vision (ECCV)
October 2012, Florence, Italy

After a decade of rapid progress in image denoising, recent methods seem to have reached a performance limit. Nonetheless, we find that state-of-the-art denoising methods are visually clearly distinguishable and possess complementary strengths and failure modes. Motivated by this observation, we introduce a powerful non-parametric image restoration framework based on Regression Tree Fields (RTF). Our restoration model is a densely-connected tractable conditional random field that leverages existing methods to produce an image-dependent, globally consistent prediction. We estimate the conditional structure and parameters of our model from training data so as to directly optimize for popular performance measures. In terms of peak signal-to-noise-ratio (PSNR), our model improves on the best published denoising method by at least 0.26dB across a range of noise levels. Our most practical variant still yields statistically significant improvements, yet is over 20x faster than the strongest competitor. Our approach is well-suited for many more image restoration and low-level vision problems, as evidenced by substantial gains in tasks such as removal of JPEG blocking artefacts.

Main article

Supp. material

 

 

 

 

Data: Includes corrupted images as well as predictions/restorations by all methods. Note that for denoising, the noisy images as well as predictions are stored as comma-delimited plaintext files rather than .PNG files to avoid truncation issues.

ECCV2012DenoisingData.zip
 size: 3164721632 bytes
 md5sum: b119b9b457f178790b7213d74ceb34a3
ECCV2012DeblockingData.zip  size: 1096537405 bytes
 md5sum: f7e0fe6a290cfb9f3b0f1da805fa08c1
ECCV2012DedustingData.zip
 size: 88312610 bytes
 md5sum: 22bfbc2d796b224cca382a68330c6b35


Code:
The code of our Regression Tree Fields library has now been made available on the Microsoft Research downloads website:

The code includes a slightly simplified implementation of the denoising application presented in the paper (note: the example application is meant for color denoising, rather than B&W, which is considered in the paper; to apply the code to the above data, you will have to modify it slighty or convert the images into color images first).

Please cite as:

@INPROCEEDINGS{Jancsary2012b,
author = {Jeremy Jancsary and Sebastian Nowozin and Carsten Rother},
title = {Loss-specific training of non-parametric image restoration models: A new state of the art},
booktitle = {12th European Conference on Computer Vision (ECCV)},
year = {2012}
}

References: Post-scriptum, the following highly related references were brought to our attention that provide an interesting additional perspective on the improvements obtained by our approach:

P. Chatterjee and P. Milanfar: Is Denoising Dead? IEEE Trans. Image Processing, vol. 19, no. 4, pp. 895-911, April 2010

P. Milanfar: A Tour of Modern Image Filtering, To appear as a feature article in IEEE Signal Processing Magazine

I kindly ask that readers of our paper take these articles into account and apologize for omission of the references.