Deep learning for inverse problems in imaging

Abstract

Efforts to solve inverse imaging problems with deep learning techniques have increased the performance results of the algorithms. However, it has been observed that the increase in the performance of deep networks is mostly directly proportional to their more advanced and powerful architectural design. Acting with a pure architectural design leads researchers to dead end in the development of new solutions. On the other hand, in the classical era before deep learning, inverse imaging problems have been solved by making use of clean image models. Among model-based methods from classical period, the brightest results belong to the algorithms based on sparsity in transform domain. Contrary to this known fact, the common habit in deep learning literature to solve inverse problems is to find a model (map) on pixel domain rather than transform domain. Only a few studies have addressed training of deep networks in transform domain. In image denoising problem, deep networks that prefer training in transform domain have mostly chosen the discrete wavelet transform. The major factor in such a choice is that the wavelet transform produces image-like spectrum coefficients (subband images). Convolution layer is widely used in architecture of networks which are proposed for inverse imaging problems and it searches for a relationship between neighboring values of the input data of a convolution layer. In other words, it is reasonable to use wavelet transform coefficients in deep networks. Therefore, these wavelet-based networks have given effective results for inverse imaging problems. However, transforms such as DCT, which are known to provide good energy compaction property for most images in solving inverse problems, have not been preferred in deep networks. This is because they do not produce spectra such as wavelet subband images. In JPEG compression artifact removal problem, the primary source of compression artifact is the quantization of the transform coefficients of an image. During the quantization, transforms which have high compression ability such as the DCT are chosen. However, the majority of compression artifact removal algorithms have used deep neural networks that find a map in pixel domain. Based on these observations above, in this thesis study, novel transform based approaches are presented for image denoising and JPEG compression artifact removal problems. DCTNet is a deep convolutional neural network that utilizes the DCT for image denoising problem. In DCTNet, DCT coefficients of image patches extracted overlappingly from noisy image are calculated. Then, the spectral coefficients of all the patches are ordered to form a channel, which are suitable for subsequent processing in convolution layers. It has been shown mathematically that such a usage corresponds to the process of convolution of 2D DCT basis images with noisy image. Similarly, it has been shown that the calculation of inverse DCT coefficients can be done by a convolution operation with the same basis images. In this way, effective training of CNN networks in the DCT domain is carried out and it is shown that the proposed DCTNet give successful results in image denoising problem. Harmonic Nets are proposed by utilizing the DCTNet structure for JPEG compression artifact removal problem. In addition to the DCT, sine and Hartley transforms are also utilized to remove compression artifacts. These two transforms having high compression capability, which have not been discussed in the context of JPEG until now, are used in deep networks for the first time. Architectural changes have occurred in designing the proposed Harmonic Nets due to some differences between all three transforms. Experimental study have shown that although the proposed networks have fewer parameters and a simpler network topology, they surpass some of the advanced deep networks with the highest performance results and lag behind the others by a small margin. Within the scope of this thesis, compressed sensing MRI problem, which is a common technique in reconstructing magnetic resonance images, is also discussed. Over the past decade, it has been shown theoretically and empirically that the solution of additive white Gaussian noise removal problem is important not only for denoising problem but also for other inverse imaging problems. In plug-and-play methods, additional constraints are added to the cost function of any inverse problem. Since one step of the resulting new problem is similar to Gaussian denoising problem, this step is attempted to be solved with any Gaussian denoiser iteratively. In this study, inspired by PnP research wave, a simple and vanilla convolutional neural network for Gaussian denoising problem is proposed for CS MRI problem. In the experiments where convergence analysis of the proposed scheme is performed, we confirmed that our algorithm is successful for MR image reconstruction.