Restoring underwater images from turbidity and motion blur – a three-step framework

Abstract

Underwater image restoration is essential for studying and visualizing marine life, aquatic ecosystems, underwater robotics, and optical communication systems. However, underwater imagery processing remains a difficult task because of the combined effects of turbidity, motion blur, and wavelength-dependent absorption. Phenomena like forward and backward scattering due to the suspended particles found in water bodies make imaging even more difficult. Degraded aquatic imagery eventually results from this loss of fine structural details, distorted color information, and decreased visibility, making image processing computationally challenging. Despite earlier underwater restoration models offering a suitable mathematical formulation to quantify such degradations, practical restoration of underwater imagery remains a difficult task. Traditional image enhancement methods that attempt to estimate degradation parameters and restore image quality manually are included in the literature review. However, these are not as reliable, particularly for multi-degradation cases where more than one artifact can be present at the same time. Recent developments in deep learning-based models have led to the widespread use of Generative Adversarial Networks (GANs) for underwater enhancement due to their capacity to produce high-quality synthetic output images. Diffusion models are also becoming increasingly popular for denoising visual data due to their exceptional ability to model real-world noise distributions. Nevertheless, the majority of approaches only deal with one kind of degradation, which restricts their use in actual underwater settings. Another important challenge is the scarcity of paired ground-truth underwater images, which hinders objective evaluation and makes supervised training much more difficult. Taking into consideration the inadequacies discussed above, the proposed approach in this study consists of enhancement framework based on noise parameters estimated using a human-made turbidity dataset. We make use of the pre-processing image enhancement techniques described in the literature to remove artifacts such as uneven illumination and haziness. In the first stage of our framework, Contrast-Limited Adaptive Histogram Equalization (CLAHE) is used to increase local contrast in the image by controlling the extent of noise augmentation. CLAHE basically improves the visibility of structural details and highlights important image features. In the second stage, an Adaptive Color Correction (ACC) module is employed that removes the color hue of the underwater environment by compensating for the absorption of certain wavelengths. In this stage, the images are classified as bluish images, greenish images, yellowish images, and color-balanced images based upon the channel statistics. This intensity adjustment allows the tonal range to be used efficiently while simultaneously enhancing global color fidelity. In the final stage, a diffusion-based denoising model refines the image further. In this case, we have approximated the values of the noise variance by the simulation of a realistic underwater scenario noise data that was prepared using soil and water. The mean and variance parameters extracted from this laboratory data were set to construct the synthesized noisy data images. This created data is then used for model training procedure via the forward diffusion process. Hence, it enables the diffusion model to learn noise patterns that closely resemble the real-life underwater degradations. The reverse diffusion model then finally reconstructs the output images by denoising the degraded images step-by-step. Conventional evaluation metrics like Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) would not be effective in the case of underwater images that lack ground truth. In this work, we evaluated our proposed framework on two datasets: The Underwater Image Enhancement Benchmark Dataset (UIEBD) and the Turbidity Underwater Dataset (TUDS). Both these datasets were chosen as there was variation in their levels of turbidity, distortions in color, and illumination issues. Here, we have used relevant quantitative metrics, including entropy H(X), color accuracy ∆E, Naturalness Image Quality Evaluator (NIQE), Fréchet Inception Distance (FID), and Kernel Inception Distance (KID) to quantify the merits obtained at every stage of the enhancements. The obtained results demonstrated that there were substantial advancements made by the final step of the diffusion model when it comes to perceptual quality, and detail retention. An examination of the resulting images obtained at every step of the architecture further revealed that there have been progressive enhancements in the local contrast, color, and clarity of the noisy images. However, in spite of the very good results achieved by the proposed architecture, several impractical but unavoidable drawbacks are pointed out. Firstly, under some conditions, both CLAHE and ACC maycorrect images too much, which may result in extreme high contrast and unrealistic colors. To avoid such difficulties, it is necessary to manually adjust all parameters pertaining to different images and levels of turbidity. Secondly, it has been observed that a diffusion model performance is greatly dependent on the properties of images being used as the training data and the noise being modeled. This results in a diffusion model with less possibility to generalize well when test images with entirely different properties are considered in the underwater domain. This novel method is a crucial step ahead compared to other existing methods in the underwater image enhancement literature. This is an innovative technique that is capable of effectively simulating the noise that occurs in the real-life environment. It utilizes the noise parameters that were estimated from the turbidity sequences prepared in the laboratory. Secondly, traditional image processing methods applied in our research are capable of preparing the raw underwater images to be processed through enhancement of structural details as well as color contrast in images. Finally, diffusion model based denoising step follows realistic noise patterns and performs well under varying levels of water turbidity. Therefore, this method provides a useful tool in natural underwater image recovery with several applications in underwater communications research and the preservation of marine life.