LEE- Uydu Haberleşme ve Uzaktan Algılama-Doktora

Bu koleksiyon için kalıcı URI

http://hdl.handle.net/11527/19488

Gözat

Yazar "Sertel, Elif" ile LEE- Uydu Haberleşme ve Uzaktan Algılama-Doktora'a göz atma
  • Öge
    A deep learning based framework for identification of ship types using optical satellite images
    (Graduate School, 2023-01-18) Kızılkaya, Serdar ; Sertel, Elif ; 705152003 ; Satellite Communication and Remote Sensing
    Today, monitoring of ship traffic in open and coastal seas is one of the primary and even indispensable activities of coastal countries for many reasons. Preventing activities such as illegal fishing, migration, smuggling, pollution of the seas, protection of underground resources and sea creatures, defensive reconnaissance, and surveillance activities can be listed as the main reasons for monitoring ship activity in the seas. Periodic monitoring of open and coastal seas using Earth observation satellites is a significant approach, providing fast and cost-effective results as well as large coverage extents. With this perspective, the thesis presents research about an effective and end to end ship monitoring approach via optical satellites. The use of deep learning (DL) techniques for the detection and classification of ships using optical satellite images, the creation of a ship database consisting of optical satellite images for the verification of the subject, and optical constellation modeling are the topics of this research. In the thesis, a comprehensive literature review is presented with the scope of gabs in the ship monitoring with using optical satellites. To mimic the real satellite image input, an optical satellite based image dataset – VHRShips - was formed. VHRShips consists of a total of 6312 images, 1000 without ships and 5312 with ships. There are a total of 11337 ships in the images. The database created includes 24 different types, and navy ships, one of these types, contain 11 different types. This dataset stands out with its rich ship database content and large sample size. Thesis reinterprets DL-based target detection and classification algorithms and proposes a flexible target detection and classification approach in a hierarchical design (HieD) that allows easy addition and removal of different algorithms. In addition, the detection and localization, recognition, and identification (DLRI) steps are staged and the outputs of the algorithm are detailed. In the phase of detecting, the presence of ships is verified in the images provided by the satellites. The determination of the positions of the ships in the images is carried out during the localization stage. The classification of the ships among the determined basic ship types is defined in the recognition stage. Lastly, the classification of navy ships, which is one of the main ship types, is handled during the identification stage. The feasibility of the developed approach has been demonstrated by the preliminary feasibility analysis for the covering of the Turkish surrounding seas with optical constellations. In this analysis, an optimization was carried out by using a software on how the satellite images required for the proposed method can be realized with a set of satellite design. It has been determined that 40 optical satellites are required to fully cover the selected sea area in 24 hours, and 100 optical satellites with the specified characteristics to be covered twice. The results of proposed method, HieD are presented in three formats which are; the individual stage performances, a comprehensive end-to-end evaluation and lastly a comparison with a well-known method, YOLOV4. The results of the thesis are very promising. F1 scores for detection, recognition, localization, and identification were respectively 99.17%, 94.20%, 84.08%, and 82.13% as a consequence of stage-by-stage optimization. The F1 scores for the same order after complete implementation of our suggested method were 99.17%, 93.43%, 74.00%, and 57.05%. End-to-end YOLOv4 produced F1-scores for DLRI of 99.17%, 86.59%, 68.87%, and 56.28%, in opposition. For the steps of localization, recognition, and identification, we outperformed YOLOv4 using HieD. In the thesis; it has been shown that the ship detection, localization, recognition and identification method developed in a hierarchical structure using a challenging data set containing images with different backgrounds in open and coastal seas in many different geographies, works successfully. The method is very open to development with the antecedent and intermediate methods to be added. The data set created in the thesis has been used with various detection and classification techniques, proving that there is no dataset that provides a limited opportunity for the study. VHRShips can serve as a standard for the developed approach as well as for further research into the use of deep learning and advancing geospatial AI applications in the maritime environment. Finally, the thesis presents a vision of the future along with a fair self-criticism.
  • Öge
    Single-frame and multi-frame super-resolution on remote sensing images via deep learning approaches
    (Graduate School, 2022-07-29) Wang, Peijuan ; Sertel, Elif ; 705172003 ; Satellite Communication and Remote Sensing
    As a quite significant computer vision task, image super-resolution (SR) has been widely applied in remote sensing (RS), medical imaging, video surveillance, and biometrics. Image SR aims to restore high-resolution (HR) images by enhancing the spatial, spectral, or temporal resolution of the low-resolution (LR) inputs. In recent years, great efforts have been made for improving the SR approaches. One of the approaches is to design deeper networks. Nevertheless, this greatly increases computation and memory consumption. As a result, some mechanisms (such as cascading networks, attention mechanisms, and back projection) are proposed to improve the performance and the training process of the complex networks. Satellite imagery can be seen in various fields, namely Land cover/Land use classification, road and building extraction, observation of climate, and earthquake prediction. However, in some cases, the resolution of satellite images can not meet the application requirements due to the technology and cost limitations during the satellite design; therefore, the improvement of image resolution might be necessary. Since it is not possible to upgrade the equipment onboard the launched satellite, the software-based SR algorithms are deserved to be explored in RS fields. This thesis aims to strengthen the spatial resolution of optical satellite imageries by using deep learning (DL) methods. Generally, image SR algorithms can be categorized as single-frame image SR (SFSR) and multi-frame image SR (MFSR). The inputs of the SFSR can be a single LR image. While multi-frame image SR aims to restore HR image by using multiple LR images, which can be obtained under different conditions and at different angles. Recently, great contributions have been made to improve the SR methods including two aspects: (1) increasing the value of PSNR (Peak-Signal-Noise-Ratio); (2) improving the image quality perceptually. Nevertheless, some algorithms obtain a high PSNR but with a low perceptual quality which is more important to human perception. Therefore, this thesis has the following objectives: (1) Explore a perceptual-driven approach to enhance the SR image quality visually on single-frame and multi-frame RS imageries; (2) Explore Generative Adversarial Network (GAN)-based models for single-frame and multi-frame RS imagery super-resolution task to fulfill the multi-scale problem and blind to the degradation model; (3) Explore an image fusion method that can generate an arbitrary size of the super-resolved image rather than a small patch. This thesis firstly gives an overview of single-frame and multi-frame RS image SR methods. The single-frame RS image SR methods are briefly classified into supervised and unsupervised methods. The former mainly includes Convolutional Neural Networks (CNN)-based, GAN-based, attention-based, and Back-projection based methods. In addition, the commonly used attention mechanisms including self-attention, channel attention, spatial attention, mixed high-order attention (MHOA), non-local attention (NLA), and non-local sparse attention (NLSA) are also introduced. Moreover, loss functions including pixel-wise loss, perceptual loss, adversarial loss, and cycle consistency loss are presented. For the single-frame RSISR, firstly, an attention CNN-based SR method is proposed. Although CNN-based algorithms have made outstanding achievements in computer vision tasks, the traditional CNNs methods treat the abundant low-frequency information included in the LR inputs equally across channels. Attention-guided algorithms play a vital role in the informative features extraction in various tasks including image SR. With the application of the attention mechanism, the proposed CNN-based method can further learn the deeper relationships among the different channels. Instead of simply integrating the attention module with the residual blocks, a Layer Attention Module (LAM) and Spatial Attention Module (SAM) are proposed to further learn the relationships among the Residual Groups (RG). Moreover, the perceptual loss function is adopted in the training process to enhance the generated image quality perceptually, and Random down-sampling is applied to strengthen the model's generalization ability. Secondly, an attention GAN-based super-resolution method is explored for the single-frame RS images. CNN-based methods have made great contributions to increasing the value of PSNR/SSIM. Nevertheless, the generated outputs tend to be overly smooth and blurry. GAN can generate more realistic images than normal CNN-based methods and has been introduced to single image super-resolution (SRGAN, ESRGAN, EEGAN). Standard GANs only function on spatially local points in LR feature maps. The attention mechanism can directly learn the long-range dependencies in the feature maps both in the generator and discriminator in a powerful way. By applying the attention mechanism, the network allocates attention based on the similarity of color and texture. Therefore, based on ESRGAN, an attention GAN-based method is for the single-frame RS image SR. The ESRGAN was mainly improved from two aspects: (1) we further improved the architecture of the residual blocks by adding more skip connections; (2) we add attention modules to the residual blocks for further feature extraction. Moreover, instead of working on aerial photographs or low-resolution and medium-resolution satellite images, we are focusing on the Very High-Resolution (VHR) satellite imageries, such as the Pleiades, and Worldview-3. The spatial resolutions of the multispectral images for the Pleiades, and Worldview-3 are 2m, and 1.24m, respectively. Furthermore, for the attention CNN-based method, we evaluated the method on the Pleiades and Worldview-3 datasets with scaling factors of 2, 4, and 8. For the attention GAN-based method, we evaluated the method on the Pleiades and Worldview-3 datasets with a scaling factor of 4. The experimental results show the attention-based method can provide better perceptual results both quantitatively and qualitatively. At last, we proposed an attention GAN-based method for the multi-frame RS image SR. Firstly, we introduced an attention mechanism to the Generator and proposed a space-based network that worked on every single frame for better temporal information extraction. Secondly, we proposed a novel attention module for better spatial and spectral information extraction. Thirdly, we applied an attention-based discriminator to enhance the discriminator's discriminative ability. Finally, the experimental results on the SpaceNet7 dataset and Jilin-1 dataset exhibit the superior of the proposed model both quantitatively and qualitatively.