LEE- Kontrol ve Otomasyon Mühendisliği Lisansüstü Programı

Bu topluluk için Kalıcı Uri

http://hdl.handle.net/11527/19359

Gözat

Yazar "Beke, Aykut" ile LEE- Kontrol ve Otomasyon Mühendisliği Lisansüstü Programı'a göz atma
  • Öge
    Design and deployment of deep learning based fuzzy logicsystems
    (Graduate School, 2023-08-29) Beke, Aykut ; Kumbasar, Tufan ; 504182102 ; Control and Automation Engineering
    In the past decade, significant progress has been made in the field of Deep Learning (DL), driven by innovative learning methods, novel layer structures, and the use of graphics cards for enhanced processing power. This progress has led to the training of neural network models with numerous hidden layers and neurons, resulting in breakthroughs in various domains such as semantic segmentation, object detection, and classification. Deep Neural Networks (DNNs) have proven to be highly effective in machine learning and artificial intelligence applications. DNNs offer advantages over traditional machine learning techniques, including the ability to learn features at multiple layers, which allows them to capture complex features of input data. Through forward pass and backpropagation, DNNs extract meaningful features and outperform other methods in many tasks. As a result, DNNs have gained popularity and are widely used in commercial and industrial applications, contributing to advancements in machine learning. Fuzzy Logic Systems (FLSs) have been employed to various fields and applications over the last years. FLSs use linguistic Fuzzy Sets (FSs) and fuzzy rules, enabling the modeling of human-like reasoning and decision-making processes. This has led to advancements in the development of intelligent control systems capable of effectively handling nonlinear and uncertain dynamics. Besides, FLSs have been applied in image processing, leveraging the FSs to represent uncertain data. FLSs provide robust image analysis, pattern recognition, and image understanding, contributing to advancements in computer vision and image processing applications. Overall, FLSs have been extensively utilized in modeling various systems and phenomena. Their ability to handle uncertainty provides a flexible and interpretable modeling approach, capturing complex relationships and uncertainties in real-world systems. Conventional FLSs, known as Type-1 FLSs (T1-FLSs), have limitations in representing uncertainty. To address this, Type-2 Fuzzy Sets (T2-FSs) have been introduced as an alternative, offering a more flexible representation. T2-FSs can better handle nonlinear and uncertain systems, and T2-Fuzzy Logic Systems (T2-FLSs) have the potential to handle complex problems. However, learning T2-FLSs presents challenges due to their design complexity and the need to learn the parameters associated with fuzzy sets. Different approaches have been proposed, including adapting pre-trained T1-FLSs to T2-FLSs and employing evolutionary algorithms or Neural Network (NN) approaches to optimize the parameters of Interval T2-FLSs (IT2-FLSs). These approaches aim to simplify the design complexity and improve the performance of T2-FLSs. Despite advancements, integrating neural networks and evolutionary algorithms with T2-FLSs faces challenges when applied to extensive datasets. The curse of dimensionality and the increasing number of parameters in T2-FLSs brings some difficulties that is not possible to solve with the current approaches. Recent research has focused on combining FLSs and deep neural networks to overcome these challenges, leading to the development of hybrid models that leverage the strengths of both generalization capabilities of the DNNs and the power of the mini-batch sampled optimization algorithms. In this thesis study, a novel approach is proposed to learn the parameters of T2-FLSs using deep learning-based parameter learning methods. The proposed approach aims to handle extensive datasets and construct models with both a good prediction accuracy and the ability to handle the uncertainties. In the scope of this thesis, specifically, three studies are conducted: the first study (i) is titled with "Learning with Type-2 Fuzzy Activation Functions to Enhance the Performance of Deep Neural Networks", in the second study (ii), we propose a framework which is titled with "More Than Accuracy: A Composite Learning Framework for Interval Type-2 Fuzzy Logic Systems" and in the (iii) last study, we propose reliable uncertainty quantification for GT2-FLSs named as "Towards Reliable Uncertainty Quantification and High Precision with General Type-2 Fuzzy Systems". In the first study (i), we introduce a new method called IT2 Fuzzy Activation Layer (IT2-FAL) that aims to enhance the learning performance of DNNs. The IT2-FAL consists of Single Input IT2 (SIT2) Fuzzy Rectifying Units (FRUs) which used as activation units within the DNN structure to improve learning capabilities. We construct a closed-form representation of the SIT2-FRU structure, and an analysis is conducted to understand how the parameters of this structure influence the generation of input-output mappings. The research findings demonstrate that these mappings can be regarded either as hyperparameters to be set or as parameters to be learned. We provide a learning algorithm to these hyperparameters using DL based frameworks. To evaluate the effectiveness of the proposed IT2-FAL, a comparison is made against existing activation units like ReLU, PReLU, and ELU. The novel SIT2-FRU not only addresses the vanishing gradient problem but also exhibits a fast convergence rate. It achieves this by pushing the mean activation close to zero through the processing of inputs defined in the negative quadrant. This property of SIT2-FRU enables DNNs to exhibit improved learning behavior. The experiments conducted using the selected benchmark datasets show the efficiency and superiority of the IT2-FAL approach. By incorporating the IT2-FAL and its activation units (SIT2-FRU components), DNNs can enhance their learning capabilities and benefit from a more robust and flexible network structure. The proposed approach has the potential to improve the performance of DNNs as the experimental results revealed and it also gives opportunity to enhance the learning capabilities of DNNs. The second study (ii) introduces a novel composite learning approach that utilizes type-reduced sets of Interval Type-2 Fuzzy Logic Systems (IT2-FLSs) to capture uncertainty and establish Prediction Intervals (PIs). Unlike mainstream training approaches that primarily focus on accuracy, the objective of this new approach is to not only achieve high prediction accuracy but also effectively address and capture uncertainty by exploiting the type-reduced sets of IT2-FLSs. In order to achieve such a goal, we identify three main challenges in this context: (1) the capability to handle uncertainty, (2) the construction of a composite loss function, and (3) the development of a learning algorithm that addresses the training complexity while considering the definitions of IT2-FLSs. In (1), to address these challenges, the proposed approach exploits the type-reduced set of IT2-FLSs by combining quantile regression and DL parameter learning methods with IT2-FLSs. The ability of IT2-FLSs to process uncertainty depends on the methods employed for calculating the center-of-sets, while their representation capability is determined by the structure of their antecedent and consequent membership functions. In the scope of thesis, we introduce various parametric IT2-FLSs and defines the learnable parameters for all IT2-FLSs, along with their constraints that need to be satisfied during the training process. In (2), the construction of the loss function is defined which involves construction of a multi-objective loss that is subsequently converted into a constrained composite loss. This composite loss comprises the log-cosh loss component, which aims to optimize accuracy, and a tilted loss component that focuses on the representation of uncertainty. Notably, the tilted loss explicitly utilizes the type-reduced set. In (3), a DL approach is presented for training IT2-FLSs using unconstrained optimizers. The study also introduces parameterization techniques to convert the constrained optimization problem of IT2-FLSs into an unconstrained one without violating the definitions of fuzzy sets. In order to evaluate the effectiveness of the proposed approach, comprehensive comparative results are provided. In the thesis, we provide a hyperparameter sensitivity analysis and inter/intra-model comparisons conducted on various benchmark datasets. These evaluations shed light on the performance and robustness of the proposed novel approach in handling uncertainty and achieving high prediction accuracy for regression problems. In the third study (iii), we present a new learning approach for 𝛼-plane based General Type-2 Fuzzy Logic Systems (GT2-FLSs) to improve pointwise prediction accuracy and generate reliable Prediction Intervals (PIs). The approach focuses on exploiting the shape and size of the Secondary Membership Functions (SMFs) through a novel composite loss function. The novel composite loss function consists of two main components: an uncertainty quantification-focused loss and an accuracy-focused term. Within the uncertainty-focused loss, only the type-reduced set of IT2-FLS associated with the 𝛼0=0 plane, known as the FOU, is explicitly utilized. This allows the SMF size parameters of the GT2-FLS to quantify uncertainty and learn PIs. For the accuracy-focused part, two alternative loss terms are provided. In one approach, the aggregated output of the GT2-FLSs is used directly, while in the other approach, only the output associated with the 𝛼𝐾=1 level is utilized. In both cases, the SMF shape parameters of the GT2-FLS are enforced to enable pointwise prediction with high precision. Thus, different roles are assigned to the IT2-FLS associated with 𝛼-planes within the proposed loss function. Since the output of the 𝛼0=0 plane does not contribute to the output calculation of the GT2-FLS, a partially independent learning of the GT2-FSs becomes possible, allowing for capturing uncertainty while maintaining high accuracy. We present a DL based parameter learning approach for GT2-FLSs to facilitate efficient learning to be able to handle the complex parameter learning problem of the GT2-FLSs and also in the presence of high-dimensional and complex data. This is achieved by defining an unconstrained learning problem. We also proposed novel parameterization tricks such that the definitions of GT2-FSs are not violated. We also provide statistical comparative analyses using benchmark datasets in order to demonstrate the superiority of the proposed learning approach. The results of these analyses show the potential of learning GT2-FLS with the proposed DL based approach as a promising solution for reliable uncertainty quantification with high precision in real-world applications.