LEE- Mekatronik Mühendisliği-Doktora
Bu koleksiyon için kalıcı URI
Gözat
Konu "deep learning" ile LEE- Mekatronik Mühendisliği-Doktora'a göz atma
Sayfa başına sonuç
Sıralama Seçenekleri
-
ÖgeApplications of deep reinforcement learning for advanced driving assistance systems(Graduate School, 2023-07-23) Yavaş, Muharrem Uğur ; Kumbasar, Tufan ; 518162005 ; Mechatronics EngineeringNowadays, advanced driving support systems are becoming more prevalent every day. For instance, although adaptive cruise control has been present in some mass-produced vehicles since 1980, it is now available in almost every new vehicle model and is becoming usable, especially in congested traffic situations, with the help of developing technology. On the other hand, the autonomous lane centering function developed for highway environments reduces the driving load on drivers. One of the main reasons for the advancement and prevalence of technology is the progress in environmental perception sensors. Decision-making algorithms can obtain high-accuracy positions of lanes and other vehicles' speed and positions on the road by blending data from intelligent camera and radar sensors. Thanks to advancements in artificial intelligence research, the main topic of this thesis is to evaluate the conditions of surrounding vehicles to achieve cruise follow speed, the amount of gas or brake applied, and finally, the lane changing decision by deep reinforcement learning. Deep reinforcement learning is the integration of reinforcement learning theory into new generation artificial neural networks that emerged with the deep learning revolution. In the proposed methods, both the adaptive cruise control and autonomous lane-changing functions designed with deep reinforcement learning have taken more optimal decisions than classical algorithms and the similarity between the decisions taken and those taken by human drivers has been revealed. Adaptive cruise control systems typically calculate the amount of acceleration required to maintain a safe following distance by using information about the distance to the closest vehicle. However, this method is not compatible with human driving behavior, as it involves scanning the entire traffic and taking into account the dynamic elements surrounding the vehicle being driven. In one of our proposed solutions, we designed the adaptive cruise control function using a model-based deep reinforcement learning method. In model-based reinforcement learning, the decision-making policy uses its own internal model during training to minimize interaction with the system. Therefore, one artificial neural network creates the decision-making policy, while a second network creates the internal model. By using the proposed meta-learning approach to train the two neural networks in a closed-loop fashion, we selected two leader vehicle data inputs for the algorithm instead of a single one. In our simulation environment, the model-based artificial intelligence algorithm performed better than the classical intelligent driver model. Additionally, we proposed a hybrid method that switches to the classical driver model if the internal model and real-world observations do not match for a certain period of time, with a fallback mechanism added to the system's internal model. xxiii In the second proposed study on adaptive cruise control, we suggested a discrete driver model inspired by human drivers' use of gas and brake pedals to manipulate them directly. In the analysis performed using data collected from real life, it was observed that drivers were driving at a stable state with certain gas and brake pedals and coped with dynamic conditions by applying delta brake or pedal. Different gas and brake delta levels were determined through statistical inference based on this dataset. In this case, as the inputs of the artificial intelligence algorithm, the position and speeds of all vehicles in a multi-lane highway in front of the vehicle were determined. When considering the superiority of the algorithms that work with a single leader vehicle compared to two leader vehicles on a single lane, the information of the vehicles on the adjacent lanes will help in case of changes in the leading vehicle of the ego vehicle. The deep Q-learning algorithm, which provides the best results in discrete outputs, was used as the decision-making algorithm. In the evaluations performed on both simulation and real test data, the proposed algorithm obtained the highest score. Especially, slowing down the vehicle in line with its own friction by giving a 0 output without pressing both gas and brake pedals, which can be evaluated as tactical decision-making, was frequently preferred by the designed algorithm. The other advanced driver assistance system studied in the thesis work is the autonomous lane-changing function. In the first original study, autonomous lane-changing was designed using deep reinforcement learning method, and the normally long training process was accelerated 5 times with the proposed safety reward feedback. In the autonomous lane-changing problem, the critical task is to process the position and speed information from all vehicles in front and behind in traffic and make safe maneuvers that will cause speed increase at the right time. Especially in complex traffic scenarios created in simulated environments, classical algorithms are adversely affected by sensor uncertainties and noises, and they cannot show optimal performance in the dynamic driving of multiple vehicles. With the uncertainty calculation in the designed deep reinforcement learning algorithm, the confidence level of the decisions made is observed, and progress is made in the important research area of explainable artificial intelligence. It seems that although deep reinforcement learning techniques have achieved significant successes, they still face integration issues in real-world applications. One of the main problems is the lengthy training process, which can take millions of steps, and the fact that policies are optimized through trial and error, making training in real systems impossible. One promising area of research is sim2real transfer, which involves transferring policies trained in simulation directly to real-world applications. In the second original study on autonomous lane changing, a new approach was introduced to measure the transferability between two simulators with different resolutions. The transferability was evaluated using a human-like usage score generated from the traffic situations when lane-changing decisions were made. In the training process, an adjusted reward function was used, and the proposed method outperformed reference methods in terms of both efficiency and safety, achieving the highest human-like lane-changing score.
-
ÖgeDeep reinforcement learning approach in control of Stewart platform- simulation and control(Graduate School, 2023-06-08) Yadavari,Hadi ; İkizoğlu, Serhat ; Aghaei Tavakol, Vahid ; 518162002 ; Mechatronics EngineeringAs named, this work approaches the Stewart platform's controlling task with reinforcement learning methods, presenting a new simulation environment. The Stewart platform, having a broad range of applications that span from flight and driving simulators to structural test platforms, is a fully parallel robot. Exact control of the Stewart platform is challenging and essential in its applications to deliver the desired performance. The fundamental aim of artificial intelligence is to address complex problems by utilizing sensory information with a high number of dimensions. Reinforcement learning (RL) is a specific area of Machine Learning (ML) that incorporates an agent interacting with its surrounding environment according to some policies to maximize the sum of the future rewards as an objective function. The agent's learning process is based on a reward-penalty scheme according to the quality of the selected action from the policy space. In this manner, RL tries to solve many problems and tasks. The primary focus of this work revolves around acquiring the ability to control a sophisticated model of the Stewart platform through the utilization of cutting-edge deep reinforcement algorithms (DRL) and model-based reinforcement learning algorithms. The question is that why do we need a simulation environment? To learn an optimal policy, reinforcement learning necessitates a multitude of interactions with the environment. Experiences with real robots are expensive, time consuming, hard to replicate, and even dangerous. To safely implement the RL algorithms in real-time applications, a reliable simulation environment that considers all the nonlinearities and uncertainties of the agent environment is inevitable. Therefore, an agent could be trained in the simulation through sufficient trials without concerns about the actual hardware issues. After having accurate parameters of the controller learned by the simulation, they can be transferred to a physical real-time system. With the objective of improving the reliability of learning performance and creating a comprehensive test bed that replicates the system's behavior, we introduce a precisely designed simulation environment. For our simulation environment, we opted for the Gazebo simulator, which is an open-source platform utilizing either Open Dynamic Engine (ODE) or Bullet physics. Integrating Gazebo with ROS can pave the way for efficient complex robotic applications due to the ability to simulate different environments involving multi-agent robots. Although some Computer-Aided Design (CAD-based) simulations of the Stewart platform exist, we choose ROS and Gazebo to benefit from the latest reinforcement learning algorithms with high yield and performance, compatible with the last developed RL frameworks. However, despite many robotic simulations in ROS, it lacks parallel applications and closed linkage structures like the Stewart platform. Consequently, our initial step involves creating a parametric representation of the Stewart platform's kinematics within the Gazebo and Robot Operating System (ROS) frameworks. This representation is then seamlessly integrated with a Python class to facilitate the generation of structures.
-
ÖgeDiagnosis of brain cancer and contour normal tissue for radiation therapy based on deep learning methods(Graduate School, 2024-07-18) Halili, Navid ; Doğan, Mustafa ; 518162006 ; Mechatronics EngineeringBrain tumors are one of the deadliest types of cancer ever identified. Rapid and accurate diagnosis of brain tumors, followed by surgical intervention or appropriate treatment, increases the probability of survival. Accurate identification of brain tumors in MRI scans allows precise location of surgical intervention or chemotherapy. Accurate segmentation of brain tumors in MRI scans is challenging due to their varied shapes and requires knowledge and accurate image interpretation. This thesis starts with analyzing machine learning and traditional methods and focuses on the study of edge detection using the Sobel and Canny edge detector algorithm. After that, we use morphology-based techniques to segment the images and evaluate the results. We use K-means techniques for Clustering. Despite various advances, these methods still show limitations in complex situations such as tumor detection and segmentation. In the next step, we analyze the process of dividing photos into parts using transformations. Specifically, we discuss the Wavelet and Contourlet transforms. By using these transformations, we get more detailed information about the analysis of the images. These transformations have many applications, and we can identify the borders of the image and combine them. Finally, we can use this transformation to process and generate deep learning masks using a supervised model. In the following, we analyze new techniques using supervised and deep learning approaches in two specific areas: image classification and image segmentation. As we introduce these methods, we introduce the obstacles facing deep learning and discuss potential strategies to solve and enhance them. Using deep neural networks and the Resnet 50 model, we classify brain images into tumor and non-tumor categories and achieve a satisfactory score of 97% in the F1 criterion. In addition, we introduce and analyze the Unet deep network in deep learning and upgrade it to a RESUNET network for segmentation. The results of this segmentation show that the proposed approach, with different criteria, such as the DICE metric with a score of 0.9434, performs exceptionally during training compared to conventional topologies and shows a faster convergence rate. In the last part, we presented the unsupervised learning system and developed the adversarial generative network to generate brain MRI images. The adversarial generative network is an intelligent network for generating the desired data, and the results show the effectiveness of the adversarial generative network in generating new data. It is of exceptional quality.
-
ÖgeImage reconstruction with deep learning and applications in MR images(Graduate School, 2022-04-22) Aghabiglou, Amir ; Ekşioğlu, Ender Mete ; 518172001 ; Mechatronics EngineeringIn this thesis, in the first step, the novel application of the U-Net structure was considered for the important inverse problem of MRI reconstruction. Deep networks are particularly efficient for the speed-up of the MR image reconstruction process by decreasing the data acquisition time, and they can significantly reduce the aliasing artifacts caused by the undersampling in the k-space. On the first try, it is aimed to develop a novel and efficient unfolding U-Net framework for reconstructing MR images from undersampled k-space data. The new framework should have improved reconstruction performance when compared to competing methodologies. In this step, a novel unfolding framework utilizing the U-Net as a sub-block is being proposed. The introduced U-Net unfolding structure is applied to the magnetic resonance image reconstruction problem. The connection between the unfolding U-Nets is realized in the form of a recently developed projection-based updated data consistency layer. The novel structure is implemented in the PyTorch environment, which is one of the standards for deep learning implementations. The recently created fastMRI dataset which forms an important benchmark for MRI reconstruction is used for training and testing purposes. Despite the many challenges in training rather large networks, novel methodologies have enhanced the capability for having clinical-grade MR image reconstruction in real-time. In recent literature, novel developments have facilitated the utilization of deep networks in various image processing inverse problems. In particular, it has been reported multiple times that the performance of deep networks can be improved by using short connections between layers. In the next step of this thesis, a novel MRI reconstruction method is introduced that utilizes such short connections. The dense connections are used inside densely connected residual blocks. Inside these blocks, the feature maps are concatenated to the subsequent layers. In this way, the extracted information is propagated until the last stage of the block. The efficiency of these densely connected residual blocks was evaluated in MRI reconstruction settings, by augmenting different types of effective deep network models with these blocks in novel structures. The quantitative and qualitative results indicate that this original introduction of the densely connected blocks to the MR image reconstruction problem improves the reconstruction performance significantly. In addition, a novel densely connected residual generative adversarial network (DCR-GAN) is proposed for fast and high-quality reconstruction of MR images. DCR blocks enable the reconstruction network to go deeper by preventing feature loss in the sequential convolutional layers. DCR block concatenates feature maps from multiple steps and gives them as the input to subsequent convolutional layers in a feed-forward manner. In this new model, the DCR block's potential to train relatively deeper structures is utilized to improve quantitative and qualitative reconstruction results in comparison to the other conventional GAN-based models. It can be see from the reconstruction results that the novel DCR-GAN leads to improved reconstruction results without a significant increase in the parameter complexity or run times. The GAN-based structures generally suffer from some limitations. They are slow in convergence and they are unstable during the training step. In this work, these limitations of GAN also was addressed by proposing a new wavelet-based structure. To accomplish this, the wavelet transform packet was incorporated into the GAN structure. The wavelet transform is used in the encoding and decoding steps to create this model. In another word, the downsampling and upsampling layers were replaced with Discrete Wavelet Transform (DWT). DWT is used to replace each pooling process during the contraction phase. As DWT is a reversible package, this downsampling approach guarantees that all information can be retained. DWT can also record both the frequency and position information of feature maps, which will aid in the preservation of fine texture. The inverse wavelet transform is employed in the expansion step to upgrade the size of feature maps. Moreover, recent breakthroughs in this field have inspired us to propose another novel deep unfolding structure for MR image reconstruction. In the last step, the model was trained using not only an iteration of the image itself but also utilizing an updated noise level parameter. The noise level parameter is calculated at each iteration using the error between the network output and the initial zero filling estimate. This new parameter is given as an additional input to the network, and it acts as an evolving regularizer for the image manipulation strength of the network over the unrolling iterations. The introduction of this adaptivity over iterations in the training step also improves the deep models reconstructed image quality in the inference stage. Empirical results indicate that the recommended technique can convergence to better reconstruction results when compared to state-of-the-art unfolding structures devoid of such an adaptive parameter. The introduction of the additional adaptive parameter results in an incremental increase in the parameter complexity, and the required reconstruction times also stand very similar. In this thesis, both quantitative and qualitative results were provided and the proposed model's results were evaluated with cutting-edge techniques in the MR image reconstruction field. Three commonly used evaluation metrics of PSNR, SSIM, and NMSE were used to evaluate simulation results. The statistical differences between developed techniques are investigated using the one-way ANOVA method. Additionally, a t-test is used to specify the major difference between the means of the two proposed structures. Additionally, the robustness of the proposed densely connected residual models was verified by testing them with another dataset type without retraining them. The other dataset differs in size and body tissue type compared to the training dataset. The suggested novel structures in this thesis are improved MR image reconstruction performance compared to state-of-the-art techniques regarding all evaluation metrics. They proved their capacity for reconstructing high-quality images. More importantly, the thesis goal was satisfied regarding the acceleration of MR imaging. The proposed models in this thesis are generally considered to be fast enough to be used even in real-time medical imaging.