Implementation and assessment of machine learning methods for multidisciplinary aerospace problems

Abstract

While many engineering problems are complex in nature nowadays, models that represent systems accurately are crucial in the design process. These models can be constructed physically or mathematically. Recently, mathematical modeling has become a significant process to solve engineering problems when compared to physical modeling which brings about experimental cost. In particular, the advancements in computer processor performance and the development of numerous numerical analysis methods enabled mathematical modeling as an inevitable part of the scientific research. In computational methods, approximate solutions are obtained for systems by utilizing various numerical discretization techniques such as the Finite Volume Method (FVM) and the Finite Element Method (FEM). Each of these methods has its own advantages and disadvantages, and the choice of a method depends largely on the complexity and nature of the problem at hand. However, it is important to note that as the complexity and size of the system increase, the computational costs associated with these methods also increase. Management of computational cost and time becomes challenging particularly when dealing with scenarios that involve multiple system variations, such as optimization and uncertainty analysis, which can be computationally expensive. For computationally expensive processes, especially for those which include optimization and uncertainty quantification processes, surrogate models are employed to reduce computational costs. Surrogate models are methods that are mostly based on mathematical approaches which aim to represent the relationship between inputs and outputs. In recent years, the use of surrogate model-based design, optimization, and uncertainty analysis has become widespread in the aviation and aerospace fields. The main objective in surrogate modeling applications is to maintain high accuracy in representation while minimizing computational costs. In the field of engineering, methods such as Polynomial Chaos Expansion (PCE), Proper Orthogonal Decomposition (POD), Kriging, Radial Basis Functions (RBF), and machine learning are the commonly-utilized surrogate models. This thesis focuses on investigation, implementation and demonstration of machine learning-based surrogate models with different complexity levels of engineering problems in the aerospace discipline. Machine learning, a subfield of artificial intelligence, is generally defined as the ability of a machine to mimic human intelligence. The term "machine learning" was first introduced in 1959 by Arthur Lee Samuel, a pioneer in the field. Machine learning can also be seen as a fusion of linear algebra, statistics, optimization, and computer science. Machine learning algorithms are generally classified into supervised, unsupervised, semi-supervised, and reinforcement learning. Nowadays, machine learning methods are widely applied in almost every domain. These methods are particularly preferred in engineering applications related to aviation and aerospace, such as aircraft design and aerodynamics, air traffic management, flight operations and optimization, and uncertainty analysis. Literature in the field of aviation and aerospace engineering encompasses various applications of machine learning and deep learning-based surrogate modeling. Deep learning methods offer the advantage of high accuracy predictions as the number of samples and the dimensionality increase. On the other hand, machine learning methods tend to perform better with a lower number of samples and dimensionality. In machine learning and deep learning methods, inputs and outputs might be in the form of scalars, vectors, matrices, or tensors. Deep learning methods handle matrix inputs effortlessly, thanks to convolutional layers, while machine learning methods require vectorization of these inputs. This has led to the widespread use of deep learning models in domains such as computer vision, speech recognition, image analysis, and natural language processing. However, in engineering applications, particularly when synthetic data is generated using sampling methods, a vector input-output relationship is observed. In other words, the variables within the system are controlled by the user, and specific outputs are generated depending on the problem. Here, it is important to distinguish between the dimension of the generated synthetic data and the model input dimension. Synthetic data is typically created by varying a set number of parameters and is represented by vectors in terms of model input dimension. For instance, considering the MNIST dataset, which is one of the most commonly used problems in machine learning, consists of 28x28 pixels and is in matrix form. When training this dataset using machine learning methods, it needs to be vectorized into a 784-dimensional vector. On the other hand, in deep learning methods, such pre-processing is not required when utilizing 2-dimensional convolutional layers. Literature shows that both machine learning and deep learning methods yield good results for this dataset. Furthermore, because of their capability of directly processing matrix-shape data, deep learning methods are known to outperform traditional machine learning methods for such kind of problems. However, it is important to note that traditional machine learning methods can also achieve good results in high-dimensional vector prediction tasks. In cases where the data is represented by vectors (except matrix and tensor dimensions), the performance of traditional machine learning methods should not be overlooked. Additionally, it is recommended to start modeling with simpler methods and gradually apply more complex and computationally expensive approaches. Therefore, in the modeling of synthetic data that can be represented by vectors, which is common in the field of engineering, beginning with simpler methods plays a crucial role in computational cost management. On the other hand, in machine learning and deep learning-based modeling, there are several steps that need to be followed in order to enhance prediction performance and accuracy. The lack of adherence to these steps is clearly observed in machine learning-based surrogate models in the field of aviation and aerospace. These steps can be listed as follows: cross-validation, hyperparameter optimization (such as grid search), and model evaluation. The first problem encountered in these steps is the application of any statistical method (e.g., normalization) to the dataset before splitting it into training and testing sets. This leads to a problem known as data leakage, where test data information is used in the modeling process, however it should not be used. Another shortcoming is not implementing cross-validation techniques that improve model accuracy and reliability. The cross-validation method helps to reduce the model dependency on the specific training set and aims for more robust models. In hyperparameter optimization, the best model is typically selected based on its performance in cross-validation, specifically the best test results. After hyperparameter optimization, model evaluation should be conducted using both the training and test data to assess the model's performance. This step helps to prevent overfitting. Presenting results based solely on the test data is not sufficient. The main objective here is to create models with high generalization and prediction capabilities. In this thesis, the machine learning-based surrogate modeling of engineering problems at different levels has been addressed. The aim of this study is to examine the performance of traditional machine learning methods on engineering problems at different complexity levels and disciplines and to develop a modeling guideline for more reliable and efficient surrogate models. Firstly, selected machine learning methods, including linear regression, random forest, gradient boosting, extreme gradient boosting, support vector machine, and multi-layer perceptron, were tested on two benchmark problems. Subsequently, aerospace engineering problems with different dimensions, complexity levels, and disciplines were selected. The first problem involved predicting the aerodynamic coefficients of a supercritical airfoil with 20-dimensional inputs. Another modeling focused on representing the aeroelastic design process of the uCRM 13.5 wing model with 3- and 6-dimensional problems. Finally, the selected methods were applied to a design problem of a supersonic transport aircraft configuration with 20-dimensional geometric variables. The results obtained with these datasets revealed that machine learning methods provided satisfactory results for engineering problems from various disciplines. Moreover, their advantages in terms of the number of hyperparameters and computational costs were demonstrated compared to deep learning methods. Furthermore, the importance of starting with simpler models (in terms of number of hyperparameters) rather than complex deep learning models was once again highlighted through the presented supersonic aircraft design problem.