Doğrusal olmayan sistemler için model öngörülü kontrol yöntemine ters optimal kontrol yapısının katılması
Doğrusal olmayan sistemler için model öngörülü kontrol yöntemine ters optimal kontrol yapısının katılması
Dosyalar
Tarih
2021-08-02
Yazarlar
Ulusoy, Lütfi
Süreli Yayın başlığı
Süreli Yayın ISSN
Cilt Başlığı
Yayınevi
Lisansüstü Eğitim Enstitüsü
Özet
Optimal kontrol probleminin amacı, bazı kontrol ve durum kısıtlamalarını sağlayacak ve bir başarım kriterini optimize edecek şekilde bir kontrol giriş fonksiyonu veya kontrol kuralı elde etmektir. Buna rağmen, optimal kontrol kuralı, kısıtsız ve doğrusal durumlarda bile oldukça kolay ve analitik olarak bulunamaz. Optimal kontrol kuralının çözümünün Hamilton-Jacobi-Bellman (HJB) denklemini çözmeyi gerektirdiği iyi bilinen bir gerçektir ki bu son derece zordur. Dahası, doğrusal olmayan sistemlerin çoğu için analitik bir HJB çözümü mevcut değildir. Sistem doğrusal olduğunda ve başarım kriteri ikinci dereceden olduğunda, HJB, belirli durumlarda analitik olarak çözülmesi zor olabilen bir Riccati denklemi olarak ortaya çıkar. Bu zorlukların üstesinden gelmek amacıyla önceden belirlenmiş bir sonlu ufuk için mevcut sistem durumunu, başlangıç durumu olarak atayarak, sistem modeli yardımıyla optimal kontrol problemini tekrar tekrar ve ardışıl olarak çözmek düşünülmüştür. Bu stratejiyi kullanan kontrol yaklaşımları, Model Öngörülü Kontrol (MÖK) olarak adlandırılır. Bu yaklaşımda, sistemin gelecekteki davranışı, sistem modeli kullanılarak tahmin edilir ve kontrol işareti, anlık sistem durumlarına göre her kontrol ufku için tekrar tekrar yenilenir. Öte yandan, HJB problemini çözmek yerine bize farklı bir bakış açısı sağlayan bir başka yaklaşım ise Ters Optimal Kontrol (TOK) teorisidir. TOK, HJB denklemini çözmenin zahmetli görevinden kaçınarak, doğrusal olmayan optimal kontrol problemini çözmek için alternatif bir yaklaşımdır. Son yıllarda, birçok gerçek zamanlı uygulamada doğrusal olmayan optimal kontrol problemlerini çözmek için ters optimizasyon yaklaşımı giderek daha fazla kullanılmaktadır. Tezde, ilk olarak model öngörülü kontrol yaklaşımının optimal kontrol problemini ele alış biçimi anlatılmıştır. Önerilecek yöntem ile karşılaştırabilmek amacıyla, klasik model öngörülü yaklaşımlarından, doğrusal sistem modelini kullanan gradyant tabanlı MÖK ve doğrusal olmayan sistem modeli Runge-Kutta tabanlı MÖK (RKMÖK) yaklaşımları verilmiştir. Daha sonra ters optimal kontrol (TOK) yaklaşımları incelenmiş ve ayrık-zamanlı girişte-afin doğrusal olmayan sistemler için TOK problemini Kontrol Lyapunov Fonksiyonu (KLF) bulma problemine dönüştürerek çözen TOK yaklaşımı anlatılmıştır. TOK yaklaşımı için takip probleminde karşılaşılabilecek sorunlar üzerinde durulmuştur. Bu tezde ilk olarak, takip problemi sorunlarını çözebilmek amacıyla kontrol işareti ağırlık matrisinin her bir elemanı için sistem durum değişkenlerine bağlı bir sigmoid fonksiyon önerilmiştir. Önerilen yaklaşımın başarımını gösterebilmek için klasik TOK yaklaşımıyla karşılaştırma yapılmıştır. Bu tez çalışmasında, ayrıca girişte-afin doğrusal olmayan sistemler için MÖK ve TOK yaklaşımları birleştirilerek yeni bir optimal kontrol yöntemi önerilmektedir. Gerçek hayatta ve literatürde karşılaşılan doğrusal olmayan sistemlerin ve sistem modellerinin çoğu, bazı doğrusal olmayan azaltma yöntemleri ile girişte-afin biçime dönüştürülebilir. Önerilen yöntemin temel özelliği, her kayan ufuk ve sonuç olarak yeni bir başlangıç koşulu için çözülmesi gereken MÖK optimizasyon problemini TOK problemi olarak ele alıp, bu TOK problemini tekrar tekrar çözmesidir. Bu yaklaşımda, sistemin gelecekteki davranışının tahminini elde etmek için sistem modeli kullanılır ve önceden belirlenmiş bir kontrol ufku için TOK yönteminden elde edilen kontrol işareti sisteme uygulanır. TOK probleminin çözümü aşamasında, belirlenmesi gereken aday kontrol Lyapunov fonksiyon matrisinin parametreleri, evrimsel Büyük Patlama-Büyük Çöküş (BP-BÇ) optimizasyon arama algoritması kullanılarak çevrim içi bir şekilde tahmin edilir. Önerilen kontrol yapısında, MÖK yaklaşımında her kontrol ufku için uygun bir KLF matrisinin aranması ile optimal kontrol problemi çözülmektedir. Diğer bir bakış açısından ise, MÖK yapısı TOK problemine dahil edilerek TOK problemi, her kayan ufkun başlangıcındaki farklı başlangıç koşulları kullanılarak tekrar tekrar çözülmekte ve böylece, TOK için çevrim içi bir düzeltme mekanizması elde edilmektedir. Bu yaklaşım ve literatürdeki diğer yöntemler kullanılarak top ve çubuk kontrol sistemi üzerinde benzetim çalışmaları ve gerçek zamanlı uygulama yapılmıştır. Elde edilen sonuçlar bazı kontrol başarım ölçütlerine karşılaştırılmış ve önerilen yaklaşımın başarımı değerlendirilmiştir.
The aim of optimal control problem is to form a control input function or control law such that a performance criterion is minimized while satisfying some control and state constraints. However, the optimal control law cannot be found quite easily and analytically even in the unconstrained and linear cases. It is a well-known fact that the solution for optimal control law requires solving Hamilton-Jacobi-Bellman (HJB) equation which is extremely difficult. Moreover, an analytical solution of HJB does not exist for most nonlinear systems. When the system is linear and the performance criterion is quadratic, HJB turns out to be a Riccati equation which may also be difficult to solve analytically for certain cases. One approach to overcome these difficulties is to solve the optimal control problem repeatedly and sequentially using the system model and assigning the current system state as the initial state for a predetermined finite horizon. The out-coming control sequence that corresponds to a predetermined control horizon is then applied to the real-time system. Control approaches using this strategy are referred to as Model Predictive Control (MPC). In this approach, the future behavior of the system is predicted using the system model and the control signal is renewed repeatedly by the current system states for each control horizon. The model of the system plays a crucial role in all MPC techniques. The methods in which the original nonlinear model is linearized around an operating point are referred to be Linear MPC methods. These methods have been used as effective and robust tools for controlling many industrial processes. However, the effect of linearization operation may degrade for the systems that possess high nonlinearity and for the operating conditions that require the whole operating region. Various Nonlinear MPC (NMPC) approaches are presented and their reviews have been given in literature. Modeling strategies for model building, model reduction and stability have still been active working topics for NMPC approaches. The computational burden and the need for an efficient optimization approach is another issue associated with the solution of the NMPC optimization problem. Although different techniques are proposed for MPC, the MPC loop approach repeated in all sampling intervals in all MPC methods is basically the same as the basis of MPC is based on the receding horizon principle. The repeated MPC loop processes for the horizon moving towards the future in each sampling interval can be summarized as follows: - Using the model of the system to be controlled, the future outputs or states are calculated depending on the past input, output and / or system state variables and the input values planned to be implemented in the future. - By solving a finite-horizon open-loop optimization problem that is determined to converge the system state variables or output to the desired reference value and to minimize the cost function, a set of future control signals are obtained. - As the control signal obtained along the control horizon, the first element of the sequence is applied to the system. This is the receding horizon strategy, a concept of predictive control. On the other hand, Inverse Optimal Control (IOC) theory provides us a different perspective for solving HJB problem. Therefore, IOC is an alternative approach to solve nonlinear optimal control problem while avoiding the tedious task of solving the HJB equation. The perspective of the inverse optimal control problem was put forward by Kalman (1964) in the early 1960s. Kalman stated that when a dynamic system and a feedback control law are given and the closed-loop system is asymptotically stable, the inverse problem is to seek the most general performance index for which this control law is optimal. In fact, the IOC can be seen as an approach rather than a methodology that perceives the optimal control problem from the opposite side. In IOC method when the controller is desired to be optimal according to stable and meaningful objective functions, the control Lyapunov function (CLF) based approaches are widely used. The formulation of CLF, which provides the design of an optimal feedback controller for typical classes of systems, has been discussed in the literature. The existence of CLF implies stabilizability. Thus, the distinguishing aspect of this approach is that the performance measure corresponding to the stabilizing feedback control is determined posteriori. Since there exists no explicit technique for the determination of CLF for general nonlinear systems, the most challenging aspect of IOC via CLF is the determination of CLF itself. Another point that should be mentioned is that currently there exists no adequate work for the optimal control of nonlinear systems. Especially the Hamilton-Jacobi- Bellman (HJB) equations related to the non-affine in control nonlinear systems are difficult to solve. The affine-in-input systems are preferred in optimal control studies because there is an explicit solution for the input as a function of derivatives of the value function when the aim is quadratic and dynamics are "affine-in-input". Thus, studies in IOC, as in optimal control problems, were generally carried out for affine-in-input nonlinear systems. In recent years, the inverse optimality approach has been increasingly used for solving the nonlinear optimal control problems in many real-time applications. The main theorem used in this thesis for IOC is related to the discrete-time affine-in-input nonlinear systems. The necessary conditions needed to construct a discrete quadratic Control Lyapunov Function CLF has been presented in establishing the control law. In this thesis, firstly, MPC method is discussed. Gradient-based classical MPC method for linear system models and Runge-Kutta-based MPC method proposed for nonlinear system models are explained in order to compare them with the method that is proposed in this thesis. Secondly, IOC approach is explained. The IOC approach that solves the problem for affine-in-input nonlinear systems and converts the IOC problem into the appropriate CLF finding problem is described. The IOC approach is explained for both the regulator and tracking cases. The problem that may be encountered in the tracking case is addressed. In this thesis, a sigmoid function that depends on the system state variables is proposed for each element of the control signal weight matrix in order to solve the problem that may arise in tracking case. A comparison with the classical IOC approach is made to show the success of the proposed approach using the coupled water tank level control problem. The performance criteria are selected as number of step, Sum of Suquared Error (SSE), Sum of Squared Error multiplied by Step (SSSE), Total Variation (TV), maximum input value. Another and more important contribution is that the thesis propose an optimal control method in which the MPC and IOC approaches are merged with each other for discrete-time affine-in-input nonlinear system models. Most of the nonlinear systems and system models encountered in real life and literature may be converted to affine-in-input form by some nonlinearity reduction procedures. Therefore, the proposed method has an intrinsic advantage over the classical MPC methods that may require linearization. The key feature in this approasach is to solve the IOC problem repeatedly for each receding horizon and consequently for a new initial condition. Thus, we use the system model to obtain the prediction of the future behavior of the system and apply the out-coming control signal, which has been obtained from IOC procedure for a pre-determined control horizon. In the solution phase of IOC, the parameters of the candidate control Lyapunov function matrix are estimated using the global evolutionary Big Bang-Big Crunch (BB-BC) optimization search algorithm in an on-line manner. The proposed control structure thus solves the optimal control problem in classical MPC approach to the search of an appropriate CLF matrix for each control horizon. From another perspective, MPC structure is inserted to IOC problem and thus, the IOC problem is solved repeatedly using different initial conditions at the beginning of each receding horizon. Therefore, IOC gains an on-line correction mechanism via this new approach. In order to understand the effectiveness of the proposed optimal control method it is compared with other control methods via simulations and a real time application done on Quanser ball and plate system. The control structures used to compare the performance of the proposed approach are gradient-based classical MPC method for linear system models, Runge-Kutta-based MPC for nonlinear systems, linear quadratic regulator and classical inverse optimal control structure. The performance criteria are selected as overshoot, settling time, rise time, integral square error (ISE), integral time square error (ITSE), integral absolute error (IAE), integral time square error (ITAE) and total variation (TV). The real-time application and simulations show that the proposed control structure provides better results in terms of almost all classical time domain criteria when compared with other related control methods.
The aim of optimal control problem is to form a control input function or control law such that a performance criterion is minimized while satisfying some control and state constraints. However, the optimal control law cannot be found quite easily and analytically even in the unconstrained and linear cases. It is a well-known fact that the solution for optimal control law requires solving Hamilton-Jacobi-Bellman (HJB) equation which is extremely difficult. Moreover, an analytical solution of HJB does not exist for most nonlinear systems. When the system is linear and the performance criterion is quadratic, HJB turns out to be a Riccati equation which may also be difficult to solve analytically for certain cases. One approach to overcome these difficulties is to solve the optimal control problem repeatedly and sequentially using the system model and assigning the current system state as the initial state for a predetermined finite horizon. The out-coming control sequence that corresponds to a predetermined control horizon is then applied to the real-time system. Control approaches using this strategy are referred to as Model Predictive Control (MPC). In this approach, the future behavior of the system is predicted using the system model and the control signal is renewed repeatedly by the current system states for each control horizon. The model of the system plays a crucial role in all MPC techniques. The methods in which the original nonlinear model is linearized around an operating point are referred to be Linear MPC methods. These methods have been used as effective and robust tools for controlling many industrial processes. However, the effect of linearization operation may degrade for the systems that possess high nonlinearity and for the operating conditions that require the whole operating region. Various Nonlinear MPC (NMPC) approaches are presented and their reviews have been given in literature. Modeling strategies for model building, model reduction and stability have still been active working topics for NMPC approaches. The computational burden and the need for an efficient optimization approach is another issue associated with the solution of the NMPC optimization problem. Although different techniques are proposed for MPC, the MPC loop approach repeated in all sampling intervals in all MPC methods is basically the same as the basis of MPC is based on the receding horizon principle. The repeated MPC loop processes for the horizon moving towards the future in each sampling interval can be summarized as follows: - Using the model of the system to be controlled, the future outputs or states are calculated depending on the past input, output and / or system state variables and the input values planned to be implemented in the future. - By solving a finite-horizon open-loop optimization problem that is determined to converge the system state variables or output to the desired reference value and to minimize the cost function, a set of future control signals are obtained. - As the control signal obtained along the control horizon, the first element of the sequence is applied to the system. This is the receding horizon strategy, a concept of predictive control. On the other hand, Inverse Optimal Control (IOC) theory provides us a different perspective for solving HJB problem. Therefore, IOC is an alternative approach to solve nonlinear optimal control problem while avoiding the tedious task of solving the HJB equation. The perspective of the inverse optimal control problem was put forward by Kalman (1964) in the early 1960s. Kalman stated that when a dynamic system and a feedback control law are given and the closed-loop system is asymptotically stable, the inverse problem is to seek the most general performance index for which this control law is optimal. In fact, the IOC can be seen as an approach rather than a methodology that perceives the optimal control problem from the opposite side. In IOC method when the controller is desired to be optimal according to stable and meaningful objective functions, the control Lyapunov function (CLF) based approaches are widely used. The formulation of CLF, which provides the design of an optimal feedback controller for typical classes of systems, has been discussed in the literature. The existence of CLF implies stabilizability. Thus, the distinguishing aspect of this approach is that the performance measure corresponding to the stabilizing feedback control is determined posteriori. Since there exists no explicit technique for the determination of CLF for general nonlinear systems, the most challenging aspect of IOC via CLF is the determination of CLF itself. Another point that should be mentioned is that currently there exists no adequate work for the optimal control of nonlinear systems. Especially the Hamilton-Jacobi- Bellman (HJB) equations related to the non-affine in control nonlinear systems are difficult to solve. The affine-in-input systems are preferred in optimal control studies because there is an explicit solution for the input as a function of derivatives of the value function when the aim is quadratic and dynamics are "affine-in-input". Thus, studies in IOC, as in optimal control problems, were generally carried out for affine-in-input nonlinear systems. In recent years, the inverse optimality approach has been increasingly used for solving the nonlinear optimal control problems in many real-time applications. The main theorem used in this thesis for IOC is related to the discrete-time affine-in-input nonlinear systems. The necessary conditions needed to construct a discrete quadratic Control Lyapunov Function CLF has been presented in establishing the control law. In this thesis, firstly, MPC method is discussed. Gradient-based classical MPC method for linear system models and Runge-Kutta-based MPC method proposed for nonlinear system models are explained in order to compare them with the method that is proposed in this thesis. Secondly, IOC approach is explained. The IOC approach that solves the problem for affine-in-input nonlinear systems and converts the IOC problem into the appropriate CLF finding problem is described. The IOC approach is explained for both the regulator and tracking cases. The problem that may be encountered in the tracking case is addressed. In this thesis, a sigmoid function that depends on the system state variables is proposed for each element of the control signal weight matrix in order to solve the problem that may arise in tracking case. A comparison with the classical IOC approach is made to show the success of the proposed approach using the coupled water tank level control problem. The performance criteria are selected as number of step, Sum of Suquared Error (SSE), Sum of Squared Error multiplied by Step (SSSE), Total Variation (TV), maximum input value. Another and more important contribution is that the thesis propose an optimal control method in which the MPC and IOC approaches are merged with each other for discrete-time affine-in-input nonlinear system models. Most of the nonlinear systems and system models encountered in real life and literature may be converted to affine-in-input form by some nonlinearity reduction procedures. Therefore, the proposed method has an intrinsic advantage over the classical MPC methods that may require linearization. The key feature in this approasach is to solve the IOC problem repeatedly for each receding horizon and consequently for a new initial condition. Thus, we use the system model to obtain the prediction of the future behavior of the system and apply the out-coming control signal, which has been obtained from IOC procedure for a pre-determined control horizon. In the solution phase of IOC, the parameters of the candidate control Lyapunov function matrix are estimated using the global evolutionary Big Bang-Big Crunch (BB-BC) optimization search algorithm in an on-line manner. The proposed control structure thus solves the optimal control problem in classical MPC approach to the search of an appropriate CLF matrix for each control horizon. From another perspective, MPC structure is inserted to IOC problem and thus, the IOC problem is solved repeatedly using different initial conditions at the beginning of each receding horizon. Therefore, IOC gains an on-line correction mechanism via this new approach. In order to understand the effectiveness of the proposed optimal control method it is compared with other control methods via simulations and a real time application done on Quanser ball and plate system. The control structures used to compare the performance of the proposed approach are gradient-based classical MPC method for linear system models, Runge-Kutta-based MPC for nonlinear systems, linear quadratic regulator and classical inverse optimal control structure. The performance criteria are selected as overshoot, settling time, rise time, integral square error (ISE), integral time square error (ITSE), integral absolute error (IAE), integral time square error (ITAE) and total variation (TV). The real-time application and simulations show that the proposed control structure provides better results in terms of almost all classical time domain criteria when compared with other related control methods.
Açıklama
Tez(Doktora) -- İstanbul Teknik Üniversitesi, Lisansüstü Eğitim Enstitüsü, 2021
Anahtar kelimeler
doğrusal olmayan kontrol teorisi,
nonlinear control theory,
optimal kontrol,
optimal control