Long Short-Term Memory based reference model method to nonlinear dynamic system identification

Abstract

The relevance of this research stems from the fact that modern engineering problems and intelligent control systems require highly efficient mathematical tools for modeling complex nonlinear dynamic systems, capable of resolving the fundamental trade-off between ensuring approximation accuracy and minimizing computational costs and model training time. Traditional approaches based on neural networks with time delays are limited by a fixed memory window size, which makes it impossible to adequately describe processes with deep delays and non-stationary modes without a significant increase in the number of parameters. The aim of this work is to reduce the time required to construct models of nonlinear dynamics while ensuring a specified modeling accuracy by developing the reference model method through the use of neural network architecture with long-term short-term memory. To achieve this goal, the following tasks are set: to carry out a theoretical development of the reference model method through a transition to dynamic recurrent structures, to develop an algorithm for parameter alignment of reference models with long short-term memory to overcome permutation invariance, and to perform experimental verification of the proposed approach. The research methods are based on system identification theory, transfer learning methodologies, and model fusion. To eliminate the permutation symmetry of hidden layers, a weight tuning optimization algorithm based on solving linear programming problems has been adapted, which allows independent parameter matrices to be transferred to a common loss function pool before their superposition. The results of the work include the formalization of a reference model method based on long short-term memory and an algorithm for parameter alignment of these models, which ensures linear connectivity of modes in parameter space. Experimental verification on a test nonlinear system demonstrated that the proposed method accelerates model convergence by nearly a factor of 2 compared to an approach based on neural networks with time delays and reduces the final error by a factor of 3.3. The obtained results confirm the effectiveness of using recurrent architectures in the model superposition process. The scientific novelty of the work lies in the development of a reference model method based on long short-term memory, which allows overcoming the limitations inherent in architectures with time delays associated with the finiteness of the memory window. This ensures adequate modeling of systems characterized by deep delay, significant inertia, and complex nonlinear hysteresis effects. Furthermore, to address the problem of permutation invariance of hidden states in independently trained recurrent networks – which traditionally precludes direct averaging of model weight parameters – this work adapts a specialized weight-tuning optimization algorithm to the structure of long short-term memory.