Vortex, the key to information processing capability: Virtual physical reservoir computing

[Background] In recent years, physical reservoir computing*1), one of the new information processing technologies, has attracted much attention. This is a physical implementation version of reservoir computing, which is a learning method derived from recurrent neural network (RNN)*2) theory. It implements computation by regarding the physical system as a huge RNN, outsourcing the main operations to the dynamics of the physical system that forms the physical reservoir. It has the advantage of obtaining optimization instantaneously with limited computational resources by adjusting linear and static readout weightings between the output and a physical reservoir without requiring optimization of the weightings by back propagation. However, since the information processing capability depends on the physical reservoir capacity, it is important that this is investigated and optimized. Furthermore, when designing a physical reservoir with high information processing capability, it is expected that the experimental cost will be reduced by numerical simulation. Well-known examples of physical reservoir computing include its application to soft materials, photonics, spintronics, and quanta, while in recent years, much attention has been paid to waves; neuromorphic devices that simulate functions of the brain by using non-linear waves have been proposed (see references 1-3). The fluid flow of water, air, etc. represents a physical system that is familiar but shows various and complicated patterns that have been thought to have high information processing capability. However, virtual physical reservoir computing using numerical simulation or investigation of information processing capability of fluid flow phenomena has not been realized due to its relatively high numerical computational cost. Therefore, the relationship between the flow vortex and information processing capability remained unknown.

[Results] In this study, Prof. Hirofumi Notsu and a graduate student of Kanazawa University in collaboration with Prof. Kohei Nakajima of the University of Tokyo investigated fluid flow phenomena as a physical system, especially the fluid flow that occurs around a cylinder, which is well understood. It is known that this physical system is governed by the incompressible Navier-Stokes equations*3), which describe fluid flow, and also includes the Reynolds number*4), a parameter indicative of the system characteristics. This physical system was virtually implemented by spatial two-dimensional numerical simulation using the stabilized Lagrange-Galerkin method*5), and the dynamics of flow velocity and pressure at the selected points in the downstream region of the cylinder were used as the physical reservoir. The information processing capability was evaluated using the NARMA model*6) (see Figure). It is known that in the flow of fluid around a cylinder, as the Reynolds number value increases, twin vortices formed in the downstream region of the cylinder gradually become larger and eventually form a Karman vortex street, the alternate shedding of vortices. In this study, it was clarified that at the Reynolds number where the twin vortices are maximal but just before the transition to a Karman vortex street, the information processing capability is the highest. In other words, before the transition to a Karman vortex street, the information processing capability increases as the size of the twin vortices increases. On the other hand, since the echo state property*7) that guarantees the reproducibility of the reservoir computing cannot be maintained when the transition to the Karman vortex street takes place, it becomes clear that the Karman vortex street cannot be used for computing.

[Future prospects] It is expected that these findings concerning fluid flow vortices and information processing capability will be useful when, in future, the information processing capability of the physical reservoir can be expanded using fluid flow, e.g. in the development of wave-based neuromorphic devices recently reported. Although the numerical computational cost of fluid flow phenomena is relatively high, this study has made it possible to handle macroscopic vortices that are physically easy to understand and has clarified the relationship between vortices and information processing capabilities by virtually implementing physical reservoir computing with spatial two-dimensional numerical simulation. Virtual physical reservoir computing, which used to be applied to a relatively large number of physical systems described as one-dimensional systems, has been expanded to include physical systems with two or more spatial dimensions. It is expected that the results of this study will allow the virtual investigation of the information processing capabilities of a wider range of physical systems. In addition, since it is revealed that vortices are the key to information processing capability, it is expected that research for creating or maintaining vortices will be further promoted.

INFORMATION:

[Glossary] *1) Physical reservoir computing
A physical implementation version of reservoir computing, which is a type of learning method for recurrent neural networks (RNN). The dynamics of a physical system (physical reservoir) is regarded as a huge RNN, with which major computing is performed. Since it outsources operations, it has the advantage of being able to obtain optimization instantaneously with limited computational resources.

*2) Recurrent neural network (RNN)
A recurrent neural network is a neural network in which the output of the middle layers is also the input of itself or another layer.

*3) Incompressible Navier-Stokes equations
Incompressible Navier-Stokes equations are partial differential equations concerning the velocity and pressure of fluid flow in which the material density is constant.

*4) Reynolds number
Reynolds number (Re) is a dimensionless number that represents the ratio of the inertial force and the viscous force of the flow where ν is the kinematic viscosity of the fluid (m2/s), U is the flow speed (m/s), and L is the characteristic linear dimension (m), Re = UL/ν.

*5) Stabilized Lagrange-Galerkin method
A numerical solution of the finite element method. An implicit finite element method in which Lagrangian coordinates are used for the inertial term, piecewise linear elements are used to approximate the fluid flow velocity and pressure for the incompressible Navier-Stokes equations, and pressure stabilization is applied. It is characterized by robustness with respect to convection, relatively small numerical diffusion, and a symmetric matrix.

*6) NARMA model
NARMA (Nonlinear Autoregressive Moving Average) model, a non-linear time series model with inputs. In NARMA2, the next value is determined by the present and previous time values and input values. In NARMA3, it also depends on the value at the pre-previous time and input values.

*7) Echo state property
The property that the internal state of the current reservoir is expressed as a function that depends only on past inputs (it does not depend on the initial value). As a result, reservoirs with this property will eventually synchronize if the same inputs are given continuously, even if they start from different states.

[References] 1. Marcucci, G., Pierangeli, D., Conti, C., Theory of neuromorphic computing by waves: Machine learning by Rogue Waves, Dispersive Shocks, and Solitons, Phys. Rev. Lett., 125, 093901 (2020).
2. Silva, N. A., Ferreira, T. D., Guerreiro, A., Reservoir computing with solitons, New Journal of Physics, 23, 023013 (2021).
3. Hughes, T. W., Williamson, I. A., Minkov, M., Fan, S. Wave physics as an analog recurrent neural network. Science Advances, 5, eaay6946 (2019).