# A mess in an ideal world. Part 3

In this series of articles I try to make visible and tangible some elements of the chaos theory that I studied professionally several years ago. In the previous parts of this series (Part 1, Part 2) we saw how a strange attractor is born in an extremely simple Hamiltonian system – a ball bouncing on a spring-loaded table. This system is capable of generating both beautiful pictures and beautiful explanations for these pictures. Now we will look at some qualitative and quantitative characteristics of strange attractors.

I had to take a long break between posts, so I recommend brushing up on the previous parts of the series, plus there are some nice pictures and animations (save your traffic!).

## Lyapunov exponents

The main qualitative property of a strange attractor and the associated dynamic chaos is *phase space mixing* and the associated exponentially growing distance between close points. Let’s look at this phenomenon by plotting the dependence of the distance between any two points on the number of iterations of the Poincaré map of our system for three types of orbits: a fixed point (a pole of the fifth order, which is shown by green dots), an invariant torus (a red closed orbit) and a strange attractor (fog of blue dots).

We should not be confused by the fact that around a fixed point the distance between neighbors also changes: in our experiment, when choosing neighbors, we inevitably miss the pole. This leads to periodic fluctuations in the distance, but it does not increase over time.

In the vicinity of an invariant torus, neighboring points also gradually diverge. This is due to the fact that each torus has its own rotation number and neighboring points, once on neighboring tori, will move at different speeds and move away from each other. Moreover, the rate of removal will not be exponential, but close to linear, which we observe on the graph in logarithmic coordinates, like some kind of logarithmic curve.

Finally, the neighborhoods of strange attractor points behave in the same way as the neighborhoods of saddle points: they stretch and contract, following a geometric progression. This stretching reflects the positive slope of the distance growth line. In logarithmic coordinates, an exponentially growing curve appears straight, and its slope reflects the rate of growth. It is possible to average the rate of retreat of neighboring points over the entire orbit, thus obtaining a characteristic not of a single point, but of the entire orbit as a whole.

To study the dynamics of the system in the vicinity of fixed points, we used the Jacobian matrix – an analogue of the derivative for a vector function, namely, we calculated the eigenvalues of this matrix. As Samuel Smale showed, a strange attractor consists of an infinite number of fixed points of different orders, but they all turn out to be unstable. To characterize not individual fixed points, but the entire attractor as a whole, a modified spectrum is used, consisting of numbers called **Lyapunov exponents**. The number of indicators coincides with the dimension of the phase space and each of them corresponds to the rate of exponential expansion of trajectories along each of the generalized coordinates. However, it makes sense to calculate this speed not along the coordinates, but along those directions along which the change in distances between points will be maximum. Such numbers are calculated on the basis of the Jacobian, using a relatively simple algorithm that not only averages the speed of divergence of neighboring points along the trajectory, but also the proper directions of this divergence. Those averaged slopes of the graphs for the distance between neighboring points versus the number of iterations that we saw correspond to **maximum Lyapunov exponents**. Over time, these indicators will dominate and be observed in the dynamics of the system.

And there is one nuance here. In dissipative non-autonomous systems (in which there are energy losses and external energy replenishment), Lyapunov spectra of stable trajectories: fixed points and limit cycles, have only negative exponents. The presence of at least one positive indicator in the trajectory spectrum suggests that this trajectory may have chaotic properties. However, our system is conservative, which leads to the fact that all mappings strictly preserve the phase volume, deforming it, but leaving its measure unchanged. This means that Lyapunov exponents in Hamiltonian systems must appear in pairs: . Consequently, in systems like ours, positive exponents are inevitable for any non-trivial trajectories; the only difference is in the magnitude of the positive exponent. Our previous experiment showed that the indicators for different types of orbits differ significantly.

Here is how the maximum Lyapunov exponent (LLE), calculated for different trajectories, changes as the energy of the system changes.

Now it is quite obvious that the poles are extremely stable, the invariant tori have very small values of the maximum Lyapunov exponent, and chaos is sharply manifested by an increase in the maximum exponent.

Lyapunov exponents have a dimension that is the inverse of time or the number of iterations for a discrete display. This allows us to consider them as a characteristic rate of forgetting by the system of the initial state or as the rate of loss of information in the system. The reciprocal of the maximum Lyapunov exponent is called Lyapunov time. It can be calculated for many natural processes, obtaining information about in what time intervals it makes sense to predict the behavior of the system based on the observed state parameters. So, for example, for the solar system this time is 5 million years, and for weather on Earth it is on the order of a couple of weeks. As you can see, our bouncing ball is capable of forgetting its initial conditions in just 5–10 steps.

We considered Lyapunov exponents as characteristics of a dynamical system and its trajectories or orbits. But aren’t they, first of all, characteristics *computing processes*, which we used to model the dynamics of our system? In other words, instead of the system, aren’t we studying Newton’s method, with the help of which the moments of collisions are calculated?

This is how the estimate of the maximum Lyapunov exponent changes when using different computational accuracy.

This experiment shows that our mathematical model is fully consistent with the described phenomenon.

## Lyapunov fractal

Lyapunov exponents allow us to see on one diagram two mechanisms leading to the appearance of strange attractors: the destruction of hetero- and homoclinic orbits, as well as a cascade of period doubling bifurcations. Our system is very simple (that’s why I chose it as an example); to fully describe all its dynamics, only two parameters are enough: the energy of the system and the starting point of the orbit. Thus, the maximum Lyapunov exponent allows us to see the dynamics of the system in the space of all its parameters , and distinguish between poles, invariant tori and strange attractors. So on one map you can see all the dynamic processes of the system and scenarios of transition to chaos. Here she is:

The blue color in this diagram corresponds to stable orbits: the poles and their surroundings. Blue color – invariant tori. Finally, the gray and brown areas represent strange attractors. Let’s take a closer look at several parts of this diagram.

Notice that within the gray-brown chaotic area some structures of its own appear: islands of stability and what geologists or petrographers would call *fluid texture*, similar to figures of flow or plastic deformation. This suggests that a strange attractor does not just generate a random set of points; its disorder has its own internal structure, which smoothly and continuously changes with changes in the parameters of the system.

#### The birth of resonances

The first thing that catches your eye in the diagram is a deep, stable trench generated by a fixed point. . As the energy increases, long blue “claws” of poles of higher orders extend from it. This is what the first appearance of chaos in the system looks like:

In the mutual arrangement of the poles, one can easily read the structure of the set of rational numbers (the Stern-Brocaut tree), which corresponds to the structure of the rotation numbers of the resonances.

From each pole extends a thin, gradually expanding unstable “ridge” generated by the heteroclinic orbits of the saddle points. They are marked with letters on the diagram , indicating the corresponding saddle points. The orbits connecting the saddle points, when destroyed, give rise to strange attractors that gradually expand, demonstrating the phenomenon of Arnold diffusion.

Lines of poles of small order cut deeply into the chaotic region, creating bays in it, which in a strange attractor we see as islands of order in a sea of chaos. An increase in energy leads to the fact that the poles also begin to generate resonances of higher orders. This gives the bifurcation diagram self-similarity and a fractal structure.

The region of resonance birth shows a very interesting picture . Its birth is preceded by growing disturbance of the pole , and the appearance of many high-order resonances. Simultaneously with the pole a saddle point is born , the heteroclinic manifold of which immediately collapses and gives rise to a diverging chaotic region. However, it now becomes clear that Arnold diffusion occurs due to the appearance and disappearance of many poles and saddles of a very high order surrounding the saddle point.

#### Period doubling

Finally, in those places where the bays of stability end, giving way to chaos, a cascade of resonance births occurs, ending with a doubling of the pole period. This is how the pole “dissolves” in a strange attractor. An archetypal case demonstrates the pole :

At the bifurcation point the fixed point turns from pole to saddle and generates a pair of new second-order poles . Homoclinic orbit connecting point collapses with itself, generating a strange attractor that absorbs poles of a higher order. The poles repeat the same fate , as shown inside the highlighted rectangle. The new poles soon experience period doubling again (an even smaller rectangle) and are very quickly swallowed up by chaos. The same scenario is observed at all scales for all poles. Even in a small ordered island, which is generated by the pole we again observe the same series of bifurcations.

We used the prime meridian on the spherical surface of the constant energy of the system to construct the diagram. Due to physical symmetry, all fixed points must be located on it. However, the regions of invariant tori extend far beyond the prime meridian and are capable of deforming and crossing it again, which is what we see as islands of order suddenly appearing.

Finally, I will classically demonstrate the self-similarity of the bifurcation diagram, increasing the approximation to the point at which the period doubling cascade leads to global chaos.

## Large animated picture

## Spectral characteristics

When discussing scenarios of transition to chaos, we repeatedly used the frequency characteristics of trajectories and orbits: periods of fixed points, rotation numbers of invariant tori, and transition to chaos through a sequence of period doubling bifurcations. We analyzed all these things while working in phase space, which practically does not contain time dependencies. Now it makes sense to take a look at Fourier transforms for time series. As with the Lyapunov exponents, let us first take a look at the spectra of individual types of trajectories: poles of different orders, invariant tori and a strange attractor.

The frequencies in these spectra correspond to the rotation numbers, which is what we see as peaks in the spectra. Pole expectedly represented by multiple peaks with frequencies of 1/5 and 2/5. The spectrum of the invariant torus has not only peaks around 1/4, but also a whole “comb” of secondary peaks, which corresponds to a very high order resonance.

The spectrum of a strange attractor is very different from the spectra of regular orbits in that it *solid*but not *ruled* and contains all frequencies from low to high, approaching noise. However, against the background of this noise, peaks stand out with frequencies that have small denominators, which correspond to the first few layers of the Stern-Brocot tree (we talked about this when discussing resonances). I wonder how universal these properties of strange attractors are: a continuous spectrum and the presence of rational frequencies with small denominators?

By combining the spectra for individual trajectories, we obtain a spectrogram for the entire family of orbits of the system at a given energy. So we are able to look at all the strange attractors for a given energy.

First of all, let’s pay attention to how, at low energies, resonances are formed at the points of intersection of lines corresponding to rotation numbers with small denominators. The same picture can be observed in a geometric demonstration of the structure of the Stern-Brocot tree, which is called **Farey diagram**. Compare the left half of this diagram with the main frequencies that stand out in the spectra.

In the left half of this tree we see exactly the same fractions as in the spectrograms. This is not surprising, since the structure of the resonances exactly corresponds to the internal structure of the set of rational numbers, which is reflected by the Farey diagram. We talked about this in the last part. Around the resonances in the spectrogram we observe a secondary structure: a pronounced resonance peak surrounded by high-frequency peaks of invariant tori.

What happens in the chaotic sea? In the smeared noisy spectrum, dark bands of those resonances stand out that gave rise to this strange attractor through the destruction of homoclinic orbits. A chaotic orbit, wandering through phase space, continually approaches fixed points of different orders and for some time falls into their rhythm. However, in the last article we concluded that a chaotic orbit “forgets” about its past in Lyapunov time, which in our system is only 5-10 steps. This means that in the spectrum of a strange attractor we should no longer observe peaks with periods greater than 10, since a chaotic orbit will not be able to stay in a small vicinity of a fixed point for such a long time.

The spectrum of a strange attractor contains both low and high frequencies. The former appear due to fluctuations and mixing of the phase space, and the latter – due to the exponential divergence of neighboring points. A continuous spectrum indicates the stochasticity of the attractor, which makes short-term observation of the orbit of any period approximately equally probable. This is perfectly demonstrated by Smale’s result, mentioned several times, which showed that the strange attractor generated by the destruction of heteroclinic manifolds contains an infinite number of fixed points of very different orders. In addition, the period doubling cascade also leads to the appearance of an infinite set of frequencies in the system.

## Dimension of orbits

What else distinguishes a strange attractor from an invariant torus or a chain of fixed points? They look different in appearance. A chaotic orbit forms a “fog” that fills a certain area of phase space, while an invariant torus in the Poincaré section forms a closed line or a set of ovals, and fixed points (cycles) form a discrete finite set of points. From a formal point of view, these orbits, as topological objects, differ **dimension**.

This characteristic can be defined in various ways, depending on the task. To characterize discrete sets of points, it is most often used **Hausdorff dimension **or approaching it **correlation dimension**. It is the latter that we will consider in more detail, since it is easily calculated and allows us to estimate not only the dimension of an object, but a whole spectrum of dimensions for various scales in the metered phase space.

What distinguishes a one-dimensional line from a discrete set of zero-dimensional points? First of all, continuity. However, if there are many points, they can be formed into discrete shapes that will be more similar to a zero-dimensional, one-dimensional or two-dimensional object, as shown in the figure.

What distinguishes them is the distribution of the number of points over the distances between them, or more precisely, how the number of points falling into the disk depends on its radius. This distribution can be estimated in the following very simple way. For a set of n points, we find the distances between all points (if there are many points, then between some random sample that forms a subset of points). An ordered set of distances defines a function Where — the number of pairs of points at a distance is less from each other. Normalizing this function by the number of pairs: , we obtain the observed probability distribution for the entire range of distances. The range of distances and the number of pairs are very large, so we will use a logarithmic scale, which allows us to relate values of different orders to each other.

Let’s see what this distribution looks like for some sets of 500 points: several very dense clusters, a one-dimensional line, a square randomly filled with points, and a Sierpinski set.

Pay attention to the distinct linear sections in the distribution graphs. Linear graphs in logarithmic coordinates correspond to a power law, and the slope of the line shows its power. This means that on some scales the number of points falling into the disk depends on the radius of the disk like this:

This indicator is called the correlation dimension of a set of points in a certain range of distances and gives an estimate for the Hausdorff dimension, which is calculated in a more complex way.

From our examples it is clear that points lying on a circle, as expected, have a dimension close to one, however, on small scales its dimension drops to zero. A regular lattice and a randomly filled disk have a dimension close to two, and the Sierpinski triangle has a fractional dimension . Sets with fractional dimensions are called fractal. They may be self-similar, like the Sierpinski set, or not, it does not matter.

Please note that the set of points forming several compact clusters has a dimension close to two on small scales, but on the scale of distances between clusters, the dimensionality drops, showing that the clusters themselves are two-dimensional, but they form a discrete set of zero dimension.

You can plot the average slopes of the dependence in logarithmic coordinates, and obtain the spectrum of dimensions of a set of points. Examples of such spectra for the sets described above are shown in the figure:

Let’s return to our system. Spectra of the correlation dimension will allow us to obtain a more detailed quantitative topological picture of the invariant manifolds of its Poincaré section.

It must be said that the correlation dimension is not the most informative characteristic of Hamiltonian systems. Unlike dispatchative systems, Hamiltonian strange attractors, as points accumulate, do not reveal any fractal structure, and uniformly fill the phase space, demonstrating one of their key properties – ergodicity.

Dimensions are good for diagnosing a strange attractor, but Lyapunov exponents and frequency spectra provide more information. However, we can construct nice dimensional spectra for families of system orbits at a fixed energy.