New approach allows artificial intelligence systems to teach each other

Reviewed by James Ives, M.Psych. (Editor)Aug 6 2019

The journal Physics of Life Reviews, which has one of the highest impact factors in the categories "Biology" and "Biophysics", has published an article entitled "Symphony of high-dimensional brain".

The authors of the article, world-famous scientists of Russian origin, including the chief researcher at the laboratory for advanced methods of multidimensional data analysis at Lobachevsky University, professor at the University of Leicester (UK) Alexander Gorban, the leading researcher at the laboratory for advanced methods of multidimensional data analysis at Lobachevsky University, professor at University of Leicester (UK) Ivan Tyukin, and the senior researcher at the laboratory of neural network technologies at Lobachevsky University, professor at the Complutense University (Madrid, Spain) Valery Makarov, sum up a broad discussion in the journal Physics of Life Reviews.

The discussion was focused on two problems:

• 1) How to quickly and effectively correct the errors of artificial intelligence (AI)?

  2) What is the reason why small neural ensembles in the multidimensional brain are so effective?

A unified approach to these problems was proposed in the article by the same authors, "The unreasonable effectiveness of small neural ensembles in high-dimensional brain", which was published earlier in the same journal and triggered a wide discussion. According to the authors, the heart of the matter is in the geometry of multidimensional spaces. One of the discussion participants, the famous expert in neuroscience R. Quian Quiroga, proposed to call the new approach using the authors' initials: the GMT approach.

As Alexander Gorban notes, data clouds acquire unexpectedly simple properties in multdimensional systems. Even in large random data sets, each point can be separated with a high degree of probability from all other points by a plane, which is constructed using explicit formulas.

This phenomenon of stochastic separation serves at the same time as the basis for the correction of artificial intelligence errors and for such neurophysiological phenomena as "grandmother or concept cells", individual neurons that selectively respond to the presentation of any image ("grandmother"). Artificial intelligence systems make mistakes and will continue to make mistakes. We must develop an effective technology for processing artificial intelligence errors or abandon the use of artificial intelligence in important projects,"

Alexander Gorban, Laboratory for Advanced Methods of Multidimensional Data Analysis, Lobachevsky University

According to the GMT approach, if an artificial intelligence error is detected, it can be corrected for the future by separating the situation with an error from other situations where there were no errors using a simple rule (a plane). It is easy to build cascades of such simple correction tools. They allow the errors of artificial intelligence to be corrected without retraining the original system, which would require incomparably more time and more resources of all types.

Also, "grandmothers or concept cells" can easily separate a specific multidimensional signal from all others signals without creating complex and essentially non-linear rules. New models of neural memory in the multidimensional brain were also proposed by professors Gorban, Makarov and Tyukin (GMT models).

The idea of the "curse of dimensionality", formulated by R.E. Bellman in 1957, is quite well known. In the framework of the GMT approach, the curse of dimensionality turns into a blessing of dimensionality. One should not be afraid of or avoid multiple dimensions: the trick is to use them properly.

The discussion of the new GMT approaches involved ten world-renowned experts. V. K?rková, President of the European Neural Network Society, focused on the need to revise the modern theory of machine learning, since data in the real world are not independent and are not equally distributed.

A. Tozzi and J.F. Peters propose their own approach to deal with the curse of dimensionality. It is based on a multidimensional topology, that is, on the most general geometric properties that are invariant with respect to any continuous transformations. R. Quian Quiroga compares the GMT approach with modern neuroscience concepts on sparse coding in the brain.

The problem of effective information exchange between modern neuroscience and artificial intelligence technology is discussed by several experts with varying degrees of optimism.

P. Varona notes that researchers may face similar problems in these areas, but artificial intelligence is still far from mastering the heterogeneous, robust and effective correction mechanisms that exist in the real brain.

On the other hand, C. van Leeuwen believes that the modern cybernetic approach to the brain is misleading, artificial intelligence models are ineffective in neuroscience, and a change of basic paradigms is required to achieve some success.

To describe the transition between low-dimensional models and the real brain, R. Barrio introduces two worlds, "brainland" and "flatland", and discusses the problem of the rational travel between them. The point of view of L. Fortuna, who studies the influence of non-linear processes of small dimension on the multidimensional brain, is close to the one of R. Barrio. G. Kreiman emphasizes that modern multidimensional geometry allows us to reduce multidimensionality to low-dimensional systems.

V. Kreinovich uses the famous lines from Boris Pasternak's poem to evaluate the GMT approach, which reveals the unexpected simplicity of very complex things:

Assured of kinship with all things
And with the future closely knit
We can't but fall - a heresy! -
To unbelievable simplicity.
But to be spared we can't expect
If we do not conceal it closely.
Men need it more than anything,
But complex things are easier for them...

The result of this discussion is as follows: the GMT approach has proven its value. Proposed models have already been confirmed by real applications. The new error correction technology really works: it corrects artificial intelligence errors on the fly and allows artificial intelligence systems to train each other. New GMT memory models allow for experimental verification and already give some unexpected predictions. In the experts' discussion, new issues arise and horizons for further work open up.