What should an ordinary person understand about the work of generative artificial intelligence? Proceedings of the competition "TRIZformashka-2024"

Abstract

Three basic aspects of the operation of generative neural network models (generative artificial intelligence) are discussed in the article: the concept of a "token", the probabilistic nature of the generated response, and the concept of a "large model", the size of which ensures the pseudo-intelligent behavior of neural network chatbots. The issues of implementing generative models, areas and methods of their application are not discussed in principle.The materials of the "TRIZformashka-2024" competition, which was dedicated to neural network models, are provided. The fact of pseudo-intelligence of generative models is demonstrated. It turns out that a model trained on a single phrase "mama myla ramu" ("mama washed the frame") and using a context of a single letter for generation can sometimes behave as if it knows the rules of declension in the Russian language and is able to change a word by case!The concept of a "token" is considered in relation to the generation of texts, pictures and passwords. On the basis of "tokens" a practically useful method of generating passwords is built, difficult to guess, but easy to reproduce (difficult to forget).The concept of a "large model" is presented clearly and intelligibly due to its "visualization" by comparing it with physical quantities. (If one parameter of a neural network weighed one gram, then 200 freight trains would be needed to transport it. If one parameter had a length one millimeter, then the neural network would revolve around the Earth at the equator 25 times. If one second was required to learn one parameter, then it would be necessary to start training a modern neural network in the times of the Cro-Magnons.)The materials will be useful for studying generative artificial intelligence at any age.