Humans as next-token predictors: Measuring the flow of memes through minds

Abstract

I develop a simple but principled method for measuring the amount of culturally-transmitted information from a written target work that is actually retained in human minds and capable of influencing behavior. Using procedures inspired by Claude Shannon’s 1951 method for estimating the entropy of written English, I estimate the entropy of samples from the target work with a treatment group (those that have read a target work) and a control group (those who are of the same culture but who have not read the target work), using human minds as encoders-decoders in the communication model. KL divergence quantifies the information that the treatment group already knows relative to the control group. This method controls for shared cultural inheritance and does not require commitments to what information from the target work is important. The general technique can be profitably extended to a variety of domains, including evolutionary theory, methods of teaching, and the study of music.