The neural processes that underlie your ability to read and understand this sentence are unknown. Sentence comprehension occurs very rapidly, and can only be understood at a mechanistic level by discovering the precise sequence of underlying computational and neural events. However, we have no continuous and online neural measure of sentence processing with high spatial and temporal resolution. Here we report just such a measure: intracranial recordings from the surface of the human brain show that neural activity, indexed by γ-power, increases monotonically over the course of a sentence as people read it. This steady increase in activity is absent when people read and remember nonword-lists, despite the higher cognitive demand entailed, ruling out accounts in terms of generic attention, working memory, and cognitive load. Response increases are lower for sentence structure without meaning ("Jabberwocky" sentences) and word meaning without sentence structure (word-lists), showing that this effect is not explained by responses to syntax or word meaning alone. Instead, the full effect is found only for sentences, implicating compositional processes of sentence understanding, a striking and unique feature of human language not shared with animal communication systems. This work opens up new avenues for investigating the sequence of neural events that underlie the construction of linguistic meaning.ow does a sequence of sounds emerging from one person's mouth create a complex meaning in another person's mind? Although we have long known where language is processed in the brain (1-3), we still know almost nothing about how neural circuits extract and represent the meaning of a sentence. A powerful method for addressing this question is intracranial recording of neural activity directly from the cortical surface in neurosurgery patients (i.e., electrocorticography or ECoG) (4, 5). Although opportunities for ECoG data collection are rare, determined by clinical-not scientific-priorities, they nonetheless offer an unparalleled combination of spatial and temporal resolution, and further provide direct measures of actual neural activity, rather than indirect measures via blood flow (as in PET, fMRI, and near infrared spectroscopy/optical imaging). ECoG data are particularly valuable for the study of uniquely human functions like language, where animal models are inadequate. Here we used ECoG to identify the neural events that occur online as the meaning of a sentence is extracted and represented.Prior intracranial recording studies of language have largely focused on speech perception and production (e.g., refs. 6-11) and word-level processes (e.g., refs. 12-26). However, the most distinctive feature of human language is its compositionality: the ability to create and understand complex meanings from novel combinations of words structured into phrases and sentences (27). As a first step toward understanding the neural basis of sentence comprehension, we recorded intracranial responses while participants read sentences and...

It has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production by identifying those cortical regions that hold substantial information about individual phones. In conclusion, the Brain-To-Text system described in this paper represents an important step toward human-machine communication based on imagined speech.