A Cognitive User Interface for a Multi-modal Human-Machine Interaction

Tschöpe, Constanze; Duckhorn, Frank; Huber, Markus; Meyer, Werner; Wolff, Matthias

doi:10.1007/978-3-319-99579-3_72

Cited by 4 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also have evidence that it works for German and French as well and hopefully for other languages also. Our approach will open up an entirely new avenue for the further development of speech-controlled cognitive user interfaces [6,58,64].…”

Section: Discussionmentioning

confidence: 99%

“…One particular demand on cognitive user interfaces are the processing and understanding of declarative or imperative sentences. Consider, e.g., a speechcontrolled heating device, a cognitive heating [15,29,58,62,63], with the operator's utterance "I am cold!" This declarative sentence must firstly be analyzed syntactically to assigning "I" to the subject position and attributing "am cold" to the predicate.…”

Section: Introductionmentioning

confidence: 99%

“…From this representation of the speaker's state, the system must compute logical inferences and then respond accordingly, by increasing the room temperature and probably by giving a linguistic feedback signal "I increase the temperature to 22 degrees". Technically, this could be achieved using feature-value relations (FVR) [62] as semantic representations and modified Markov-decision processes (MDP) for behavior control [58,62,63].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reinforcement learning of minimalist grammars

Graben,

Römer,

Meyer

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Speech-controlled user interfaces facilitate the operation of devices and household functions to laymen. State-of-the-art language technology scans the acoustically analyzed speech signal for relevant keywords that are subsequently inserted into semantic slots to interpret the user's intent. In order to develop proper cognitive information and communication technologies, simple slot-filling should be replaced by utterance meaning transducers (UMT) that are based on semantic parsers and a mental lexicon, comprising syntactic, phonetic and semantic features of the language under consideration. This lexicon must be acquired by a cognitive agent during interaction with its users. We outline a reinforcement learning algorithm for the acquisition of syntax and semantics of English utterances, based on minimalist grammar (MG), a recent computational implementation of generative linguistics. English declarative sentences are presented to the agent by a teacher in form of utterance meaning pairs (UMP) where the meanings are encoded as formulas of predicate logic. Since MG codifies universal linguistic competence through inference rules, thereby separating innate linguistic knowledge from the contingently acquired lexicon, our approach unifies generative grammar and reinforcement learning, hence potentially resolving the still pending Chomsky-Skinner controversy.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement learning of minimalist grammars

Graben,

Römer,

Meyer

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using attribute-value logics [42] and its associated term algebra, it should be possible to encode the semantics of arbitrary utterances in a compositional fashion. This will open up an entirely new avenue for the further development of speech-controlled cognitive user interfaces [8].…”

Section: Discussionmentioning

confidence: 99%

“…One particular demand on cognitive user interfaces are the processing and understanding of numerals, e.g. in instructions like "increase the heating to 22.5 degrees", where the device may probably respond with a sensor registration: "the current room temperature is 18.3 degrees" [8]. Numerals are an important research domain in cognitive linguistics and language technology [9]- [14].…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning of Minimalist Numeral Grammars

Graben¹,

Römer²,

Meyer³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Speech-controlled user interfaces facilitate the operation of devices and household functions to laymen. State-of-the-art language technology scans the acoustically analyzed speech signal for relevant keywords that are subsequently inserted into semantic slots to interpret the user's intent. In order to develop proper cognitive information and communication technologies, simple slot-filling should be replaced by utterance meaning transducers (UMT) that are based on semantic parsers and a mental lexicon, comprising syntactic, phonetic and semantic features of the language under consideration. This lexicon must be acquired by a cognitive agent during interaction with its users. We outline a reinforcement learning algorithm for the acquisition of the syntactic morphology and arithmetic semantics of English numerals, based on minimalist grammar (MG), a recent computational implementation of generative linguistics. Number words are presented to the agent by a teacher in form of utterance meaning pairs (UMP) where the meanings are encoded as arithmetic terms from a suitable term algebra. Since MG encodes universal linguistic competence through inference rules, thereby separating innate linguistic knowledge from the contingently acquired lexicon, our approach unifies generative grammar and reinforcement learning, hence potentially resolving the still pending Chomsky-Skinner controversy.

show abstract