InproTKs: A Toolkit for Incremental Situated Processing

Kennington, Casey; Kousidis, Spyros; Schlangen, David

doi:10.3115/v1/w14-4312

Cited by 11 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To extend INPROTK to handle situated, multimodal input, we used a recently extended version, INPROTK S [12]. The system we implemented does not represent what is generally termed as a dialogue system, however, we used the same modularisation as in more typical dialogue systems by using a dialogue management (DM) component that controls the system actions based on user actions.…”

Section: System Setupmentioning

confidence: 99%

Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation

Kennington

Kousidis

Baumann

et al. 2014

Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Self Cite

View full text Add to dashboard Cite

It is established that driver distraction is the result of sharing cognitive resources between the primary task (driving) and any other secondary task. In the case of holding conversations, a human passenger who is aware of the driving conditions can choose to interrupt his speech in situations potentially requiring more attention from the driver, but in-car information systems typically do not exhibit such sensitivity. We have designed and tested such a system in a driving simulation environment. Unlike other systems, our system delivers information via speech (calendar entries with scheduled meetings) but is able to react to signals from the environment to interrupt when the driver needs to be fully attentive to the driving task and subsequently resume its delivery. Distraction is measured by a secondary short-term memory task. In both tasks, drivers perform significantly worse when the system does not adapt its speech, while they perform equally well to control conditions (no concurrent task) when the system intelligently interrupts and resumes.

show abstract

Section: System Setupmentioning

confidence: 99%

Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation

Kennington

Kousidis

Baumann

et al. 2014

Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the communication with the dialog manger PaMini some kind of dialog acts needed to be sent. As explained in [10] only strings can be sent by these informers. We created a new custom data type dialog act for the communication via RSB.…”

Section: Inprotk Extensionsmentioning

confidence: 99%

“…Additionally to the processing modules, InproTK provides listener and informer modules [10] that connect the speech processing module with other system components, which send events over the network (generally via a specific middleware such as the Robotics Service Bus [22]). Listeners receive information from the network, combine this data into an IU and put them onto their right buffer.…”

Section: Inprotkmentioning

confidence: 99%

Towards Closed Feedback Loops in HRI

Carlmeyer

Schlangen

Wrede

2014

Proceedings of the 2014 Workshop on Multimodal, Multi-Party, Real-World Human-Robot Interaction

Self Cite

View full text Add to dashboard Cite

In this paper, we present a first step towards incremental processing for modeling asynchronous human-robot interactions, to allow closed feedback loops in HRI. We achieve this by combining the incremental natural language processing framework InproTK with the human-robot dialog manager PaMini, which is based on generic interaction patterns. This enables the robot to provide incremental feedback during interaction and allows the user to give online feedback and corrections. We provide a first realization scenario as a proof of concept for our approach.

show abstract

“…Such mechanisms are offered by the incremental unit (IU) framework for SDS (Schlangen and Skantze, 2011), which we apply here. Following Kennington et al (2014), the IU framework consists of a network of processing modules. A typical module takes input, performs some kind of processing on that data, and produces output.…”

Section: Incremental Dialoguementioning

confidence: 99%

Supporting Spoken Assistant Systems with a Graphical User Interface that Signals Incremental Understanding and Prediction State

Kennington

Schlangen

2016

Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Self Cite

View full text Add to dashboard Cite

Arguably, spoken dialogue systems are most often used not in hands/eyes-busy situations, but rather in settings where a graphical display is also available, such as a mobile phone. We explore the use of a graphical output modality for signalling incremental understanding and prediction state of the dialogue system. By visualising the current dialogue state and possible continuations of it as a simple tree, and allowing interaction with that visualisation (e.g., for confirmations or corrections), the system provides both feedback on past user actions and guidance on possible future ones, and it can span the continuum from slot filling to full prediction of user intent (such as GoogleNow). We evaluate our system with real users and report that they found the system intuitive and easy to use, and that incremental and adaptive settings enable users to accomplish more tasks.

show abstract

InproTKs: A Toolkit for Incremental Situated Processing

Cited by 11 publications

References 6 publications

Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation

Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation

Towards Closed Feedback Loops in HRI

Supporting Spoken Assistant Systems with a Graphical User Interface that Signals Incremental Understanding and Prediction State

Contact Info

Product

Resources

About