An Embodied Cognitive Assistant for Visualizing and Analyzing Exoplanet Data

Kephart, Jeffrey O.; Dibia, Victor; Ellis, Jason; Srivastava, Biplav; Talamadupula, Kartik; Dholakia, Mishal

doi:10.1109/mic.2019.2906528

Cited by 9 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another is to extend the pointing capabilities to serve multiple users sharing a common workspace. The cognitive applications that we are building (Kephart et al 2018;Farrell et al 2016) can run in a wide range of physical environments, from laptops to large cognitive rooms. While technologies such as HTC Vive and Remote Mouse facilitate interactions in large-scale environments, leveraging standard mouse event capture capabilities in Reagent currently limits it to a single user.…”

Section: Discussionmentioning

confidence: 99%

Reagent: Converting Ordinary Webpages into Interactive Software Agents

Peveler

Kephart

2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

We introduce Reagent, a technology that readily converts ordinary webpages containing structured data into software agents with which one can interact naturally, via a combination of speech and pointing. Previous efforts to make webpage content manipulable by third-party software components in browsers or desktop applications have generally relied upon specialized instrumentation included in the webpages -a practice that neither scales well nor applies to preexisting webpages. In contrast, Reagent automatically captures semantic details and semantically-meaningful mouse events from arbitrary webpages that contain no pre-existing special instrumentation. Reagent combines these events with text transcriptions of user speech to derive and execute parameterized commands representing human intent. Thus, users may request various visualization or analytic operations to be performed on data displayed on a page by speaking to it and/or pointing to elements within it. When unable to infer translations between event labels and human terminology, Reagent proactively asks users for definitions and adds them to its dictionary. We demonstrate Reagent in the context of a collection of pre-existing webpages that contain football team and player statistics.

show abstract

Section: Discussionmentioning

confidence: 99%

Reagent: Converting Ordinary Webpages into Interactive Software Agents

Peveler

Kephart

2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…6. A plan from the Exoplanet Exploration domain, taken with permission from [19]. The nodes are as follows: dark green -calculated variable; gray -function; and orange -retrieved from database.…”

Section: Orchestratementioning

confidence: 99%

“…The first deployment that we describe is the use of a smart assistant to help astrophysicists visualize and analyze data about extra-solar (exo) planets, which are planets outside the traditional Solar System. This assistant -described more fully in a report to appear early next year [19] -enables subject matter experts (in this case, astrophysicists) to interact with data, visualizations, and analyses about exoplanets. The exoplanet assistant provides various capabilities including deixis using speech and gestures; proactive recognition and guidance; interactive query refinement; symbolic model discovery; and explainable self-programming using automated planning techniques.…”

Section: Exoplanet Explorationmentioning

confidence: 99%

See 1 more Smart Citation

Planning and visualization for a smart meeting room assistant1

Chakraborti

Fadnis

Talamadupula

et al. 2019

AIC

Self Cite

View full text Add to dashboard Cite

In this paper, we report on the planning and visualization capabilities of Mr.Jones -a proactive orchestrator and decision-support agent for a collaborative decision making setting embodied by a smart room. The duties of such an agent may range across interactive problem solving with other agents in the environment, developing automated summaries of meetings, visualization of the internal decision-making process, proactive data and resource management, and so on. Specifically, we focus on how the visualization of the planning and plan recognition processes forms a key component of the smart assistant, and establishes transparency in the decision-making process. We also highlight how these processes contribute to the proactive nature of the agent. We demonstrate some of these functionalities in a successful deployment of the system in the CEL -the Cognitive Environments Laboratory at IBM's T.J. Watson Research Center (Yorktown, USA), and report on emerging deployments of the system that have turned into success stories.

show abstract

“…This taxonomy covers a number of prevalent terms (conversational assistant, CI, chatbot, digital assistant, virtual assistant, or dialog system) and generalizes them for advanced scenarios where both users and system are expected to work even more collaboratively on complex tasks in natural environments. 17 , 18 Hence, I use the term collaborative assistants henceforth and refer to it with CA or chatbot as the short form.…”

Section: Introductionmentioning

confidence: 99%

Did chatbots miss their “Apollo Moment”? Potential, gaps, and lessons from using collaboration assistants during COVID-19

Srivastava

2021

Patterns

Self Cite

View full text Add to dashboard Cite

Summary Artificial intelligence (AI) technologies have long been positioned as a tool to provide crucial data-driven decision support to people. In this survey paper, I look at how collaboration assistants (chatbots for short), a type of AI that allows people to interact with them naturally (such as using speech, gesture, and text), have been used during a true global exigency—the COVID-19 pandemic. The key observation is that chatbots missed their “Apollo Moment” when at the time of need, they could have provided people with useful and life-saving contextual, personalized, and reliable decision support at a scale that the state-of-the-art makes possible. By “Apollo Moment”, I refer to the opportunity for a technology to attain the pinnacle of its impact. I review the chatbot capabilities that are feasible with existing methods, identify the potential that chatbots could have met, and highlight the use-cases they were deployed on, the challenges they faced, and gaps that persisted. Finally, I draw lessons that, if implemented, would make them more relevant in future health emergencies.

show abstract

An Embodied Cognitive Assistant for Visualizing and Analyzing Exoplanet Data

Cited by 9 publications

References 13 publications

Reagent: Converting Ordinary Webpages into Interactive Software Agents

Reagent: Converting Ordinary Webpages into Interactive Software Agents

Planning and visualization for a smart meeting room assistant1

Did chatbots miss their “Apollo Moment”? Potential, gaps, and lessons from using collaboration assistants during COVID-19

Contact Info

Product

Resources

About