Enabling the Discovery of Manual Processes Using a Multi-modal Activity Recognition Approach

Rebmann, Adrian; Emrich, Andreas; Fettke, Peter

doi:10.1007/978-3-030-37453-2_12

Cited by 12 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The IoT introduces new data sources for monitoring activities, behavior and daily routines, e.g., using multi-modal motion and vision sensors to detect objects or humans [37]. Many approaches specialize in the detection of specific activities from one or more sensors [38,39].…”

Section: Bpm Meets Iotmentioning

confidence: 99%

An Interactive Method for Detection of Process Activity Executions from IoT Data

2023

View full text Add to dashboard Cite

The increasing number of IoT devices equipped with sensors and actuators pervading every domain of everyday life allows for improved automated monitoring and analysis of processes executed in IoT-enabled environments. While sophisticated analysis methods exist to detect specific types of activities from low-level IoT data, a general approach for detecting activity executions that are part of more complex business processes does not exist. Moreover, dedicated information systems to orchestrate or monitor process executions are not available in typical IoT environments. As a consequence, the large corpus of existing process analysis and mining techniques to check and improve process executions cannot be applied. In this work, we develop an interactive method guiding the analysis of low-level IoT data with the goal of detecting higher-level process activity executions. The method is derived following the exploratory data analysis of an IoT data set from a smart factory. We propose analysis steps, sensor-actuator-activity patterns, and the novel concept of activity signatures that are applicable in many IoT domains. The method shows to be valuable for the early stages of IoT data analyses to build a ground truth based on domain knowledge and decisions of the process analyst, which can be used for automated activity detection in later stages.

show abstract

Section: Bpm Meets Iotmentioning

confidence: 99%

An Interactive Method for Detection of Process Activity Executions from IoT Data

2023

View full text Add to dashboard Cite

show abstract

“…Enriching process execution environments with IoT technologies brings several benefits but does not come without challenges [1]. One of the main benefits is undoubtedly the improved recognition of (manual) activities and processes from sensor data, e. g., with the help of multi-modal approaches such as the one by Rebmann et al [16] that combines motion and vision sensors with user feedback for detecting and disambiguating known activity types in realtime. Activity recognition is often combined with process mining for extracting knowledge to analyze and optimize the underlying processes [17].…”

Section: Related Workmentioning

confidence: 99%

Towards IoT-driven Process Event Log Generation for Conformance Checking in Smart Factories

Seiger

Zerbato

Burattin

et al. 2020

2020 IEEE 24th International Enterprise Distributed Object Computing Workshop (EDOCW)

View full text Add to dashboard Cite

The Internet of Things (IoT) enables software-based access to vast amounts of data streams from sensors measuring physical and virtual properties of smart devices and their surroundings. While sophisticated means for the control and data analysis of single IoT devices exist, a more process-oriented view of IoT systems is often missing. Such a lack of process awareness hinders the development of process-based systems on top of IoT environments and the application of process mining techniques for process analysis and optimization in IoT. We propose a framework for the stepwise correlation and composition of raw IoT sensor streams with events and activities on a process level based on Complex Event Processing (CEP). From this correlation we derive refined process event logs-possibly with ambiguities-that can be used for process analysis at runtime (i. e., online). We discuss the framework using examples from a smart factory.

show abstract

“…Therefore, we need a machine learning approach that allows for multi-class classification. We selected a Random Forest, as it produced promising results in past applications [21] and does not require vast amounts of training data, as opposed to, e.g., neural networks. The Random Forest classifier we finally deployed was trained as an ensemble of 200 trees with a maximum tree depth of 16.…”

Section: Data Acquisitionmentioning

confidence: 99%

“…Presence Infrared sensors were used as a single data source. In [21] the development and evaluation of an approach which recognizes and logs manually performed assembly and commissioning activities is presented. The goal is to enable the application of process discovery methods.…”

Section: Related Workmentioning

confidence: 99%

IoT-Based Activity Recognition for Process Assistance in Human-Robot Disaster Response

Rebmann

Rehse

Pinter

et al. 2020

Lecture Notes in Business Information Processing

Self Cite

View full text Add to dashboard Cite

Mobile robots like drones or ground vehicles can be a valuable addition to emergency response teams, because they reduce the risk and the burden for human team members. However, the need to manage and coordinate human-robot team operations during ongoing missions adds an additional dimension to an already complex and stressful situation. BPM approaches can help to visualize and document the disaster response processes underlying a mission. In this paper, we show how data from a ground robot's reconnaissance run can be used to provide process assistance to the officers. By automatically recognizing executed activities and structuring them as an ad-hoc process instance, we are able to document the executed process and provide real-time information about the mission status. The resulting mission progress process model can be used for additional services, such as officer training or mission documentation. Our approach is implemented as a prototype and demonstrated using data from an ongoing research project on rescue robotics.

show abstract

Enabling the Discovery of Manual Processes Using a Multi-modal Activity Recognition Approach

Cited by 12 publications

References 12 publications

An Interactive Method for Detection of Process Activity Executions from IoT Data

An Interactive Method for Detection of Process Activity Executions from IoT Data

Towards IoT-driven Process Event Log Generation for Conformance Checking in Smart Factories

IoT-Based Activity Recognition for Process Assistance in Human-Robot Disaster Response

Contact Info

Product

Resources

About