Towards an Evaluation Framework for Autonomous Systems

Bombarda, Andrea; Bonfanti, Silvia; Sanctis, Martina De; Gargantini, Angelo; PelliccioneT, Patrizio; Riccobene, Elvinia; Scandurra, Patrizia

doi:10.1109/acsosc56246.2022.00025

Cited by 4 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The starting point for the customization is the characterization of the class C of systems under consideration, and then the analysis of the underlying domain and features of interest. The third step refers to the analysis of the structure (in terms of abilities and sub-abilities) of LENS and of other existing frameworks in LENS_ZOO (available in our supplementary material [15]), i.e., frameworks obtained from previous customizations of LENS, for similar classes of systems. Other suitable external frameworks might also be considered for the purpose.…”

Section: Customization Of Lens: Key Points and Lessons Learnedmentioning

confidence: 99%

“…(5) it assesses the SUE and assigns an assessment value to the ability levels: Not applicable, Satisfied, Improvable (low effort), Improvable (high effort) and Unable (see Table 5 for an example of evaluation; some aspects of the evaluation are explained in the following items); (6) it provides examples explaining the given assessment for Satisfied levels (see Table 4 for an example); (7) it provides recommendations for potential extensions of the SUE for Improvable (low effort) and Improvable (high effort) levels (see Table 4 for an example).…”

Section: Assessment Processmentioning

confidence: 99%

“…Over the years, SASs are increasingly becoming "smarter" to be able to adapt and learn how to handle and manage new and unexpected events with autonomy. However, the precise meaning of "making a system smarter" is not always obvious, and, more pragmatically, it is not straightforward to decide how to concretely operate to achieve the ambition [7]. Making a system smarter might involve various system's abilities, such as configurability, autonomy, adaptability, perception, cognitive, and interaction with other systems and humans, to mention a few.…”

Section: Introductionmentioning

confidence: 99%

“…Making a system smarter might involve various system's abilities, such as configurability, autonomy, adaptability, perception, cognitive, and interaction with other systems and humans, to mention a few. These abilities can have various levels of importance in specific systems [7]. Then, it is important to understand in which direction it is worth and useful to invest to make systems smarter, and how and how much to improve a system in each specific direction.…”

Section: Introductionmentioning

confidence: 99%

“…The preliminary idea of LENS has been presented in the short paper published at the ACSOS 2022 conference [7]. In the previous paper, we motivated the need for LENS, we described the process that we followed to create it, and briefly sketched the framework.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Evaluation Framework for Autonomous Systems: The Case of Programmable Electronic Medical Systems

Bombarda,

Bonfanti,

De Sanctis

et al. 2024

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

This paper proposes an evaluation framework for autonomous systems, called LENS. It is an instrument to make an assessment of a system through the lens of abilities related to adaptation and smartness. The assessment can then help engineers understand in which direction it is worth investing to make their system smarter. It also helps to identify possible improvement directions and to plan for concrete activities. Finally, it helps to make a re-assessment when the improvement has been performed in order to check whether the activity plan has been accomplished. Given the high variability in the various domains in which autonomous systems are and can be used, LENS is defined in abstract terms and instantiated to a specific and important class of medical devices, i.e., Programmable Electronic Medical Systems (PEMS). The instantiation, called LENSPEMS, is validated in terms of applicability, i.e., how it is applicable to real PEMS, generalizability, i.e., to what extent LENSPEMS is generalizable to the PEMS class of systems, and usefulness, i.e., how it is useful in making an assessment and identifying possible directions of improvement towards smartness.

show abstract

Section: Customization Of Lens: Key Points and Lessons Learnedmentioning

confidence: 99%

Section: Assessment Processmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Evaluation Framework for Autonomous Systems: The Case of Programmable Electronic Medical Systems

Bombarda,

Bonfanti,

De Sanctis

et al. 2024

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

Enhancing self-adaptation for efficient decision-making at run-time in streaming applications on multicores

Vogel,

Danelutto,

Torquati

et al. 2024

J Supercomput

View full text Add to dashboard Cite

Parallel computing is very important to accelerate the performance of computing applications. Moreover, parallel applications are expected to continue executing in more dynamic environments and react to changing conditions. In this context, applying self-adaptation is a potential solution to achieve a higher level of autonomic abstractions and runtime responsiveness. In our research, we aim to explore and assess the possible abstractions attainable through the transparent management of parallel executions by self-adaptation. Our primary objectives are to expand the adaptation space to better reflect real-world applications and assess the potential for self-adaptation to enhance efficiency. We provide the following scientific contributions: (I) A conceptual framework to improve the designing of self-adaptation; (II) A new decision-making strategy for applications with multiple parallel stages; (III) A comprehensive evaluation of the proposed decision-making strategy compared to the state-of-the-art. The results demonstrate that the proposed conceptual framework can help design and implement self-adaptive strategies that are more modular and reusable. The proposed decision-making strategy provides significant gains in accuracy compared to the state-of-the-art, increasing the parallel applications’ performance and efficiency.

show abstract

Leveraging Large Language Models for Auto-remediation in Microservices Architecture

Sarda

2023

2023 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)

View full text Add to dashboard Cite

Towards an Evaluation Framework for Autonomous Systems

Cited by 4 publications

References 21 publications

Evaluation Framework for Autonomous Systems: The Case of Programmable Electronic Medical Systems

Evaluation Framework for Autonomous Systems: The Case of Programmable Electronic Medical Systems

Enhancing self-adaptation for efficient decision-making at run-time in streaming applications on multicores

Leveraging Large Language Models for Auto-remediation in Microservices Architecture

Contact Info

Product

Resources

About