Building Trustworthy AI Solutions: A Case for Practical Solutions for Small Businesses

Abstract

Building trustworthy AI solutions, whether in academia or industry, must take into consideration a number of dimensions including legal, social, ethical, public opinion and environmental aspects. A plethora of guidelines, principles and toolkits have been published globally, but have seen limited grassroots implementation, especially among small and medium sized enterprises (SME), mainly due to lack of knowledge, skills, and resources. In this paper, we report on qualitative SME consultations over two events to establish their understanding of both data and AI ethical principles and to identify the key barriers SMEs face in their adoption of ethical AI approaches. We then use independent experts to review and code 77 published toolkits designed to build and support ethical and responsible AI practices, based on 33 evaluation criteria. The toolkits were evaluated considering their scope to address the identified SME barriers to adoption, human-centric AI principles, AI lifecycle stages, and key themes around responsible AI and practical usability. Toolkits were ranked based on criteria coverage and expert inter-coder agreement. Results show that there is not a one-size-fits-all toolkit that addresses all criteria suitable for SMEs. Our findings show few exemplars of practical application, little guidance on how to use/apply the toolkits and very low uptake by SMEs. Our analysis provides a mechanism for SMEs to select their own toolkits based on their current capacity, resources, and ethical awareness levelsfocusing initially at the conceptualization stage of the AI lifecycle and then extending throughout.Impact Statement -In parallel to the recent acceleration in development and adoption of artificial intelligence, there has been intense and worldwide discourse around the ethics of such systems. This debate has highlighted that without good governance, transparency and monitoring, indiscriminate use of AI could lead to significant harms, discrimination, and injustice.Consensus has settled on a broad set of overarching principles for ethical AI; now myriad resources and toolkits exist to assist with embedding ethical practices along the researchdevelopment-deployment value chain. Our evaluation of 77 toolkits reveals the breadth and depth of the themes they cover and barriers to their use, including a lack of adoption case studies. We provide organizations, especially SMEs, with an easy-to-use lookup table (Table V) to help them select a set of toolkits to ensure that as well as addressing all key ethical themes, they can also match their resources, skills and priority areas for implementing ethical best practice.

An Overview of Artificial Intelligence Ethics

“…The general lifecycle or development process of a ML-based AI system [75] or product [76] often involves the following stages: business analysis, data engineering, ML modeling, model deployment, and operation and monitoring. Usually, the lifecycle of AI products starts from the business analysis, which mainly involves identifying and understanding the business problem to be solved and business metrics (or criteria of success).…”

From Ethical Artificial Intelligence Principles to Practice: A Case Study of University-Industry Collaboration

“…There is abundance of ethical toolkits that could be applied within the context of the GMAIF project to help organizations and businesses develop responsible and trustworthy AI [36]. Crockett et al [37] evaluated and ranked 77 published toolkits based on 33 evaluation criteria which included: toolkit scope in addressing identified SME barriers to adoption; human-centric AI principles, relevance to specific AI lifecycle stages, and key themes around responsible AI and practical usability [37]. The results of this study showed that there is no one-sizefits-all toolkit that addresses all criteria and is suitable for all organizations, including SMEs.…”

“…The results of this study showed that there is no one-sizefits-all toolkit that addresses all criteria and is suitable for all organizations, including SMEs. This research therefore selects some of the most promising toolkits and practices identified by Crockett et al [37] and applies them within the context of the GMAIF project. Additionally, it explores some approaches that this prior work fails to consider and evaluate.…”

“…The Microsoft Responsible Innovation Toolkit aims to help developers consider the effects of technology and future science on society [43]. In 2021, it was ranked, the number one toolkit for SMEs out of 77 [37] due to its coverage of AI principals, suitability for SMEs, coverage of the AI lifecycle stages and addressing responsible AI and practical usability. Despite being in the early stages, the toolkit has been shared so that feedback can be provided.…”

Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics

“…Morley et al [43] proposes a typology of AI ethics approaches condensed from a variety of toolkits, also critiquing toolkits' lack of usability and their focus on individuals rather than social groups. Crockett et al [12] analyze 77 AI ethics toolkits, finding that many lack instructions or training to facilitate adoption. They focus on the toolkits' dominant principles, product lifecycle stages, and measure how well they may be adopted by small and medium enterprises.…”