Data-driven accessibility repair revisited: on the effectiveness of generating labels for icons in Android apps

Icon Label Generation for MobileApplications by Mean Teacher Learning

“…Table 2 compares the efficiency of our approach in generating icon labels to LabelDroid and COALA. It can be seen that although COALA model has a significant effect on non-predefined labels [6], the overall efficiency is not as good as LabelDroid, and the exact match rate with ground-truth is only 53.1%. And our proposed method surpasses all of the existing methods by a wide margin with 62.8% ground-truth match rate.…”

“…Chen [3] scanned 10,408 applications from Google Play, built a related datasets, and utilized deep learning algorithms to automatically predict labels based on image icons, which has attracted widespread attention in app accessibility. COALA re-qualified the LabelDroid datasets and developed a context-aware method to predict image-based icon labels, and is one of the most relevant to this work [6]. With the help of mean teacher learning, our two-stream language model outperforms the COALA approach significantly on different COALA datasets.…”

“…Furthermore, many specialized apps on the market are less concerned about the usage of visually impaired users [3,4]. Therefore, the research on the accessibility of mobile programs has received extensive attention, and the most noticeable issue is a lack of labeling [3,5,6].…”

“…Chen [3] proposed a deep learning method called LabelDroid for the first time, constructed a large-scale image-based icon dataset, and utilized the transformer model to automatically predict natural language labels based on image icons. Mehralian [6] has conducted extensive research on image icons and labels since then, discovered data-driven biases in the LabelDroid dataset. He proposed a method COALA to automatically extract high-quality labels and leverage contextual information to generate icon labels.…”

“…In order to solve this problem, Chen [3] crawls image-based icons from large-scale applications, organizes them into large-scale databases, and proposes a method for automatically generating natural language labels for icons. Subsequently, COALA [6] studies the qualities of icons and label features, and proposes an automatic label extraction and generation approach. Although COALA performs well on non-predefined data sets, its prediction accuracy on the whole imbalanced data set is even worse than that of LabelDroid.…”