Automated Testing of Mobile Applications Using a Robotic Arm

Frister, Demian; Oberweis, Andreas; Goranov, Aleksandar I.

doi:10.1109/csci51800.2020.00321

Cited by 4 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MAEIT drives the robot to perform specific test tasks. 25,26 When MAEIT provides operation instructions to the robot, the coordinates of the operating controls of the test case should be converted into the executable space coordinates of the robot in the real physical world to control the attitude of the robot. The mobile application robot executes executable use cases to realize the test evaluation of the software on different devices.…”

Section: Execution Of Robot Imitation Testmentioning

confidence: 99%

Machine vision-based testing action recognition method for robotic testing of mobile application

Zhang

Cheng

et al. 2022

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

The explosive growth and rapid version iteration of various mobile applications have brought enormous workloads to mobile application testing. Robotic testing methods can efficiently handle repetitive testing tasks, which can compensate for the accuracy of manual testing and improve the efficiency of testing work. Vision-based robotic testing identifies the types of test actions by analyzing expert test videos and generates expert imitation test cases. The mobile application expert imitation testing method uses machine learning algorithms to analyze the behavior of experts imitating test videos, generates test cases with high reliability and reusability, and drives robots to execute test cases. However, the difficulty of estimating multi-dimensional gestures in 2D images leads to complex algorithm steps, including tracking, detection, and recognition of dynamic gestures. Hence, this article focuses on the analysis and recognition of test actions in mobile application robot testing. Combined with the improved YOLOv5 algorithm and the ResNet-152 algorithm, a visual modeling method of mobile application test action based on machine vision is proposed. The precise localization of the hand is accomplished by injecting dynamic anchors, attention mechanism, and the weighted boxes fusion in the YOLOv5 algorithm. The improved algorithm recognition accuracy increased from 82.6% to 94.8%. By introducing the pyramid context awareness mechanism into the ResNet-152 algorithm, the accuracy of test action classification is improved. The accuracy of the test action classification was improved from 72.57% to 76.84%. Experiments show that this method can reduce the probability of multiple detections and missed detection of test actions, and improve the accuracy of test action recognition.

show abstract

Section: Execution Of Robot Imitation Testmentioning

confidence: 99%

Machine vision-based testing action recognition method for robotic testing of mobile application

Zhang

Cheng

et al. 2022

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

show abstract

“…Several studies involve robot-assisted inspection of TPs [6][7][8][9][10][11][12]. Jenkinson [6] used a touchscreen testing platform with a robotic tester and an electronic control system for repeatable testing of TPs.…”

Section: Introductionmentioning

confidence: 99%

“…Lu and Juang [9] used a five-DOF robot arm to input words onto the TPs for smartphones using the Fuzzy theory to position the robot arm rapidly. Frister et al [10] proposed several methods to test mobile applications using robotic arms. The proposed system executes black-box tests using a tree-search algorithm.…”

Section: Introductionmentioning

confidence: 99%

A Robust FOPD Controller That Allows Faster Detection of Defects for Touch Panels

Wang

2024

MCA

View full text Add to dashboard Cite

This study aims to synthesize and implement a robust fractional order PD (RFOPD) controller to increase the speed at which defects in automated touch panel inspection systems (ATPISs) are detected. A three-dimensional orthogonal stage (TDOS) driven by BLDC servo motors moves the inspection pen (IP) vertically and horizontally. The dynamic equation relating the BLDC servo motor input to the tip motion is established. A touch position identification (TPI) system is used to locate the touch point rapidly. An RFOPD controller is used to actuate the BLDC servo motors and move the TDOS rapidly and accurately in three dimensions. This method displaces the IP to any specified position and shows user-defined inspection trajectories on the touch screens. The gain-phase margin tester (GPMT) and stability equation methods are exploited to schedule the RFOPD controller gain settings and to maintain the specific safety margins for the controlled system. The simulation studies show that the proposed RFOPD controller exhibits better tracking and disturbance rejection responses than a conventional PID controller. The robustness of the RFOPD-controlled ATPIS, considering unmodeled uncertainties and friction-induced disturbances, is verified through simulation and experimental studies. Several user-defined inspection patterns are used to verify performance, and the experimental results show that the proposed RFOPD controller is effective.

show abstract