Robotic-Based Touch Panel Test System Using Pattern Recognition Methods

Abstract

In this study, pattern recognition methods are applied to a five-degrees-of-freedom robot arm that can key in words on a touch screen for an automatic smartphone test. The proposed system can recognize Chinese characters and Mandarin phonetic symbols. The mechanical arm is able to perform corresponding movements and edit words on the screen. Pattern matching is based on the Red-Green-Blue (RGB) RGB color space and is transformed to binary images for higher correct rate and geometric matching. A web camera is utilized to capture patterns on the tested smartphone screen. The proposed control scheme uses a support vector machine with a histogram of oriented gradient classifier to recognize Chinese Mandarin phonetic symbols and provide correct coordinates during the control process. The control scheme also calculates joint angles of the robot arm during the movement using the Denavit–Hartenberg parameters (D-H) model and fuzzy logic system. Fuzzy theory is applied to use the position error between the robot arm and target location then resend the command to adjust the arm’s position. From the experiments, the proposed control scheme can control the robot to press desired buttons on the tested smartphone. For Chinese Mandarin phonetic symbols, recognition accuracy of the test system can reach 90 percent.

A Robust FOPD Controller That Allows Faster Detection of Defects for Touch Panels

“…Several studies involve robot-assisted inspection of TPs [6][7][8][9][10][11][12]. Jenkinson [6] used a touchscreen testing platform with a robotic tester and an electronic control system for repeatable testing of TPs.…”

“…This robotic arm consists of a stylus that moves in three dimensions. Lu and Juang [9] used a five-DOF robot arm to input words onto the TPs for smartphones using the Fuzzy theory to position the robot arm rapidly. Frister et al [10] proposed several methods to test mobile applications using robotic arms.…”