Smartphone Handwritten Circuits Solver Using Augmented Reality and Capsule Deep Networks for Engineering Education

Abstract

Resolving circuit diagrams is a regular part of learning for school and university students from engineering backgrounds. Simulating circuits is usually done manually by creating circuit diagrams on circuit tools, which is a time-consuming and tedious process. We propose an innovative method of simulating circuits from hand-drawn diagrams using smartphones through an image recognition system. This method allows students to use their smartphones to capture images instead of creating circuit diagrams before simulation. Our contribution lies in building a circuit recognition system using a deep learning capsule networks algorithm. The developed system receives an image captured by a smartphone that undergoes preprocessing, region proposal, classification, and node detection to get a Netlist and exports it to a circuit simulator program for simulation. We aim to improve engineering education using smartphones by (1) achieving higher accuracy using less training data with capsule networks and (2) developing a comprehensive system that captures hand-drawn circuit diagrams and produces circuit simulation results. We use 400 samples per class and report an accuracy of 96% for stratified 5-fold cross-validation. Through testing, we identify the optimum distance for taking circuit images to be 10 to 20 cm. Our proposed model can identify components of different scales and rotations.

A Comprehensive Survey on the Investigation of Machine-Learning-Powered Augmented Reality Applications in Education

“…Capsule networks, a form of DL, played a vital role in recognizing and classifying characters within circuit diagrams. With a remarkable 96% accuracy, capsule networks outperformed traditional CNNs, making circuit simulation more accessible and engaging for students [79].…”

“…Another critical challenge in the combination of ML and AR is the accuracy and speed of object recognition within complex diagrams [79]. Aligning AR objects seamlessly with real-world scenes and training models with a large amount of data are also formidable tasks [79].…”

“…Another critical challenge in the combination of ML and AR is the accuracy and speed of object recognition within complex diagrams [79]. Aligning AR objects seamlessly with real-world scenes and training models with a large amount of data are also formidable tasks [79]. To ensure a comfortable visual experience with head-mounted displays (HMDs), ideally, frame rates should reach around 60 frames per second.…”

ElectroNet: An Enhanced Model for Small-Scale Object Detection in Electrical Schematic Diagrams

“…Yamakami [28] sought a quantum analogue of the schematic (inductive or constructive) definition of (primitive) recursive functions for quantum functions mapping finite-dimensional Hilbert spaces to themselves. The field of object-detection has seen significant advancements, particularly with the adaptation of algorithms such as You Only Look Once (YOLO) for specific tasks like electric-circuit recognition, as highlighted by Alhalabi et al [22]. In a comprehensive review of YOLO and Convolutional Neural Networks (CNN) for real-time object-detection, Viswanatha et al [29] underscored the effectiveness of these models in generalized object representation without precision losses.…”

Teaching Fourier series with tone experiments based on smartphone applications

“…For example, a smartphone-based instant response system (IRS) can immediately reflect students' learning status [6,7]. In addition, a smartphone-based augmented reality (AR) system captures hand-drawn or printed circuit diagrams, produces its simulation circuit, and outputs simulation results to speed up circuit design and simulation [1,3].…”