Wearable Nail Deformation Sensing for Behavioral and Biomechanical Monitoring and Human-Computer Interaction

Sakuma, Katsuyuki; Abrami, Avner; Blumrosen, Gaddi; Lukashov, Stanislav; Narayanan, Rajeev; Ligman, Joseph W.; Caggiano, Vittorio; Heisig, Stephen

doi:10.1038/s41598-018-36834-x

Cited by 30 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the work in [23], we showed first feasibility of using the sensor for writing and recognizing basic shapes using only two pressure points. This paper extends the work in [22], and [23], and focus on using any pressure based sensor in task of writing full sentences in real-time in challenging conditions like different surfaces.…”

Section: Introductionmentioning

confidence: 78%

“…The sensor is placed on the fingernail of the index finger. In the work in [22], a pressure sensor based on strain-gauge technology placed on the fingernail was introduced. It was shown, how the sensor is capable of measuring pressure point in real-time, can be used for assessing parameters like grip strength vital for medical diagnosis, and with machine learning tools, can also be used for coarse gesture recognition that can be used for enabling enhanced humanmachine interface.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Back to Finger-Writing: Fingertip Writing Technology Based on Pressure Sensing

et al. 2020

Self Cite

View full text Add to dashboard Cite

Handwriting was since the start of the history, a higher expression of human skills, and was used for documentation of experiences, and for communication. Existing writing technology require a writing tool, like a pen, and a dedicated writing surface, like paper, or more recently an electronic tablet. These accessories of writing, of writing tool and service, are not available in many daily life situations. Furthermore, the writing accessories, are not natural, in many cases are not ergonomic, and thus can cause fatigue, and in extreme cases contribute to muscular and neurological diseases. In this work, we suggest to step back in history and step forward in technology, and to create, for the first time, an alternate writing solution without any accessories, using one own finger as writing tool, and write on almost any surface. For this, we used directional pressure sensors attached to the fingernail. Changes in the pressure induced on the fingertip in different directions while writing, are projected to the fingernail, and then assessed as a voltage pattern by the sensor. Decoding the pattern, can reveal symbols like letters, punctuations, and writing commands. In this paper, we describe the new pressure sensing modality and tailor processing methods. We tested the new technology on two subjects having different writing patterns while writing alphabet and sentences on different surfaces. We reached letter detection of over 80% while writing on a table, and the word detection rate, was near 70%, after applying the correction algorithm include language priors. The results of this work can revolutionize the way people write and communicate using more convenient, and more approachable, fingertip writing. INDEX TERMS Handwriting recognition, human machine interface, natural language processing.

show abstract

Section: Introductionmentioning

confidence: 78%

Section: Introductionmentioning

confidence: 99%

Back to Finger-Writing: Fingertip Writing Technology Based on Pressure Sensing

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several tactile sensors can detect human tactile sensations that resemble a fingernail-color sensor, such as a near infrared light-emitting diode (LED) and photodiode types that detect variations in blood distribution in nails [11] [12], a photoplethysmogram (PPG) type that measures the pulse waves of the proximal part of a finger [13], the deformation detection of fingernail types by strain gauges [14,15], and a fingernail image type that uses the average value of the pixel data in each cell of the fingernail images acquired in a darkroom [16]. Although the LED and photodiode types can detect a three-axis force, their precision depends on the positional settings of the photodiodes on the nails.…”

Section: Introductionmentioning

confidence: 99%

Tangential-force detection ability of three-axis fingernail-color sensor aided by CNN

et al. 2023

View full text Add to dashboard Cite

We create a new tactile recording system with which we develop a three-axis fingernail-color sensor that can measure a three-dimensional force applied to fingertips by observing the change of the fingernail’s color. Since the color change is complicated, the relationships between images and three-dimensional forces were assessed using convolution neural network (CNN) models. The success of this method depends on the input data size because the CNN model learning requires big data. Thus, to efficiently obtain big data, we developed a novel measuring device, which was composed of an electronic scale and a load cell, to obtain fingernail images with 0 $^\circ$ to 360 $^\circ$ directional tangential force. We performed a series of evaluation experiments to obtain movies of the color changes caused by the three-axis forces and created a data set for the CNN models by transforming the movies to still images. Although we produced a generalized CNN model that can evaluate the images of any person’s fingernails, its root means square error (RMSE) exceeded both the whole and individual models, and the individual models showed the smallest RMSE. Therefore, we adopted the individual models, which precisely evaluated the tangential-force direction of the test data in an $F_x$ - $F_y$ plane within around $\pm$ 2.5 $^\circ$ error at the peak points of the applied force. Although the fingernail-color sensor possessed almost the same level of accuracy as previous sensors for normal-force tests, the present fingernail-color sensor acts as the best tangential sensor because the RMSE obtained from tangential-force tests was around 1/3 that of previous studies.

show abstract

“…As sensor devices attached to fingernails, force sensors , touchpads , hand and finger motion sensors , visual displays , tactile displays , and small photoplethysmographic (PPG) recording circuits are reported. Katsuyuki et al .…”

Section: Introductionmentioning

confidence: 99%

“…Katsuyuki et al . reviewed and summarized advantages among these devices in their Table . Tanaka et al .…”

Section: Introductionmentioning

confidence: 99%

Nail tip sensor: Toward reliable daylong monitoring of heart rate

Ishii

Hiraoka

2020

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

A nail tip sensor that can provide long-term vital data acquisition without uncomfortable wearing feel is described and discussed. Through 24 h continuous measurement under daily life environment, the proposed nail tip photoplethysmographic (PPG) sensor circuit works effectively and shows no deterioration in the quality of PPG waveform due to aging, peeling, flooding, or vibration. The waveform of measured pulse obtained from the nail tip PPG sensor is consistent with that from electrocardiograph (ECG), excluding the time when saturation or undershoot due to body motion artifact is dominant. Heart rates calculated from PPG and ECG pulse waves with simple algorithm are correlated with each other, and show circadian variation and respiratory variation.

show abstract

Wearable Nail Deformation Sensing for Behavioral and Biomechanical Monitoring and Human-Computer Interaction

Cited by 30 publications

References 22 publications

Back to Finger-Writing: Fingertip Writing Technology Based on Pressure Sensing

Back to Finger-Writing: Fingertip Writing Technology Based on Pressure Sensing

Tangential-force detection ability of three-axis fingernail-color sensor aided by CNN

Nail tip sensor: Toward reliable daylong monitoring of heart rate

Contact Info

Product

Resources

About