Gradient‐based 3D skin roughness rendering from an in‐vivo skin image for dynamic haptic palpation

Abstract

Background/Purpose Many studies have revealed the importance of palpation for dermatologists; however, palpation is not always possible due to the risk of secondary infections or the risk of damaging the affected area. Thus, haptic rendering for indirect palpation using in‐vivo skin images, which will enable to examine a real three‐dimensional (3D) skin sample by virtual touch without directly palpating the infected skin area, could be a useful technology in dermatology. Methods We propose a new method of accurate 3D skin surface reconstruction using simple gradients from a single skin image for accurate 3D roughness rendering with a haptic device. Our approach takes advantage of bilateral filtering to preserve skin roughness and image gradients in order to generate a 3D skin surface (polygonal meshes) while preserving skin wrinkles and rough surface textures. Results Our method was evaluated using two experiments. The accuracy was tested with six 3D ground‐truth surfaces and four clinical skin images (acne, miliaria, sweet syndrome, and herpes simplex). For objective evaluation, a well‐known 3D roughness estimation method and the Hausdorff distance were adopted to compute errors. All results showed that the accuracy of our method is superior to that of the two existing methods. Conclusion Haptic roughness rendering for skin palpation examination requires efficient and accurate 3D surface reconstruction. In this study, we developed a new method that can be used to reconstruct a 3D skin surface accurately while preserving roughness through the use of a single skin image.

Skin tactile surface restoration using deep learning from a mobile image: An application for virtual skincare

“…In our method, 3D reconstruction is performed using the gradients of the skin tactile image obtained above. 14,38 The gradient is obtained by measuring the change between the intensity of each pixel and the intensity of the surrounding pixels, and the gradient in the image I T is represented as a partial derivation.…”

“…In the above equation, E(M) is a function to find the shape variation of M. The optimal situation is that the target normal vector of S and the normal vector of facet x,y are equal (see 14,38 for details).…”

“…For this reason, technology development has actively sought to precisely recover tactile information using optical imaging techniques for skin images acquired by an electronic microscope, a mobile phone camera, or a light-field camera. [5][6][7][8][9][10][11][12][13][14] These technologies are aimed at generating tactile skin surfaces and providing realistic tactile force feedback using a haptic feedback device that enables virtual palpation or virtual skincare through touch, which was not possible in the past. The most important aspect of virtual touch (haptic) interface technology for tele-skincare is the generation of an accurate 3D tactile surface model that is free from light distortions, which often results in an overelaborated surface, especially on the focus spot when reconstructing the 3D surface from a mobile skin image.…”