Prototype tactile feedback system for examination by skin touch

2019

Background/Purpose Many studies have revealed the importance of palpation for dermatologists; however, palpation is not always possible due to the risk of secondary infections or the risk of damaging the affected area. Thus, haptic rendering for indirect palpation using in‐vivo skin images, which will enable to examine a real three‐dimensional (3D) skin sample by virtual touch without directly palpating the infected skin area, could be a useful technology in dermatology. Methods We propose a new method of accurate 3D skin surface reconstruction using simple gradients from a single skin image for accurate 3D roughness rendering with a haptic device. Our approach takes advantage of bilateral filtering to preserve skin roughness and image gradients in order to generate a 3D skin surface (polygonal meshes) while preserving skin wrinkles and rough surface textures. Results Our method was evaluated using two experiments. The accuracy was tested with six 3D ground‐truth surfaces and four clinical skin images (acne, miliaria, sweet syndrome, and herpes simplex). For objective evaluation, a well‐known 3D roughness estimation method and the Hausdorff distance were adopted to compute errors. All results showed that the accuracy of our method is superior to that of the two existing methods. Conclusion Haptic roughness rendering for skin palpation examination requires efficient and accurate 3D surface reconstruction. In this study, we developed a new method that can be used to reconstruct a 3D skin surface accurately while preserving roughness through the use of a single skin image.

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Gradient‐based 3D skin roughness rendering from an in‐vivo skin image for dynamic haptic palpation

Vicente

2019

“…However, relying solely on visual information obtained from captured images has a limitation in providing sufficient information (roughness and hardness) on the skin surface for a dermatologist. Therefore, the use of palpation information as well as visual information through a haptic device has been steadily gaining attention . Thus, in order to utilize a haptic device to obtain palpation information, a sophisticated three‐dimensional (3D) skin surface reconstruction method should be explored.…”

Section: Introductionmentioning

confidence: 99%

Accurate depth estimation of skin surface using a light‐field camera toward dynamic haptic palpation

2019

Background Haptic skin palpation with three‐dimensional skin surface reconstruction from in vivo skin images in order to acquire both tactile and visual information has been receiving much attention. However, the depth estimation of skin surface, using a light field camera that creates multiple images with a micro‐lens array, is a difficult problem due to low‐resolution images resulting in erroneous disparity matching. Methods Multiple low‐resolution images decoded from a light field camera have limitations to accurate 3D surface reconstruction needed for haptic palpation. To overcome this, a deep learning method, Generative Adversarial Networks, was employed to generate super‐resolved skin images that preserve surface detail without blurring, and then, accurate skin depth was estimated by taking multiple subsequent steps including lens distortion correction, sub‐pixel shifted image generation using phase shift theorem, cost‐volume building, multi‐label optimization, and hole filling and refinement, which is a new approach for 3D skin surface reconstruction. Results Experimental results of the deep‐learning‐based super‐resolution method demonstrated that the textural detail (wrinkles) of super‐resolved skin images is well preserved, unlike other super‐resolution methods. In addition, the depth maps computed with our proposed algorithm verify that our method can produce more accurate and robust results compared to other state‐of‐the‐art depth map computation methods. Conclusion Herein, we first proposed depth map estimation of skin surfaces using a light field camera and subsequently tested it with several skin images. The experimental results established the superiority of the proposed scheme.

“…Despite the importance of palpation, it adds the risks of secondary infection and damage to diseased skin. Fortunately, recent 3D skin surface reconstruction and rendering technologies have enabled haptic feedback, which is generated from image‐based skin surface imaging . This type of reconstruction technology enables remote palpation diagnoses of skin disease.…”

Section: Introductionmentioning

confidence: 99%

“…Fortunately, recent 3D skin surface reconstruction and rendering technologies have enabled haptic feedback, which is generated from image-based skin surface imaging. [7][8][9][10][11][12] This type of reconstruction technology enables remote palpation diagnoses of skin disease. Therefore, precise 3D skin surface reconstruction leads to a very important and challenging study toward an ultimate solution of 3D haptic skin palpation.…”

mentioning

confidence: 99%

3D skin surface reconstruction from a single image by merging global curvature and local texture using the guided filtering for 3D haptic palpation

Lee

2018

Self Cite