An evaluation of augmented reality music notation

Liu, Zeruo; Adcock, Matt; Gardner, Henry

doi:10.1145/3359997.3365748

Cited by 3 publications

(2 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A related field for potential research involves the use of AR to help people visualize, annotate, and follow scores without the need for physical music and stands. One study from Liu et al (2019) uses a Microsoft HoloLens AR headset to overlay annotations on a user's FoV to manipulate a virtual score using voice commands, and a "bouncing ball" to track a performance in real time. A second, Kohen et al (2020)'s MiXR (Music in Mixed Reality) system uses a Microsoft HoloLens headset and a tablet to present scores and enable their annotation in real-time.…”

Section: Opportunities For Further Researchmentioning

confidence: 99%

Feedback in augmented and virtual reality piano tutoring systems: a mini review

Wilson

Pfeiffer

2023

Front. Virtual Real.

View full text Add to dashboard Cite

Researchers in music education are exploring the use of virtual reality (VR) and augmented reality (AR) to support piano instruction. Beginner piano students tend to receive short, infrequent lessons, which they practice on their own. This lack of instructor feedback creates opportunities for students to develop improper technique. Current strategies for using AR and VR to guide solo practice use moving shapes to help students to identify what notes to play. Improvements in commercial AR/VR technology will be needed to provide more detailed real-time feedback.

show abstract

Section: Opportunities For Further Researchmentioning

confidence: 99%

Feedback in augmented and virtual reality piano tutoring systems: a mini review

Wilson

Pfeiffer

2023

Front. Virtual Real.

View full text Add to dashboard Cite

show abstract

“…These processes often require manual work from artists or rely on a special capture system such as multi‐view camera sets or 3D scanners. To alleviate these conditions, numerous methods have been proposed to reconstruct a 3D human avatar [KPD19; APTM19; LCL*19; LIP19; XCM*21; NSH*19; SHN*19; SSSJ20; HCJS20; HXL*20; HXS*21; AZS22] from a single image. In contrast to the reported successes in reconstructing body shapes and poses, restoring the occluded texture for 3D human avatars has been relatively less studied.…”

Section: Introductionmentioning

confidence: 99%

Generating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks

Cha

Seo

Ashtari

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

There has been significant progress in generating an animatable 3D human avatar from a single image. However, recovering texture for the 3D human avatar from a single image has been relatively less addressed. Because the generated 3D human avatar reveals the occluded texture of the given image as it moves, it is critical to synthesize the occluded texture pattern that is unseen from the source image. To generate a plausible texture map for 3D human avatars, the occluded texture pattern needs to be synthesized with respect to the visible texture from the given image. Moreover, the generated texture should align with the surface of the target 3D mesh. In this paper, we propose a texture synthesis method for a 3D human avatar that incorporates geometry information. The proposed method consists of two convolutional networks for the sampling and refining process. The sampler network fills in the occluded regions of the source image and aligns the texture with the surface of the target 3D mesh using the geometry information. The sampled texture is further refined and adjusted by the refiner network. To maintain the clear details in the given image, both sampled and refined texture is blended to produce the final texture map. To effectively guide the sampler network to achieve its goal, we designed a curriculum learning scheme that starts from a simple sampling task and gradually progresses to the task where the alignment needs to be considered. We conducted experiments to show that our method outperforms previous methods qualitatively and quantitatively.

show abstract