Abstract. In recent years, deep learning (DL), a rebranding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, and natural language processing. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV, e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should not only be aware of advancements such as DL, but also be leading researchers in this area. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools, and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as they relate to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) nontraditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL. © The Authors.Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Wearables are a multi-billion-dollar business with more growth expected. Wearable technology is fully entrenched at multiple levels of athletic competition, especially at the National Collegiate Athletic Association (NCAA) and professional levels where these solutions are used to gain competitive advantages by assessing health and performance of elite athletes. However, through the National Science Foundation (NSF) Innovation Corps (I-Corps) training experience, a different story emerged based on pilot interviews from coaches and trainers regarding the lack of trust in wearables, and how the technology falls short of measuring what practitioners need. An NSF I-Corps project was funded to interview over 100 strength and conditioning coaches (S&CCs) and athletic trainers (ATs) regarding the current state of wearables at the NCAA and professional levels. Through 113 unstructured interviews, a conceptual map of relationships amongst themes and sub-themes regarding wearable technology emerged through the grouping of responses into meaning units (MUs). Interview findings revealed that discussions by S&CCs and ATs regarding wearables could be grouped into themes tied to (a) the organizational environment, (b) the athlete, and (c) the analyst or data scientist. Through this project, key findings and lessons learned were aggregated into sub-themes including: the sports ecosystem and organizational structure, brand development, recruiting, compliance and gamification of athletes, baselining movement and injury mitigation, internal and external loads, “return tos,” and quantifying performance. These findings can be used by practitioners to understand general technology practices and where to close the gap between what is available versus what is needed.

The linearity of soft robotic sensors (SRS) was recently validated for movement angle assessment using a rigid body structure that accurately depicted critical movements of the foot–ankle complex. The purpose of this study was to continue the validation of SRS for joint angle movement capture on 10 participants (five male and five female) performing ankle movements in a non-weight bearing, high-seated, sitting position. The four basic ankle movements—plantar flexion (PF), dorsiflexion (DF), inversion (INV), and eversion (EVR)—were assessed individually in order to select good placement and orientation configurations (POCs) for four SRS positioned to capture each movement type. PF, INV, and EVR each had three POCs identified based on bony landmarks of the foot and ankle while the DF location was only tested for one POC. Each participant wore a specialized compression sock where the SRS could be consistently tested from all POCs for each participant. The movement data collected from each sensor was then compared against 3D motion capture data. R-squared and root-mean-squared error averages were used to assess relative and absolute measures of fit to motion capture output. Participant robustness, opposing movements, and gender were also used to identify good SRS POC placement for foot–ankle movement capture.

Interviews from strength and conditioning coaches across all levels of athletic competition identified their two biggest concerns with the current state of wearable technology: (a) the lack of solutions that accurately capture data “from the ground up” and (b) the lack of trust due to inconsistent measurements. The purpose of this research is to investigate the use of liquid metal sensors, specifically Liquid Wire sensors, as a potential solution for accurately capturing ankle complex movements such as plantar flexion, dorsiflexion, inversion, and eversion. Sensor stretch linearity was validated using a Micro-Ohm Meter and a Wheatstone bridge circuit. Sensors made from different substrates were also tested and discovered to be linear at multiple temperatures. An ankle complex model and computing unit for measuring resistance values were developed to determine sensor output based on simulated plantar flexion movement. The sensors were found to have a significant relationship between the positional change and the resistance values for plantar flexion movement. The results of the study ultimately confirm the researchers’ hypothesis that liquid metal sensors, and Liquid Wire sensors specifically, can serve as a mitigating substitute for inertial measurement unit (IMU) based solutions that attempt to capture specific joint angles and movements.

The purpose of this study was to use 3D motion capture and stretchable soft robotic sensors (SRS) to collect foot-ankle movement on participants performing walking gait cycles on flat and sloped surfaces. The primary aim was to assess differences between 3D motion capture and a new SRS-based wearable solution. Given the complex nature of using a linear solution to accurately quantify the movement of triaxial joints during a dynamic gait movement, 20 participants performing multiple walking trials were measured. The participant gait data was then upscaled (for the SRS), time-aligned (based on right heel strikes), and smoothed using filtering methods. A multivariate linear model was developed to assess goodness-of-fit based on mean absolute error (MAE; 1.54), root mean square error (RMSE; 1.96), and absolute R 2 (R 2 ; 0.854). Two and three SRS combinations were evaluated to determine if similar fit scores could be achieved using fewer sensors. Inversion (based on MAE and RMSE) and plantar flexion (based on R 2 ) sensor removal provided second-best fit scores. Given that the scores indicate a high level of fit, with further development, an SRS-based wearable solution has the potential to measure motion during gait-based tasks with the accuracy of a 3D motion capture system.