An architecture entropy regularizer for differentiable neural architecture search

Jing, Kun; Chen, Luoyu; Jin, Xu

doi:10.1016/j.neunet.2022.11.015

Cited by 13 publications

(2 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The type of physics-based knowledge vary across methods, for example, (a) a dictionary of basis functions (e.g., sin, cos, d dt ) (Schmidt & Lipson, 2009;Brunton et al, 2016;Martius & Lampert, 2016;Raissi, 2018;Cranmer et al, 2020b) related to the task, (b) a completely specified physics model (Raissi et al, 2017a;Raissi, 2018;Jiang et al, 2019) or with missing terms (Yin et al, 2021), and (c) different domain-specific physical constraints such as energy conservation (Greydanus et al, 2019;Cranmer et al, 2020a), symmetries (Wang et al, 2020bFinzi et al, 2021;Brandstetter et al, 2022a). While these PIML methods improve upon standard neural networks, Figure 2 shows that they are generally not designed for OOD forecasting tasks.…”

Section: Physics-informed Machine Learning (Piml)mentioning

confidence: 99%

Physics-Informed Machine Learning for Accelerated Testing of Roll-to-Roll Printed Sensors

Mouli

Sedaghat

Oduncu

et al. 2022

Volume 2: Manufacturing Processes; Manufacturing Systems

View full text Add to dashboard Cite

Roll-to-roll printing has significantly shortened the time from design to production of sensors and IoT devices, while being cost-effective for mass production. But due to less manufacturing tolerance controls available, properties such as sensor thickness, composition, roughness, etc., cannot be precisely controlled. Since these properties likely affect the sensor behavior, roll-to-roll printed sensors require validation testing before they can be deployed in the field. In this work, we improve the testing of Nitrate sensors that need to be calibrated in a solution of known Nitrate concentration for around 1–2 days. To accelerate this process, we observe the initial behavior of the sensors for a few hours, and use a physics-informed machine learning method to predict their measurements 24 hours in the future, thus saving valuable time and testing resources. Due to the variability in roll-to-roll printing, this prediction task requires models that are robust to changes in properties of the new test sensors. We show that existing methods fail at this task and describe a physics-informed machine learning method that improves the prediction robustness to different testing conditions (≈ 1.7× lower in real-world data and ≈ 5× lower in synthetic data when compared with the current state-of-the-art physics-informed machine learning method).

show abstract

Section: Physics-informed Machine Learning (Piml)mentioning

confidence: 99%

Physics-Informed Machine Learning for Accelerated Testing of Roll-to-Roll Printed Sensors

Mouli

Sedaghat

Oduncu

et al. 2022

Volume 2: Manufacturing Processes; Manufacturing Systems

View full text Add to dashboard Cite

show abstract

“…The majority of works in the field of NAS use either RL or EA. However, with the growing interest in this research field, other techniques have been proposed, such as Bayesian Optimization [55], Monte Carlos Tree Search [56,57], Hill Climbing Algorithm [58] and Gradient-based optimization [59,60,61,62,63] for a continuos-relaxed version of the search space.…”

Section: Other Methodsmentioning

confidence: 99%

Quantum-Inspired Neural Architecture Search Applied to Semantic Segmentation

G¹

View full text Add to dashboard Cite

Deep neural networks are responsible for great progress in performance for several perceptual tasks, especially in the fields of computer vision, speech recognition, and natural language processing. These results produced a paradigm shift in pattern recognition techniques, shifting the demand from feature extractor design to neural architecture design. However, designing novel deep neural network architectures is very time-consuming and heavily relies on experts' intuition, knowledge, and a trial and error process. In that context, the idea of automating the architecture design of deep neural networks has gained popularity, establishing the field of neural architecture search (NAS). To tackle the problem of NAS, authors have proposed several approaches regarding the search space definition, algorithms for the search strategy, and techniques to mitigate the resource consumption of those algorithms. Q-NAS (Quantuminspired Neural Architecture Search) is one proposed approach to address the NAS problem using a quantum-inspired evolutionary algorithm as the search strategy. That method has been successfully applied to image classification, outperforming handcrafted models on the CIFAR-10 and CIFAR-100 datasets and also on a real-world seismic application. Motivated by this success, we propose SegQNAS (Quantum-inspired Neural Architecture Search applied to Semantic Segmentation), which is an adaptation of Q-NAS applied to semantic segmentation. We carried out several experiments to verify the applicability of SegQNAS on two datasets from the Medical Segmentation Decathlon challenge. SegQNAS was able to achieve a 0.9583 dice similarity coefficient on the spleen dataset, outperforming traditional architectures like U-Net and ResU-Net and comparable results with a similar NAS work from the literature but with fewer parameters network. On the prostate dataset, SegQNAS achieved a 0.6887 dice similarity coefficient, also outperforming U-Net, ResU-Net, and outperforming a similar NAS work from the literature.

show abstract