rwightman/pytorch-image-models: v0.4.12. Vision Transformer AugReg support and more

Wightman, Ross; Arora, Aman; Ha, Chris; Mike_, _; contrastive,; morizin,; Kertész, Csaba; Mehta, Dushyant; Cucurull, Guillem; Singh, Kushajveer; Soare, Alexander; Lavin, Andrew; Zhuang, Jian; Hollemans, Matthijs; Shults, Vyacheslav; Uchida, Yusuke; Zhong, Zhun; Comar,; Kim, Tae-Hoon; Chernov, Alexey; Han, Greg Dongyoon; Uriegas, Eli; Jasha,; Kardas, Marcin; Monashev, Michael; Chen, Min‐Qin; Rashad, Mohamed; Mu, Norman

doi:10.5281/zenodo.5047297

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The average of the accuracies on the testing set over all folds is used to evaluate the classification performance. All the compared models are implemented by open-sourced project Timm [60], and ImageNet pre-trained weights are used to finetune and boost the model training. For each model, we train 50 epochs by using AdamW optimizer and a cosine decay learning rate scheduler.…”

Section: A Experimental Configurationmentioning

confidence: 99%

The self-supervised spectral-spatial attention-based transformer network for automated, accurate prediction of crop nitrogen status from UAV imagery

Zhang¹,

Han²,

Sobeih³

et al. 2021

Preprint

View full text Add to dashboard Cite

Nitrogen (N) fertiliser is routinely applied by farmers to increase crop yields. At present, farmers often over-apply N fertilizer in some locations or timepoints because they do not have high-resolution crop N status data. N-use efficiency can be low, with the remaining N lost to the environment, resulting in high production costs and environmental pollution. Accurate and timely estimation of N status in crops is crucial to improving cropping systems' economic and environmental sustainability. The conventional approaches based on tissue analysis in the laboratory for estimating N status in plants are time consuming and destructive. Recent advances in remote sensing and machine learning have shown promise in addressing the aforementioned challenges in a non-destructive way. We propose a novel deep learning framework: a channel-spatial attention-based vision transformer (CSVT) for estimating crop N status from large images collected from a UAV in a wheat field. Unlike the existing works, the proposed CSVT introduces a Channel Attention Block (CAB) and a Spatial Interaction Block (SIB), which allows capturing nonlinear characteristics of spatial-wise and channel-wise features from UAV digital aerial imagery, for accurate N status prediction in wheat crops. Moreover, since acquiring labeled data is time consuming and costly, local-to-global self-supervised learning is introduced to pre-train the CSVT with extensive unlabelled data. The proposed CSVT has been compared with the state-of-the-art models, tested and validated on both testing and independent datasets. The proposed approach achieved high accuracy (0.96) with good generalizability and reproducibility for wheat N status estimation.

show abstract

Section: A Experimental Configurationmentioning

confidence: 99%