DataMUX: Data Multiplexing for Neural Networks

Murahari, Vishvak; Jiménez, Carlos E.; Yang, Runzhe; Narasimhan, Karthik

doi:10.48550/arxiv.2202.09318

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The top row shows the results on the test dataset, the bottom row shows the results on the test dataset with augmentations. The base is a pointwise single network, Ensemble [16], MC Dropout [8], DUN [1], EE [22], MIMO [11], DataMux [20], MIMMO γ does not optimize θ and MIMMO.…”

Section: Methodsmentioning

confidence: 99%

“…However, the applicability of [24] to M > 2 is unclear along with its confidence calibration. The authors of [20] have demonstrated the advantages of per-member non-linear encoding and decoding the inputs and outputs of a MIMO network. Nevertheless, their method was aimed primarily towards vision transformers [5] and they were not able to provide substantial gains on CNNs without significant hardware overhead.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MIMMO: Multi-Input Massive Multi-Output Neural Network

Ferianc,

Rodrigues

2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Neural networks (NNs) have achieved superhuman accuracy in multiple tasks, but NNs predictions' certainty is often debatable, especially if confronted with out of training distribution data. Averaging predictions of an ensemble of NNs can recalibrate the certainty of the predictions, but an ensemble is computationally expensive to deploy in practice. Recently, a new hardware-efficient multi-input multi-output (MIMO) NN was proposed to fit an ensemble of independent NNs into a single NN. In this work, we propose the addition of early-exits to the MIMO architecture with inferred depth-wise weightings to produce multiple predictions for the same input, giving a more diverse ensemble. We denote this combination as MIMMO: a multiinput, massive multi-output NN and we show that it can achieve better accuracy and calibration compared to the MIMO NN, simultaneously fit more NNs and be similarly hardware efficient as MIMO or the early-exit ensemble.

show abstract

Section: Methodsmentioning

confidence: 99%