Most deep neural networks deployed today are trained using GPUs via high-level frameworks such as Tensor-Flow [1] and PyTorch [2]. This paper describes changes we made to the GPGPU-Sim simulator [3], [4] to enable it to run PyTorch by running PTX kernels included in NVIDIA's cuDNN [5] library. We use the resulting modified simulator, which has been made available publicly with this paper 1 , to study some simple deep learning workloads. With our changes to GPGPU-Sim's functional simulation model we find GPGPU-Sim performance model running a cuDNN enabled implementation of LeNet for MNIST reports results within 30% of real hardware. Using GPGPU-Sim's AerialVision performance analysis tool we observe that cuDNN API calls contain many varying phases and appear to include potentially inefficient microarchitecture behavior such as DRAM partition bank camping, at least when executed on GPGPU-Sim's current performance model.

No abstract

No abstract