An intelligent cloud-based neural network algorithm for cross-platform migration and deployment optimization

Abstract

With the emergence of AI application ecology, an increasing number of AI applications are being developed and deployed on end devices. For some applications, due to various reasons (such as delay, bandwidth and privacy issues), inference must be performed on the edge nodes. To realize the efficient deployment of multiple networks on different chips, this paper uses cloud service technology and container resource management technology to achieve cloud deployment and uses a variety of model optimization technologies, such as model format conversion, graph optimization, chip optimization, model low-precision calculation optimization, model cutting and distillation. To achieve the effect of saving considerable memory and reducing energy consumption on the premise of satisfying the accuracy.