This paper exploits the advantage of Artificial Neural Network (ANN) to design and implement an improved algorithm of power prediction on heterogeneous computing server with various types of jobs.First, the impact of multi-type of jobs has been taken into account. We divide the jobs into four categories: CPU-intensive, memory-intensive, I/O-intensive and GPU-intensive. We collect the data trace from these jobs by a set of benchmarks.Then, based on the collected data trace, power prediction algorithm has been established, so that developing an improved ANN-based power prediction algorithm with CNN-BiLSTM neural network and attention mechanism.Last, the experiment has been conducted via real-world heterogeneous computing based platform.Compared with the traditional neural network based prediction algorithms and non-neural network based prediction algorithms, our proposed prediction algorithm has better performance in prediction accuracy on power consumption.