Internet of Thing (IoT) systems usually have less storage and computing power than desktop systems. This paper proposes an efficient BGV-type homomorphic encryption scheme in order fit for secure computing on IoT system. Our scheme reduces the storage space for switch keys and ciphertext evaluation time comparing with previous BGV-type cryptosystems. Specifically, the switch key in homomorphic computations can be a constant but no longer one for each level. Moreover, the product of two ciphertexts can be at the same sublayer as them and the multiplication operations can be repeated between two sublayers. As a result, the multiplication times will not be limited by L in an L-level circuit and, thus, the ciphertext evaluation time will decrease significantly. We implement the scheme with the C language. The performance test shows that the efficiency of the improved scheme is better than Helib in same configurations.