Analysis of Posit and Bfloat Arithmetic of Real Numbers for Machine Learning

Abstract

Modern computational tasks are often required to not only guarantee predefined accuracy, but get the result fast. Optimizing calculations using floating point numbers, as opposed to integers, is a nontrivial task. For this reason, there is a need to explore new ways to improve such operations. This paper presents analysis and comparison of various floating point formatsfloat, posit and bfloat. One of the promising areas in which the problem of using such values can be considered to be the most acute is neural networks. That is why we pay special attention to algorithms of linear algebra and artificial intelligence to assess efficiency of new data types in this area. The research results showed that software implementations of posit16 and posit32 have high accuracy, but they are not particularly fast; on the other hand, bfloat16 is not much different from float32 in accuracy, but significantly surpasses it in performance for large amounts of data and complex machine learning algorithms. Thus, posit16 can be used in systems with less stringent performance requirements, as well as in conditions of limited computer memory; and also in cases when bfloat16 cannot provide required accuracy. As for bfloat16, it can speed up systems based on the IEEE 754 standard, but it cannot solve all the problems of conventional floating point arithmetic. Thus, although posits and bfloats are not a full fledged replacement for float, they provide (under certain conditions) advantages that can be useful for implementation of machine learning algorithms.

Energy-efficient neural network learning with accuracy-adjustable floating-point multiplier

“…The advantage of representing the numerical value O in the bfloat16 format is, that it keeps one sign bit s(O) and the 8-bit exponent e(O) equal to the IEEE 754 single-precision floatingpoint format but shortens the mantissa m(O) to 7 bits. Thus, it enables using tiny numerical values, important in the neural network learning phase [18] for example. While the multiplier determines the sign and the exponent exactly, it follows the idea of the approximate iterative logarithmic multiplier to compute the mantissa.…”

Energy-efficient neural network learning with accuracy-adjustable floating-point multiplier

“…The advantage of representing the numerical value O in the bfloat16 format is, that it keeps one sign bit s(O) and the 8-bit exponent e(O) equal to the IEEE 754 single-precision floatingpoint format but shortens the mantissa m(O) to 7 bits. Thus, it enables using tiny numerical values, important in the neural network learning phase [18] for example. While the multiplier determines the sign and the exponent exactly, it follows the idea of the approximate iterative logarithmic multiplier to compute the mantissa.…”