In computing, half precision (sometimes called FP16) is a binaryfloating-pointcomputer number format that occupies 16 bits (two bytes in modern computers) in computer memory. It is intended for storage of floating-point values in applications where higher precision is not essential, in particular computer graphics images and neural networks.
Almost all modern uses follow the IEEE 754-2008 standard, where the 16-bit base-2 format is referred to as binary16, and the exponent uses 5 bits. This can express values in the range ±65,504, with the minimum value above 1 being 1 + 1/1024.
Depending on the computer, half-precision can be over an order of magnitude faster than double precision, e.g. 550 PFLOPS for half-precision vs 37 PFLOPS for double precision on one cloud provider.
Several earlier 16-bit floating point formats have existed including that of Hitachi’s HD61810 DSP of 1982, Scott’s WIF and the 3dfx Voodoo Graphics processor.
ILM was searching for an image format that could handle a wide dynamic range, but without the hard drive and memory cost of single or double precision floating point). The hardware-accelerated programmable shading group led by John Airey at SGI (Silicon Graphics) invented the s10e5 data type in 1997 as part of the ‘bali’ design effort. This is described in a SIGGRAPH 2000 paper (see section 4.3) and further documented in US patent 7518615. It was popularized by its use in the open-source OpenEXR image format.
Nvidia and Microsoft defined the halfdatatype in the Cg language, released in early 2002, and implemented it in silicon in the GeForce FX, released in late 2002. Since then support for 16-bit floating point math in graphics cards has become very common.
The F16C extension in 2012 allows x86 processors to convert half-precision floats to and from single-precision floats with a machine instruction.