BITCOS squeezes ternary AI weights to 1.485 bits

BITCOS squeezes ternary AI weights to 1.485 bits

BITCOS reaches 1.485 bits per weight on the sparsest of 29 tested models, Evangelos Georganas, Alexander Heinecke and Pradeep Dubey report in a 14 September preprint. The presence-bitmap plus sign-vector layout records which ternary {-1,0,+1} weights are nonzero separately from their signs, under the log2(3) ≈1.585 reference and 1.625-bit five-trit packing.

Published

Read at another depth