GradFreeBits: Gradient-Free Bit Allocation for Mixed-Precision Neural Networks

Benjamin Jacob Bodner, Gil Ben-Shalom, Eran Treister

Research output: Contribution to journalArticlepeer-review


Quantized neural networks (QNNs) are among the main approaches for deploying deep neural networks on low-resource edge devices. Training QNNs using different levels of precision throughout the network (mixed-precision quantization) typically achieves superior trade-offs between performance and computational load. However, optimizing the different precision levels of QNNs can be complicated, as the values of the bit allocations are discrete and difficult to differentiate for. Moreover, adequately accounting for the dependencies between the bit allocation of different layers is not straightforward. To meet these challenges, in this work, we propose GradFreeBits: a novel joint optimization scheme for training mixed-precision QNNs, which alternates between gradient-based optimization for the weights and gradient-free optimization for the bit allocation. Our method achieves a better or on par performance with the current state-of-the-art low-precision classification networks on CIFAR10/100 and ImageNet, semantic segmentation networks on Cityscapes, and several graph neural networks benchmarks. Furthermore, our approach can be extended to a variety of other applications involving neural networks used in conjunction with parameters that are difficult to optimize for.

Original languageAmerican English
Article number9772
Issue number24
StatePublished - 1 Dec 2022


  • gradient-free optimization
  • mixed-precision quantization
  • neural network compression
  • quantization

All Science Journal Classification (ASJC) codes

  • Analytical Chemistry
  • Information Systems
  • Instrumentation
  • Atomic and Molecular Physics, and Optics
  • Electrical and Electronic Engineering
  • Biochemistry


Dive into the research topics of 'GradFreeBits: Gradient-Free Bit Allocation for Mixed-Precision Neural Networks'. Together they form a unique fingerprint.

Cite this