Network Calibration by Class-based Temperature Scaling

It is well known that modern neural networks are poorly calibrated. They tend to overestimate or underestimate probabilities when compared to the expected accuracy. This results in misleading reliability and corrupting our decision policy. We show that the amount of calibration error differs across the classes. As a result, we propose to calibrate each class separately. We apply this class-level calibration paradigm to the concept of temperature scaling and describe an optimization method that finds the suitable temperature scaling for each class. We report extensive experiments on a variety of image datasets, and a wide variety of network architectures, and show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.