Separating accuracy from forecast certainty: A modified miscalibration measure

Interval forecasting tasks are commonly used to test for forecastoverconfidence. Pointing at deficiencies of the methodology, we advance a modified assignment, where subjects provide point predictions and assess the likelihood of return falling within small intervals around their estimates. The difference between the subjective likelihood assessments and the realized hit rates is advanced as an improved forecast-overprecision measure. Over three incentivized studies, 163 of 195 participants overestimate their hit rates, and a closer look at the data illustrates that inaccuracy and excessive certainty act as distinct sources of overprecision. Applications where the adapted task may prove more powerful than standard interval forecasting are discussed.