Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

Koby Bibas, Meir Feder, Tal Hassner

פרסום מחקרי: פרק בספר / בדוח / בכנספרסום בספר כנסביקורת עמיתים

תקציר

Detecting out-of-distribution (OOD) samples is vital for developing machine learning based models for critical safety systems. Common approaches for OOD detection assume access to some OOD samples during training which may not be available in a real-life scenario. Instead, we utilize the predictive normalized maximum likelihood (pNML) learner, in which no assumptions are made on the tested input. We derive an explicit expression of the pNML and its generalization error, denoted as the regret, for a single layer neural network (NN). We show that this learner generalizes well when (i) the test vector resides in a subspace spanned by the eigenvectors associated with the large eigenvalues of the empirical correlation matrix of the training data, or (ii) the test sample is far from the decision boundary. Furthermore, we describe how to efficiently apply the derived pNML regret to any pretrained deep NN, by employing the explicit pNML for the last layer, followed by the softmax function. Applying the derived regret to deep NN requires neither additional tunable parameters nor extra data. We extensively evaluate our approach on 74 OOD detection benchmarks using DenseNet-100, ResNet-34, and WideResNet- 40 models trained with CIFAR-100, CIFAR-10, SVHN, and ImageNet-30 showing a significant improvement of up to 15.6% over recent leading methods.

שפה מקוריתאנגלית
כותר פרסום המארחAdvances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021
עורכיםMarc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
עמודים1179-1191
מספר עמודים13
מסת"ב (אלקטרוני)9781713845393
סטטוס פרסוםפורסם - 2021
אירוע35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online
משך הזמן: 6 דצמ׳ 202114 דצמ׳ 2021

סדרות פרסומים

שםAdvances in Neural Information Processing Systems
כרך2

כנס

כנס35th Conference on Neural Information Processing Systems, NeurIPS 2021
עירVirtual, Online
תקופה6/12/2114/12/21

ASJC Scopus subject areas

  • ???subjectarea.asjc.1700.1705???
  • ???subjectarea.asjc.1700.1710???
  • ???subjectarea.asjc.1700.1711???

טביעת אצבע

להלן מוצגים תחומי המחקר של הפרסום 'Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection'. יחד הם יוצרים טביעת אצבע ייחודית.

פורמט ציטוט ביבליוגרפי