Permutation Invariant Individual Batch Learning

Yaniv Fogel, Meir Feder

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper considers the individual batch learning problem. Batch learning (in contrast to online) refers to the case where there is a "batch"of training data and the goal is to predict a test outcome. Individual learning refers to the case where the data (training and test) is arbitrary, individual. This batch individual setting poses a fundamental issue of defining a plausible criterion for a universal learner since in each experiment there is a single test sample. We propose a permutation invariant criterion that, intuitively, lets the individual training sequence manifest its empirical structure for predicting the test sample. This criterion is essentially a min-max regret, where the regret is based on a leave-one-out approach, minimized over the universal learner and maximized over the outcome sequences (thus agnostic). To show its plausibility, we analyze the criterion and its resulting learner for two cases: Binary Bernoulli and 1-D deterministic barrier. For both cases the regret behaves as O(c/N), N the size of the training and c = 1 for the Bernoulli case and log4 for the 1-D barrier. Interestingly, in the Bernoulli case, the regret in the stochastic setting behaves as O(1/2N) while here, in the individual setting, it has a larger constant.

Original languageEnglish
Title of host publication2023 IEEE Information Theory Workshop, ITW 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages142-146
Number of pages5
ISBN (Electronic)9798350301496
DOIs
StatePublished - 2023
Event2023 IEEE Information Theory Workshop, ITW 2023 - Saint-Malo, France
Duration: 23 Apr 202328 Apr 2023

Publication series

Name2023 IEEE Information Theory Workshop, ITW 2023

Conference

Conference2023 IEEE Information Theory Workshop, ITW 2023
Country/TerritoryFrance
CitySaint-Malo
Period23/04/2328/04/23

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Signal Processing
  • Control and Optimization

Fingerprint

Dive into the research topics of 'Permutation Invariant Individual Batch Learning'. Together they form a unique fingerprint.

Cite this