Foundations of Learning Theory

Machine Learning relates to many different domains: data analysis, data processing, information retrieval, statistics, knowledge discovery, reasoning under uncertainty, game theory, decision theory...

But the main distinguishing feature of the learning problem is that it requires a form of reasoning which is non-deductive: it requires induction, transduction, abduction or generalization.

As a result, the learning problem has its roots in the following theories

• Philosophy: Incredible military presentation! describing many fundamental ideas about induction. Some are reproduced here
• Formalizing reasoning / Foundational issues
  • Logic W
  • Induction Theory W, Philosophy of induction
  • Inductive Reasoning (Wikipedia)
  • Inquiry Logic of Science (Wikipedia)
  • Abductive Reasoning (Wikipedia)
  • Conjectures Peter Flach's PhD thesis
  • Peter A. Flach. Logical characterisations of inductive learning. In: Handbook of defeasible reasoning and uncertainty management systems, Vol. 4: Abductive reasoning and learning, Dov M. Gabbay and Rudolf Kruse, editors, pages 155--196. Kluwer Academic Publishers, October 2000. pdf
  • F. Huber The Logic of Theory Assessment
  • F. Huber The Logic of Confirmation and Theory Assessment
  • F. Huber The Plausibility-Informativeness Theory

• Formalizing decision-making and interactions
  • Decision Theory W
  • Auction Theory W
  • Rational Choice Theory, Public choice theory, Social choice theory, General equilibrium theory W...
  • Portfolio Theory W, Sharpe ratio W
  • Theory of Risk Aversion W, Utility W
  • Game Theory W, Mechanism Design W, What is Game Theory?, by D. K. Levine.
  • Finance, Gambling Theory W, Arbitrage W, Prediction Markets W, Efficient market theory W, Value-at-Risk W, Hedge W
  • Interpretation of Probability W

• Formalizing information and regularity
  • Information Theory W
  • Algorithmic Information Theory W
  • Compression W, Universal compression
  • Randomness W

• Formalizing Inference
  • Logic Inference
  • Inductive Inference W
  • Statistical Inference
  • Estimation Theory

• Tools
  • Probability Theory
  • Optimization Theory
  • Algorithms, Computation Theory
  • Functional Analysis
  • Approximation Theory