Mathematical Foundations of Learning Theory

Machine Learning relates to many different domains: data analysis, data processing, information retrieval, statistics, knowledge discovery, reasoning under uncertainty...

But the main distinguishing feature of the learning problem is that it requires a form of reasoning which is non-deductive: it requires induction, transduction, abduction or generalization.

As a result, the learning problem has its roots in the following theories

• Philosophy: Incredible military presentation! describing many fundamental ideas about induction. Some are reproduced here
• Logic: theory of induction
  • Inductive Reasoning (Wikipedia)
  • Inquiry Logic of Science (Wikipedia)
  • Abductive Reasoning (Wikipedia)
  • Conjectures Peter Flach's PhD thesis
  • Peter A. Flach. Logical characterisations of inductive learning. In: Handbook of defeasible reasoning and uncertainty management systems, Vol. 4: Abductive reasoning and learning, Dov M. Gabbay and Rudolf Kruse, editors, pages 155--196. Kluwer Academic Publishers, October 2000. pdf
  • F. Huber The Logic of Theory Assessment
  • F. Huber The Logic of Confirmation and Theory Assessment
  • F. Huber The Plausibility-Informativeness Theory

• Decision Theory
  • Rational Choice Theory (Wikipedia), Decision Theory (Wikipedia), General equilibrium theory (Wikipedia)
• Probability Theory
• Statistics
• Estimation Theory
• Approximation Theory
• Game Theory
  • Wikipedia
  • What is Game Theory?, by D. K. Levine.
• Inductive Inference
  • Wikipedia
• Computation Theory

Another similar list (to be merged with the previous one)

• Formalizing reasoning / Foundational issues
  • Logic W
  • Induction Theory W, Philosophy of induction

• Formalizing decision-making and interactions
  • Decision Theory W
  • Auction Theory W
  • Rational Choice Theory, Public choice theory, Social choice theory...
  • Portfolio Theory W, Sharpe ratio W
  • Theory of Risk Aversion W, Utility W
  • Game Theory W, W
  • Finance, Gambling Theory W, Arbitrage W, Prediction Markets W, Efficient market theory W, Value-at-Risk W, Hedge W
  • Interpretation of Probability W

• Formalizing information and regularity
  • Information Theory W
  • Algorithmic Information Theory W
  • Compression W, Universal compression
  • Randomness W

• Formalizing Inference
  • Logic Inference
  • Inductive Inference
  • Statistical Inference

• Tools
  • Probability Theory
  • Optimization Theory
  • Algorithms
  • Functional Analysis