Offtopia » Machine Learning

anglican.ml

dvd — Fri, 07 Oct 2016 13:14:30 +0000

http://anglican.ml/, the proper domain for the Anglican way of machine learning.

Immanuel Kant and Probability

dvd — Thu, 08 Oct 2015 20:18:21 +0000

Kant said: there are two a priori intuitions — space and time. There are also categories, and “the number of the categories in each class is always the same, namely, three”, like unity-plurality-modality, or possibility-existence-necessity. It would be fun to have three a priori intuitions, but only two exist, sigh. Really though?

Kant probably did not realize: there is a third one — probability, to wit, certainty of our experience. Just like space, probability precedes any experience. Every object is uncertain as much as it is extended.

The three a priori intuitions are related — infinite and undirected space, infinite and directed time, finite and undirected probability. Physics knows of uncertainty principle, we are uncertain about relation of time and space: both time and space cannot be intuited with certainty. Probability is as basic and fundamental as time and space for our cognition.

Just like geometry deals with a priori intuition of space, and mathematical analysis — with intuition of time, theory of probability deals with intuition of probability. There is philosophical justification for studying uncertainty, probability, and bayesian inference.

Maximum a Posteriori Estimation by Search in Probabilistic Programs

dvd — Wed, 10 Jun 2015 20:33:25 +0000

Paper, slides, and poster as presented at SOCS 2015.

We introduce an approximate search algorithm for fast maximum a posteriori probability estimation in probabilistic programs, which we call Bayesian ascent Monte Carlo (BaMC). Probabilistic programs represent probabilistic models with varying number of mutually dependent finite, countable, and continuous random variables. BaMC is an anytime MAP search algorithm applicable to any combination of random variables and dependencies. We compare BaMC to other MAP estimation algorithms and show that BaMC is faster and more robust on a range of probabilistic models.

Path Finding under Uncertainty through Probabilistic Inference

dvd — Mon, 08 Jun 2015 07:43:06 +0000

An early workshop paper, superseded by current research but still relevant, slides, and a poster.

Abstract

We introduce a new approach to solving path-finding problems under uncertainty by representing them as probabilistic models and applying domain-independent inference algorithms to the models. This approach separates problem representation from the inference algorithm and provides a framework for efficient learning of path-finding policies. We evaluate the new approach on the Canadian Traveller Problem, which we formulate as a probabilistic model, and show how probabilistic inference allows efficient stochastic policies to be obtained for this problem.

Anglican the Probabilistic Programming Concept

dvd — Tue, 05 May 2015 22:44:57 +0000

Anglican is a probabilistic programming language, or better yet, a concept, living in symbiosis with Clojure. Anglican stands for Church of England (because we are here in Oxford). To create your Turing-complete probabilistic models, clone anglican-user and hack away. Or, look at cool examples.

Output-Sensitive Adaptive MH for Probabilistic Programs

dvd — Wed, 10 Dec 2014 10:20:18 +0000

A poster for the 3rd NIPS Workshop on Probabilistic Programming; also available as A0 PDF. Slides for a 15-minute talk.

Abstract

We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve convergence of the program output. We show that AdLMH converges to the correct equilibrium distribution and compare convergence of AdLMH to that of LMH on several test problems to highlight different aspects of the adaptation scheme. We observe consistent improvement in convergence on the test problems.

Full paper.

Slides for my Tea Talk

dvd — Wed, 01 Oct 2014 22:55:22 +0000

My Tea Talk slides, on October 1st, 2014.