Work in Pattern Theory
Several essential ideas brought me to realize how Grenander's Pattern Theory was the right way to understand almost all cognitive skills and especially vision. One was the emphasis on pattern synthesis as well as pattern analysis. Bayesian inference requires you construct a prior probability model of whatever signals or situations you are modeling and you should always test your prior by sampling to see which features it models accurately and which not. When I first got involved, all speech recognition algorithms were based on finding the invariant features of phonemes, independent of the speaker. Naturally you couldn't synthesize from such algorithms -- they had thrown away the raw signal. Humans, of course, not only understand speech, they recognize hundreds of individuals from their distinctive voices. Progress was made in speech recognition when these distinctive voices began to be modeled and stochastic models for each phoneme (or triphone) and each speaker type were crafted. You can sample from these and see how much of the signal's features you have modeled.
A second revelation was that natural signals given by functions f vary not only by random additive perturbations but often by composition with random rearrangements of their domain. The resulting probability distibution in the vector space of signals is nothing like Gaussian. Its support is usually a twisted snakey submanifold. This puts the lie to all simplistic Gaussian pattern recognition algorithms.
A third realization was that graphical structures were everywhere in the representations of ideas in cognitive domains. Roget's Thesaurus, semantic nets, linguistic grammars were all based on graphs of various kinds and vision was not going to an exception.
I have given a series of expository talks and written expository articles to preach this gospel. These are listed below, culminating in my book with Agnes Desolneux. That book is written for grad students and colleagues and develops the mathematical tools and the algorithms along with pattern theory itself. The background on this page is an amazing Islamic inlaid door with pentagonal symmetry. The full image has amazing decagons which this page is too small to contain.
- Parametrizing Exemplars of Categories, J. Cognitive Neuroscience, 1991, 3, pp. 87-88. Scanned reprint.
- Commentary on Grenander & Miller "Representations of Knowledge in Complex Systems", Proc. Royal Stat. Soc., 1994.[check pages] Scanned manscript.
- Pattern Theory: a Unifying Perspective, in Proceedings 1st European Congress of Mathematics, Birkhauser-Boston, 1994. Revised version in Perception as Bayesian inference, ed. D.Knill and W.Richards, Cambridge Univ. Press, 1996, pp. 25-62. Scanned reprint.
- The Bayesian Rationale for Energy Functionals, in Geometry-Driven Diffusion in Computer Vision, Bart Romeny editor, Kluwer Academic, 1994, pp. 141-153. Scanned reprint.
- Two and Three dimensional Patterns of the Face, (with Peter Giblin, Gaile Gordon, Peter Hallinan and Alan Yuille), AKPeters, 1999.
- Pattern Theory: The Mathematics of Perception, in Proceedings of ICM 2002, Beijing, 1, 2002, pp.401-422. This talk was aimed at explaining to the pure and applied math community some of the mathematical ideas behind the Pattern Theory approach.
Digital reprint and ArXiv reprint.
- Pattern Theory, the Stochastic Analysis of Real World Signals, (with Agnes Desolneux), AKPeters/CRC Press, 2010. The book is available at Amazon.