|
|
|
Link to Pattern Theory GroupLink to my recent Courses Link to some
Publications available online by my students and me |
I
work in the area of machine and natural intelligence, following the
statistical approach pioneered by Ulf Grenander, which we call "Pattern
Theory". In this approach, thinking is modeled as statistical inference
rather than logic and learning results from the accumulation of massive data
from interactions with the world. My work concentrates on visual perception
which seems to be more approachable than high-level thinking, more complex
than auditory and tactile perception yet has been solved by three distinct
classes of animals (cephalopods, birds and mammals) so it can't be that hard!
The aspect of this research with which I have been most involved is the
construction of probability models for the variables of vision: direct models
of the raw images, the shapes of objects, the texture of their surfaces etc.
A second question is how to sample and estimate with these models, e.g.
compute conditional means and modes. Many approaches involve computing with
one or more samples from the distribution which evolve deterministically or
stochastically, smoothly or with jumps. Sampling from the distribution can be
viewed as feedback, i.e. prior knowledge of high level structures guides the
reconstruction of an image. A third set of questions concerns how such
statistical estimation may be performed in cortex, in neural nets with
feedback. A major open question here is whether the information being handled
by neurons is contained only in their firing rates or in the precise timing
and synchrony of their spikes in the full network. Another is how cortex
handles ambiguity, the need for maintaining multiple hypotheses while waiting
for more data.  Recent Vision Projects ·
Studies of
the infinite dimensional manifold of all 2D shapes, various metrics
that can be put on this manifold and their geodesics and curvature, as well as its epsilon-entropy with Kathryn
Leonard, Michael Miller, Tilak Ratnanather and Peter Michor. ·
Studies of
3D surface shape, principle curvatures and axes, plus applications of these
ideas to faces and potsherds, with Gaile Gordon, Peter Giblin,
Conglin Lu and Yan Cao. See Surface
Evolution under Curvature Flows, with Conglin Lu and Yan Cao and
Extracting the Symmetric Axis of Small Fragments of Pots, with Yan Cao. ·
Empirical
and modeling studies of Optical and Range Image Statistics, with Jinggang
Huang, Ann Lee, Kim Pedersen and Basilis Gidas. ii) Huang's thesis summarizing
the above and with more results, esp. an Ising-type model for images. iii) Statistics of range images , with James Huang and Ann Lee, iv) Stochastic
Models for Generic Images with Basilis Gidas, vii) Kim Pedersen and Ann Lee. Toward a Full
Probability Model of Edges in Natural Images ·
Studies of
cortex, especially a) modeling the effects of electrical junctions between
inhibitory neurons, b) possible cortical mechanisms involving feedback and
particle filters and c) use of local field potentials in decoding motor
actions, with T. S. Lee and Wei Wu.  Survey Articles in Vision and Pattern Theory ·
The Bayesian Rationale for Energy Functionals
, in "Geometry-driven Diffusion in Computer Vision", Bart
Romeny editor, Kluwer Academic Publ., 1994, pp. 141-153. ·
Pattern theory: a
unifying perspective, in "Proceedings of the 1st European Congress of
Mathematics", Birkhauser-Boston, 1994, and in revised form in
"Perception as Bayesian Inference", edited by D.Knill and
W.Richards, Cambridge Univ. Press, 1996. ·
The Statistical Description of Visual Signals , in
"ICIAM 95", edited by K.Kirchgassner, O.Mahrenholtz and
R.Mennicken, Akademie Verlag, 1996. · Pattern Theory: The Mathematics of Perception, in “Proceedings of the International Congress of Mathematicians, Beijing, 2002”, vol. 1, Higher Educ. Press, Bejing, 2002. Books in Vision and Pattern
Theory ·
Filtering,
Segmentation and Depth, (with M. Nitzberg and T. Shiota), Springer Lecture
Notes in Computer Science vol. 662, 1993. ·
Two and Three
Dimensional Patterns of the Face, with P.Hallinan, G.Gordon, A.Yuille and
P.Giblin, AKPeters, 1999. ·
Pattern Theory
through Examples, with Agnes Desolneux, in preparation. Selected Articles in Vision ·
Optimal
Approximations of Piecewise Smooth Functions and Associated Variational
Problems (with J. Shah), Comm. in Pure and Appl. Math., 1989, vol. 42. ·
Mathematical Theories
of Shape: do they model perception?, in "Proc. Conference 1570",
Soc. Photo-optical & Ind. Engineers, 1991, pp. 2-10. ·
Minimax Entropy
Principle and its Application to Texture Modeling (with S-C Zhu and Y.Wu),
Neural Computation, vol.9, 1997, pp.1627-1650. · Stochastic Models of Generic Images (with B.Gidas), Quarterly Appl. Math., vol.59, 2001, pp.85-111. Selected Articles in Biology ·
On the Computational
Architecture of the Neocortex, I: The role of the thalamo-cortical loop; II:
The role of cortico-cortical loops, Biological Cybernetics, 1991, vols. 65,
66. ·
Neuronal Architectures for
Pattern-theoretic Problems , in "Large-Scale Neuronal Theories of
the Brain", C.Koch & J.Davis, editors, MIT Press, 1994, pp.125-152. ·
The Mathematical Modeling
of Cortical Functioning and Thought, in "Proc. Norbert Wiener Centennial
Conference", Amer. Math. Society, 1997. ·
The Role of Primary
Visual Cortex in Higher Level Vision (with T.S.Lee, R.Romero and Victor
Lamme), Vision Research, vol.38, 1998, pp.2429-2454. ·
Neural Activity in
early visual cortex reflects behavioral experience and higher-order
perceptual saliency (with T.S.Lee, C.Yang and R.Romero), Nature Neuroscience,
vol.5, 2002, pp.589-597. ·
Hierarchical
Bayesian Inference in the Visual System (with T.S.Lee), submitted to J.
Optical Soc. Amer. Older Vision Projects (1983-1998) ·
Exponential models
and the statistics of textures, with Song-Chun Zhu . See Zhu's home page and annotated
publications ·
The representation of
higher level features and the role of feedback in primary visual cortex V1,
with Tai Sing Lee . See Lee's
publications. ·
Mixed Markov Models,
with Artur Fridman . See his thesis
in PDF and his recent
article. ·
Face Recognition
with flexible templates and lighting variation, with Peter Hallinan and
Alan Yuille ·
Stereo Image Matching
with Half-occluded Pixels, with Peter Belhumeur ·
Texture Segmentation
with varying spatial frequency spectrum, with Tai Sing Lee ·
2.1D or Layered Image
Representations and Gestalt Perception, with Mark Nitzberg and Taka Shiota ·
The Principle
Curvatures of Faces, with Gaile
Gordon, ·
The Transport Metric
and Classification of Leaves, with David Fry ·
Variational Methods
for Image Segmentation, with Jayant Shah and Yang Wang ·
Shape Categorization
in Humans and Birds, with Richard Herrnstein and Steven Kosslyn . Selected Algebraic Geometry Publications
(1959-1982): ·
Geometric
Invariant Theory, Springer-Verlag,
1965; 2nd enlarged edition, (with J. Fogarty), 1982; 3rd enlarged edition,
(with F. Kirwan and J. Fogarty), 1994. ·
On the Equations
Defining Abelian Varieties I, II, III, Inv. Math., vol. 1, 1966, and vol. 3, 1967. ·
Enriques' Classification
of Surfaces in Char. p, I, in
Global Analysis, Spencer and Iyanaga editors, U. of Tokyo Press, 1969; II and
III (with E. Bombieri), in Complex Analysis and Algebraic Geometry, Baily and
Shioda editors, Cambridge Univ. Press, 1977, and Invent. Math., 1976, 35. ·
The Irreducibility
of the Space of Curves of Given Genus (with P. Deligne), Publ. Math. de l'I.H.E.S., 1969, vol. 36. ·
Abelian Varieties, Oxford University Press, 1st edition 1970, 2nd
edition 1974. ·
The Structure of the
Moduli Spaces of Curves and Abelian Varieties, Congress Int. du Math., Nice,
1970. ·
Algebraic Geometry
I: Complex Projective Varieties,
Springer-Verlag, New York, 1976. ·
Tata Lectures on
Theta (with C. Musili, M. Nori, P.
Norman, E. Previato and M. Stillman), Birkhauser-Boston, Part I, 1982, Part
II, 1983, Part III, 1991. ·
On the Kodaira
Dimension of the Moduli Space of Curves (with J. Harris), Inv. Math., 1982. |
