David Mumford

Archive for Reprints, Notes, Talks, and Blog

Professor Emeritus
Brown and Harvard Universities

skip to content

Work on Biology and Psychophysics of Vision

Speculation about how the brain works goes back to the earliest cultures but I would argue, it remains in an essentially "pre-DNA" stage. From studying single neurons in isolation or in situ, we know a lot about how neurons work and and what external events tend to cause them to fire. From dissection and from fMRIs, we have a crude "circuit diagram" of the areas of the brain and what sort of thinking is going on many of these areas. Moreover we have some very suggestive psychophysics. But what we don't have is any information about how neurons work collectively, esp. do they form some sort of "cell assemblies" in the course of formulating a thought. In terms of computational simulation of human skills, we have some major successes using raw statistical power but a program which duplicates human powers of reasoning about space, time, converses with humans at length and formulates plans for achieving complex goals eludes us. I hope in the next few decades there will be an experimental and theoretical breakthrough comparable to the understanding of DNA which will transform the field.

My first work in this area was a collaboration with Richard Herrnstein and Steve Kosslyn. Our interest was to explore how well ideas in computer vision modeled animal and human vision and led to four papers, two published, but was cut short by Dick's very untimely death.

  • Discriminating Figure from Ground: the role of edge detection and region growing (with S. Kosslyn, L. Hillger and R. Herrnstein), Proc. Nat. Acad. Sci., 1987, 84, pp.7354-7358. Digital reprint.
  • Analysis and Synthesis of Human and Avian Categorizations of Fifteen Simple Polygons (with R. Herrnstein, S. Kosslyn and W. Vaughan), unpublished. Scanned manuscript.
  • Teaching pigeons the Gujurati alphabet (with S. Kosslyn and R. Herrnstein), unpublished [manuscript?]
  • Teaching Pigeons an Abstract Relational Rule: Insideness (with R. Herrnstein, W. Vaughan and S. Kosslyn), Perception and Psychophysics, 1989, 46, pp. 56-64. Scanned reprint.

My second initiative in biology was a theory of the function of the feedback pathways in mammalian cortex. In the cortex, the projections of the neurons fall into three classes: local axons not leaving grey matter; long axons passing through white matter originating in superficial layers and synapsing in layer IV; long axons passing through white matter originating in deep layers and synapsing in superficial layers. These appear to carry out (i) local computations, (ii) feed-forward information and (iii) feedback information respectively. The generally accepted interpretation of feedback among neuro-physiologists was and is that feedback in sensory areas serves only to focus attention on particular aspects of incoming signals. My view was that feedback was much more central to analyzing signals and that in keeping with Pattern Theory, it carries detailed information on what sort of sensory signals are expected, that is the Bayesian priors built out of memories. The theory also extends to feedback between the cortex and the thalamus, developing ideas of E. Harth. The next set of papers develop this idea.

  • On the Computational Architecture of the Neocortex, I: The role of the thalamo-cortical loop, Biological Cybernetics, 1991, 65, pp.135-145; II: The role of cortico-cortical loops, Biological Cybernetics, 66, pp. 241-251.
    scanned reprints of both parts and DASH part I, DASH part II. Also reprinted in Biology and Computation: a Physicist's Choice, edited by H. Gutfreund and G. Toulouse, World Scientific, 1994, pp. 759-780.
  • Commentary on the article "Banishing the Homunculus" by Horace Barlow, Behavorial and Brain Sciences, 1991.[check pages] Scanned manuscript.
  • Neuronal Architectures for Pattern-theoretic Problems, in Large Scale Neuronal Theories of the Brain, MIT Press, 1994, pp. 125-152. Scanned reprint.
  • Thalamus, in The Handbook of Brain Theory and Neural Networks, M. Arbib editor, MIT Press, 1995. Scanned reprint.
  • Issues in the mathematical modeling of cortical functioning and thought, in The Legacy of Norbert Wiener: A Centennial Symposium, ed. D.Jerison et al, Amer. Math. Society, 1997, pp. 235-260. Scanned reprint.
  • Thalamus, in MIT Encyclopedia of the Cognitive Sciences, MIT Press, 1999.
  • I was very fortunate to have Tai-Sing Lee as a student. He is equally at home with mathematical theory and laboratory experiments and we collaborated on many experiments after he received his PhD. These joint papers are next.

  • Neural correlates of boundary and medial axis representations in primate striate cortex, (with T.S.Lee, K.Zipser & P.H.Schiller), ARVO abstract, 1995. Scanned reprint.
  • Visual Search and Shape from Shading Modulate Contextual Processing in Macaque Early Visual Cortices, (with T.S.Lee, R.Romero, A.Tobias & T.Moore), Neuroscience Abstract, 1997.
  • The Role of V1 in Shape Representation (with Tai Sing Lee, Song Chun Zhu & Victor Lamme), Computational Neuroscience, ed. Bower, Plenum Press, 1997. Scanned reprint.
  • The Role of Primary Visual Cortex in Higher Level Vision (with T.S.Lee, R.Romero and V.Lamme), Vision Research, 38, 1998, 2429-2454.
    Digital reprint and DASH reprint.
  • Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency (with Tai Sing Lee, C.Yang, R.Romero), Nature Neuroscience, 5, 2002, 589-597. Digital reprint.
  • Hierarchical Bayesian Inference in the Visual Cortex, (with Tai Sing Lee), Journal of the Optical Society of America, 20, 2003, 1434-1448.
    Digital reprint and DASH reprint.
  • A final group of papers involved my student Wei Wu, who worked with John Donoghue on training monkeys to perform tasks by thought alone, using an implanted array of electrodes. This has led to major successes in devices enabling paralyzed people to control computers by thought alone.

  • Modeling and Decoding Motor Cortical Activity using a Switching Kalman Filter, (with W.Wu, M.Black, Y.Gao, E.Bienenstock, J.Donoghue), IEEE Trans. on Biomedical Engineering, 51, pp. 933-942, 2004.
    Digital reprint and DASH reprint.
  • Movement Direction Decoding using Fast oscillation in Local Field Potential and Neural Firing, (with Wei Wu, W.Truccolo, M.Saleh and J.Donoghue), 13th Computational Neuroscience Meeting, 2004.

This last paper was a plea for closer collaboration between theoreticians and experimentalists.

  • Minds must unite: It's time for experimentalists to stop ignoring computational modelers (with David Donoho and Bruno Olshausen), `Opinion' section, The Scientist, June 6, 2005. Digital reprint