
|

|
Cognitive science is currently a federation of disciplines—psychology,
AI, linguistics, logic, neuroscience,
etc.—lacking a unified theory. Cognitive models are broadly divided
between a logical paradigm, or "cognitivism", and a dynamical
paradigm, or "connectionism". Bridging this gap will
require an intermediate or mesoscopic level of description, at a
finer granularity than symbols but larger structural complexity
than small artificial neural nets.
|
Logical systems define high-level
symbols and formal grammars, but do not have a microstructure needed
to explain the fuzzy complexity of perception, memory or learning.
Conversely, dynamical systems define functionality as emerging from
neural networks and low-level activation equations, but lack a
macroscopic level supporting the systematic symbolic composition of
language and reasoning.
In sum, it is as if cognitive science found itself in the same situation
as biology a century ago: back then, evolution and genetics were both
recognized but still uncorrelated scientifically. It is only later, when
DNA, RNA and proteins were discovered, that these two levels became finally
linked and united through the Modern Synthesis. In other terms, by
elucidating the mesoscopic level of life's complex self-organization
(molecular biology), macroscopic phenomena (heredity, speciation)
could be explained on the basis of microscopic elements
(atoms, molecules; Fig. a).
|
|
Likewise, metaphorically speaking, we should ask ourselves what the
"molecules" of cognitive science's future Mind-Brain Modern Synthesis
will be. In theory, what composite and complex objects could possibly explain
the laws of perception and language as an emergence from of elementary
neuronal activities? (Fig. b). The structure and properties of
cognitive states has long been a subject of debate since the beginnings
of modern neuroscience. It is generally assumed that the average
firing rate of "cell assemblies" constitute an important part of
the neural code. In contrast, following the tracks of my postdoc and
thesis advisors, Christoph von der Malsburg (1981) and Elie Bienenstock,
I have promoted temporal coding as another
important format of representation, one that involves higher-order
moments or "correlations" among neuronal activities.
Generally, if xi(t) denotes the
time-varying potential of neuron i, the answer could lie
in large dynamic groups { x1(t), ...,
xn(t) } characterized by spike timing series
{ t11, t12,
t13, ...,
tn1, tn2,
tn3, ... }, which
contain many high-order moments
<xi(t)
xj(t - τij)
xk(t - τik) ...>,
typically combinations of synchronized groups
(where delays τij = 0) and waves or "rhythms"
(where delays τij > 0). Called spatiotemporal patterns
(STPs), these reproducible correlations among electric signals
must also be supported by underlying regular patterns of connectivity.
Similarly to proteins, STPs could then interact in several ways and
assemble ("bind") at several levels, forming a hierarchy of composite structures
from simpler ones, like "building blocks" of intelligent behavior
(Bienenstock 1996). My own research goal is to outline a new theoretical
framework for this type of complex mesoscopic neurodynamics with
compositional properties. I have conducted different studies (see below)
that all construe the cortical substrate of neuronal units and synaptic
contacts as an "excitable medium" and have potential applications to
of artificial systems in perceptual, linguistic or behavioral tasks.
|
References
Bienenstock (1996) Composition. In Brain Theory: pp. 269-300. Elsevier.
von der Malsburg (1981) The Correlation Theory of Brain Function. Max-Planck Inst.
|
Publication
|
|
|
Parallel self-organization of connectivity and activity in an
initially random spiking neural network, to support a
hierarchy of structured representations applied to visual,
auditory or linguistic tasks
|
 |
Striking regularities in the connectivity structure of
the visual system and other cortical areas account for their
functional specialization.
Elie Bienenstock and myself have designed a
neural network model that reproduces the development of such
regularities as a phenomenon of spatiotemporal pattern
formation. We show the spontaneous and simultaneous emergence
of regular chains of synaptic connectivity together with a
wave-like propagation of neural activity.
|
Starting from an undifferentiated random state, our dynamical,
self-reconfiguring neural network transitions into an ordered
regime of activity and connectivity. In this regime, chains
sustain and guide waves, which in turn create and reinforce chains.
Experiments in mammalian neocortex have gathered some evidence for
these patterns, which were hypothetically named "synfire chains"
(Abeles 1982; uniform connection delays, Fig. a) or "synfire
braids" (Bienenstock 1995; unequal delays, Fig. b).
We also postulate that synfire patterns might constitute
the elementary components or "molecular building blocks" at the
mesoscopic level of the mind's symbolic abilities (see
above),
in particular the faculty of compositionality at the
core of linguistic and perceptual functions (Fig. b-d).
|
|
Multiple synfire patterns have the required combinatorial structure
to assemble in various ways through wave synchronization and fast
synaptic binding (Fig. c). Thus, synfire growth could be only
the first step in an infinitely productive "network of networks"
hierarchy.
Under the influence of external stimuli, chain motifs
dynamically bind into higher structural compositions characterized
by complex modes of activity (spatiotemporal patterns; STPs).
This model suggests that neocortex is another instance of a
"self-made tapestry" (Ball 1999) showing pattern formation,
as are many natural complex systems.
|
References
Abeles (1982) Local cortical circuits. Springer.
Ball (1999) The Self-Made Tapestry. Oxford Univ. Press.
Bienenstock (1995) A model of neocortex. Network 6: 179-224.
|
Publications
|
|
|
Coding visual metric with temporal correlations: a model of
deformable template matching based on phase-locking
|
 |
In this work, started when I was a postdoc with
Christoph von der Malsburg
and pursued today with my PhD student
Carlos Sánchez, I propose that
nonzero phase-locking (i.e. delayed correlations) among
oscillating units can be exploited to code for metric
relationships among the features of an image. If spiking times
are interpreted as coordinates, then two spatiotemporal patterns
of activity (STPs) can represent a 2D visual pattern.
|
Visual perception likely involves two processing levels: bottom-up
processing, during which features spontaneously group according to
low-level cues, and top-down processing, during which global
segmentation is completed from above by schemas previously stored
in memory. Accordingly, there are two components in the schematic recognition
task studied here: the input image and the stored shape, which rely
on a common internal metric formed by accurate spatiotemporal
relationships among action potentials.
|
|
Based on this setup, I designed a model of dynamical matching
between lattices of coupled stochastic oscillators (local
excitatory/inhibitory assemblies), behaving like "excitable units"
able to spontaneously generate traveling waves.
The interaction between input pattern and schema pattern, formally
interpreted as a graph-matching task, is implemented through the growth
of dynamical links from one pair of STPs to another pair of STPs.
|
Abstract
|
|
|
Bridging the gap between vision and language by importing complex
system modeling into linguistics
|
 |
How can the same relationship 'in' apply to containers as topologically
different as 'box', 'tree' or 'bowl'? The objective of this study
is to categorize the infinite diversity of schematic visual scenes
into a small set of grammatical elements. To elucidate this "topology
of language", I have proposed with
Jean Petitot a novel dynamical system approach
to cognitive linguistics based on the generation of traveling waves
in cellular automata and spiking neural networks.
|
How can the same linguistic preposition 'in' apply to scenes as
different as "the shoe in the box" (small, hollow, closed volume),
"the bird in the tree" (large, dense, open volume) or "the fruit in
the bowl" (curved surface)? How can language be so insensitive to
wide topological and morphological differences among visual percepts?
Gestalt-inspired cognitive linguistics (Talmy 2000) has shown that
spatial prepositions such as 'in' or 'above' are neutral toward the
shape and size of objects. This core invariance of spatial semantics
can be referred to as cognitive topology, which is at the same
time more flexible (e.g., allowing holes) and more metrically
constrained (e.g., limiting distortions) than mathematical topology.
The goal of this study is to understand and reproduce how our
cognitive system creates schemas and categories at the interface
between vision and language (Regier 1996), in particular its ability to effortlessly
map an infinite diversity of visual scenes onto a small set of
grammatical elements (Fig. a).
Through this challenge we seek to elucidate the fundamental
transformation of continuous dynamical stimuli into discrete
symbolic contents (Petitot 2003), or ontology, which is emblematic of
human intelligence and will be crucial to a reunited,
biologically inspired AI.
|
|
We suggest that language-specific invariants toward shape, size or
other object attributes can be explained using morphological
transforms (Fig. b): expansion processes erase image details
and create virtual structures (domains, skeletons), which yield a
characteristic "signature" for the scene.
Such singularities can arise from a large-scale lattice of coupled
excitable units exhibiting pattern formation through spatiotemporal
order, especially traveling waves. The originality of our approach
is thus to embed semantic processing in complex systems, in
particular to exploit dynamic pattern formation, or morphodynamics,
that can emerge from cellular automata or lattices of coupled
oscillatory neurons (Fig. c).
In summary, our work addresses the crucial cognitive mechanisms of spatial
schematization and categorization at the interface between vision
and language and anchors them to dynamical processes such as activity
diffusion or wave propagation.
|
References
Petitot (2003) Morphogenesis of Meaning. Peter Lang.
Regier (1996) The Human Semantic Potential. MIT Press.
Talmy (2000) Toward a Cognitive Semantics. MIT Press.
|
Selected publications
|
|