Computational
Cognition 2021

Robots receive increasingly more attention in scientific areas beyond robotics. The field of human-robot interaction, for example, focuses not only on developing new technological solutions for robots, but also on how the human interacts with such artificial entities. In my research, I use robots as sophisticated stimuli for examining the mechanisms of human cognition. As such, robots allow for more ecological validity than screen-based stimuli and for excellent experimental control at the same time. In this talk, I will present a series of studies that, with the use of a humanoid robot iCub, addressed specific cognitive mechanisms, such as attention, cognitive control, decision making processes, and theory of mind. Results of these studies reveal that the social component should be taken into account in models of cognition, specifically attention and cognitive control. This might also inspire the way in which artificial cognitive architectures are built.

Using experimental psychology and neuroimaging, my team and I investigate the brain dynamics associated with conscious processing of sensory stimuli. I will present empirical evidence suggesting that conscious processing might specifically relate to a bifurcation in global brain activity following the first 200ms of sensory processing. Our results also suggest that, contrary to the first stages of sensory processing, the onset of these “conscious” mechanisms can be quite flexible in time. I will discuss how these findings might help update the global neuronal workspace model of conscious access.

We study the geometry of deep learning through the lens of approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. For instance, conditioned on the input signal, the output of a MASO DN can be written as a simple affine transformation of the input. This implies that a DN constructs a set of signal-dependent, class-specific templates against which the signal is compared via a simple inner product; we explore the links to the classical theory of optimal classification via matched filters and the effects of data memorization. The spline partition of the input signal space that is implicitly induced by a MASO directly links DNs to the theory of vector quantization (VQ) and K-means clustering, which opens up new geometric avenue to study how DNs organize signals in a hierarchical and multiscale fashion.

Human language is a fundamental biological signal with computational properties that are markedly different than in other perception-action systems: hierarchical relationships between sounds, words, phrases, and sentences, and the unbounded ability to combine smaller units into larger ones. These and other formal properties have long made language difficult to account for from a biological systems perspective, and within models of cognition. I focus on this foundational puzzle – essentially “what does a system need to represent information that is both algebraic and statistical?” - and discuss the computational requirements, including the role of neural oscillations across time, for what I believe is necessary for a system to represent language. I build on examples from cognitive neuroimaging data and computational simulations, and outline a developing theory that integrates basic insights from linguistics and psycholinguistics with the currency of neural computation, in turn demarcating the boundary conditions for artificial systems making contact with human language.

Neural language models are trained on text corpora to place probability distributions over sequences of words. They produce representations of language that have led to dramatic improvements in downstream tasks as diverse as translation, question answering, and image captioning. Language models' usefulness is partly explained by the fact that they robustly encode aspects of linguistic *structure*, including syntactic categories and dependency relations. But the extent to which language modeling induces representations of *meaning*---and the broader question of whether it is even in principle possible to learn about meaning from text alone---have remained a subject of ongoing debate in NLP and linguistics. I'll describe recent work showing that current neural language models build structured representations of meaning that simulate entities and situations as they evolve throughout a discourse. These representations can be linearly decoded into formal descriptions of semantic state analogous to the "file cards" of Heim (1983) and discourse representation structures of Kamp (1984). They can be directly manipulated to produce predictable changes in generated text, and supervised to improve generation quality. Together, these results suggest that the effectiveness of the modern NLP toolkit stems in part from its ability to learn some aspects of meaning with only linguistic form as training data.