Towards an Artificial Cognitive Science

Introduction

With the advent of capable artificial agents comes the urgency to improve our understanding of how they learn, reason, and decide. This proposal aims to shape a new field of artificial cognitive science in which tools from cognitive psychology are used to understand and improve the behavior of the notoriously opaque artificial agents to which we increasingly delegate autonomy and authority.

Focus on Foundation Models

We will focus on the newly-emerged, disruptive, and increasingly powerful foundation models, huge neural networks with billions of parameters that are trained on hundreds of billions of words. We will study them using tools from cognitive psychology.

Experimental Paradigms

We will subject these models to experimental paradigms from the psychological literature and analyze their trial-by-trial behavior using detailed computational modeling. Psychological paradigms have been engineered to uncover precisely the mechanisms that underpin behavior.

Prompt Engineering

We will use cognitive psychology to study and improve prompt engineering, i.e., the design of textual inputs to these models that lead to a desired behavior.

Cognitive Benchmarks

Afterward, we will build large cognitive benchmarks to assess foundation models' learning, decision-making, and reasoning abilities in both the pure language and the combined language and vision domains.

Cognitive Therapy

Finally, we will attempt to improve these models' behavior using cognitive therapy.

Conclusion

This proposal will help to create transparent and human-like agents and define an innovative and interdisciplinary research field in which cognitive psychologists and artificial intelligence researchers jointly examine and improve intelligent systems.