Active inference#

Guest instructor: Ryan Smith, Laureate Brain Institute

Description of the tutorial#

Active inference is a recently developed computational framework for jointly modeling perception, learning, and decision-making as partially observable Markov decision processes (POMDPs). This framework assumes the brain represents a generative model of the (internal and external) causes of the sensory input it receives; and that it uses approximate Bayesian inference to infer hidden states of the world causing sensory input (perception), model parameters (learning), and optimal action policies (decision-making) within this generative model. It places special emphasis on how the brain infers the observations that will be generated by its own actions. In empirical studies, generative models of task behavior can be constructed; and the parameters of these models can be fit to behavior in individual participants. These individual parameter estimates can then be used as predictors in between-subjects analyses. This framework also affords simulation studies in which precise predictions can be derived from generative models designed to capture broader theoretical proposals.

At the algorithmic level, active inference proposes that the brain uses variational message passing approaches to approximate Bayesian inference in a biologically plausible manner. This entails a neural process theory of how messages passed between neurons in the brain can be viewed as variational messages – providing information about the sufficient statistics of various probability distributions. Following assumptions associated with neural mass models, this allows active inference to provide empirically testable predictions of neural responses that could be measured with fMRI and EEG, among other neuroimaging measures. This tutorial will first introduce the active inference framework at a conceptual and mathematical level. It will then focus on teaching attendees how to build generative models of multiple types of behavioral tasks, and how to fit these models to behavioral data. It will also introduce the neural process theory and teach attendees how to derive simulated neuronal responses that could be applied in their own neuroscientific research.

Software tools#

  • MATLAB: https://matlab.mathworks.com/

  • SPM12: https://www.fil.ion.ucl.ac.uk/spm/software/download/

  • Note: While SPM12 is free, MATLAB requires a license. If you do not have access to a MATLAB license, then you could also download Octave (https://octave.org/). With a few exceptions, most of the tutorial MATLAB code we will use is at least partially compatible with Octave.

References and Background Reading#

This tutorial will be based largely around the following paper:

  • Smith, R., Friston, K. J., & Whyte, C. J. (2022). A step-by-step tutorial on active inference and its application to empirical data. Journal of mathematical psychology, 107, 102632. Smith 2022 Download

  • MATLAB code from this paper that we will use can be found here: rssmith33/Active-Inference-Tutorial-Scripts

Before the course: Read Appendix A of Smith et al. 2022 tutorial paper, and try the “paper and pencil” exercises in Appendix B.

Also, read this paper by Griffiths et al. It is an excellent introduction to Bayesian analysis and some key concepts that are broadly used.

Some other papers that may be of interest for further conceptual background or mathematical details include:

  • Smith, R., Badcock, P., & Friston, K. J. (2021). Recent advances in the application of predictive coding and active inference models within clinical neuroscience. Psychiatry and Clinical Neurosciences, 75(1), 3-13.

  • Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020). Active inference on discrete state-spaces: A synthesis. Journal of Mathematical Psychology, 99, 102447.

  • Sajid, N., Ball, P. J., Parr, T., & Friston, K. J. (2021). Active inference: demystified and compared. Neural computation, 33(3), 674-712.

  • Friston, K. J., Parr, T., & de Vries, B. (2017). The graphical brain: belief propagation and active inference. Network neuroscience, 1(4), 381-414.

  • Friston, K. J., Rosch, R., Parr, T., Price, C., & Bowman, H. (2018). Deep temporal models and active inference. Neuroscience & Biobehavioral Reviews, 90, 486-501.

  • Parr, T., Markovic, D., Kiebel, S. J., & Friston, K. J. (2019). Neuronal message passing using Mean-field, Bethe, and Marginal approximations. Scientific reports, 9(1), 1-18.

  • Parr, T., & Friston, K. J. (2018). The anatomy of inference: generative models and brain structure. Frontiers in computational neuroscience, 90.

Schedule:#

9-11:45am and session 2 from 2-5pm

  • 9:00 – 10:00 – Initial introduction to the theory and mathematics of perception and decision-making within active inference

  • 10:00 – 10:05 – Short break

  • 10:05 - 10:50 – Introduction to building generative (POMDP) models of behavioral tasks, with worked examples

  • 10:50 – 10:55 – Short break

  • 10:55 – 11:45 – Introduction to learning in active inference & practice exercises building generative models of reinforcement learning tasks

  • 11:45 – 2:15 – Lunch break

  • 2:15 – 3:15 – Introduction to fitting models to behavioral data, with practice exercises

  • 3:15 – 3:20 – Short break

  • 3:20 – 4:05 – Introduction to variational (marginal) message passing and building deep temporal models

  • 4:05 – 4:10 – Short break

  • 4:10 - 5:15 – Introduction to simulating neural responses with worked examples