A menu of topics#

There is a very large list of topics to potentially know about. These are all things that could potentially be covered. The catch is that this list is already probably a couple of stats/biostats Ph.D.s long. So think of it as a menu.

My thought is that many of these techniques build on the same foundational mathematics and techniques. Most are applications of linear algebra. Most also involve probability theory in some way, more or less explicitly. So learning some of them can help to learn others later.

Science is a life-long process of exploration and self-education.

Linear models (univariate)#

  • The general linear model (GLM)

  • Contrasts and parametric inference (t, F)

  • Model diagnostics and VIFs

  • Assumptions and their violations: Simulating the GLM

  • Model parameterization: Setting up a model to test effects, interactions, centering

  • Covariates and causality: Lord’s paradox, collider bias

  • Controlling for baseline conditions: Subtraction vs. covariation

  • Robust regression

  • Time series models (autoregressive models)

  • Transformations (power transformations, Spearman’s)

  • Nonparametric models and basis functions (splines, FIR, basis functions)

  • Mixed effects (and estimating degrees of freedom)

  • Generalized linear models (logistic regression, poisson)

  • Generalized additive models (GAMs)

  • Gaussian processes

Inference#

  • Bootstrapping

  • Permutation

  • Cross-validation

  • Model comparisons (AIC, BIC, cross-val)

  • Multiple comparison correction approaches

  • Bayes Factors and support for the null hypothesis (Rouder)

  • Sequential sampling models - MCMC

  • Chinese restaurant/Dirichlet, indian buffet processes

Multivariate linear models#

  • Dimensionality reduction (PCA, ICA, EFA, CFA, NNMF, manifold learning)

  • Principal components analysis

  • Multidimensional scaling

  • Factor analysis and ICA

  • Structural Equation Modeling (SEM)

  • Clustering (k-means/medoids, hierarchical, density-based, spectral, mixture models)

Multivariate predictive models:#

Classification models:#

  • SVMs

  • Naive Bayes classifiers

  • Classification trees and random forests

Regression models:#

  • Kernel-form and penalized regression (lasso, ridge)

  • Canonical correlation and partial least squares (PLS)

  • Regression trees and random forests

  • Gaussian processes

Other considerations:#

  • Interpretable ML: Explaining features in multivariate predictive models

  • Stacked/hierarchical multivariate predictive models

Bayesian statistics#

  • Principles of probability and Bayes rule

  • Empirical Bayes

  • Bayesian regression

  • Hierarchical Bayesian inference (for sequential models) (Matthys/K. Stephan)

Mediation and causal models#

  • Principles of causal inference

  • Potential outcomes notation

  • Mediation analysis

  • Mediation and neuroimaging

  • Directed acyclic graphs

  • Instrumental variables

  • Structural mean models

Experimental design#

  • Types of designs

  • Statistical efficiency and design optimization

  • Power and effect sizes

Signal processing#

  • Fourier transform

  • Linear filters (e.g., high-pass, low-pass)

  • Aliasing and sampling theory

  • Signal detection theory

Sequential models#

  • Reinforcement learning models

  • Kalman filters

  • Dynamic decision-making models (Roy, Pillow)

  • Dynamic Causal Models (DCM)

  • State space models

  • Markov models and semi-Markov models (Martin Lindquist)

  • POMDPs

  • Active inference (Ryan Smith)

Graph theory#

  • Graph theory (e.g., social network analysis)

fMRI applications#

  • Pattern classification and predictive models

  • RSA

  • RSA with deep neural network features

  • Dynamic connectivity & Connectome-based predictive modeling

  • Empirical Bayes for improving individual ROIs and predictive models

  • Neuromark Group ICA and template ICA

Deep learning and AI#

  • Fundamentals of neural networks

  • Training deep learning models

  • Applying deep learning models to fMRI analysis

  • Natural language processing models (Jeremy Manning)

  • Recurrent neural networks (John Murray)

  • Transformer models (Vosoughi)