[PNAS2021] The neural architecture of language: Integrative modeling converges on predictive processing

Related tags

Deep Learning natural-language-processing deep-learning computational-neuroscience fmri ecog human-language neural-recording
Overview

The neural architecture of language: Integrative modeling converges on predictive processing

Code accompanying the paper The neural architecture of language: Integrative modeling converges on predictive processing by Schrimpf, Blank, Tuckute, Kauf, Hosseini, Kanwisher, Tenenbaum, and Fedorenko.

Large-scale evaluation of neural network language models as predictive models of human language processing. This pipeline compares dozens of state-of-the-art models and 4 human datasets (3 neural, 1 behavioral). It builds on the Brain-Score framework and can easily be extended with new models and datasets.

Installation

git clone https://github.com/mschrimpf/neural-nlp.git
cd neural-nlp
pip install -e .

You might have to install nltk by hand / with conda.

Run

To score gpt2-xl on the Blank2014fROI-encoding benchmark:

python neural_nlp run --model gpt2-xl --benchmark Blank2014fROI-encoding --log_level DEBUG

Other available benchmarks are e.g. Pereira2018-encoding (takes a while to compute), and Fedorenko2016v3-encoding.

You can also specify different models to run -- note that some of them require additional download of weights (run ressources/setup.sh for automated download).

Data

When running a model on a benchmark, the data will automatically be downloaded from S3 (e.g. https://github.com/mschrimpf/neural-nlp/blob/master/neural_nlp/benchmarks/neural.py#L361 for the Pereira2018 benchmark). Costly ceiling estimates have also been precomputed and will be downloaded since they can take days to compute.

Precomputed scores

Scores for models run on the neural, behavioral, and computational-task benchmarks are also available, see the precomputed-scores.csv file. You can re-create the figures in the paper using the analyze scripts.

Citation

If you use this work, please cite

@article{Schrimpf2021,
	author = {Schrimpf, Martin and Blank, Idan and Tuckute, Greta and Kauf, Carina and Hosseini, Eghbal A. and Kanwisher, Nancy and Tenenbaum, Joshua and Fedorenko, Evelina},
	title = {The neural architecture of language: Integrative modeling converges on predictive processing},
	year = {2021},
	journal = {Proceedings of the National Academy of Sciences},
	url = {https://www.pnas.org/content/118/45/e2105646118}
}
You might also like...
EfficientMPC - Efficient Model Predictive Control Implementation

efficientMPC Efficient Model Predictive Control Implementation The original algo

Vin 8 Dec 4, 2022
Natural Posterior Network: Deep Bayesian Predictive Uncertainty for Exponential Family Distributions

Natural Posterior Network This repository provides the official implementation o

Oliver Borchert 54 Dec 6, 2022
Implementation of CVPR'2022:Surface Reconstruction from Point Clouds by Learning Predictive Context Priors
Implementation of CVPR'2022:Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors (CVPR 2022) Personal Web Pages | Paper | Project Page This repository c

null 136 Dec 12, 2022
This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

The Neural Process Family This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CN

DeepMind 892 Dec 28, 2022
Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

ImageProcessingTransformer Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

null 61 Jan 1, 2023
A task-agnostic vision-language architecture as a step towards General Purpose Vision
A task-agnostic vision-language architecture as a step towards General Purpose Vision

Towards General Purpose Vision Systems By Tanmay Gupta, Amita Kamath, Aniruddha Kembhavi, and Derek Hoiem Overview Welcome to the official code base f

AI2 79 Dec 23, 2022
[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [PDF] Wuyang Chen, Xinyu Gong, Zhangyang Wang In ICLR 2

VITA 156 Nov 28, 2022
[ICLR 2021] HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark
[ICLR 2021] HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark

HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark Accepted as a spotlight paper at ICLR 2021. Table of content File structure Prerequi

null 72 Jan 3, 2023
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

Changlin Li 127 Dec 26, 2022
Comments
  • running base example give xarray issue

    running base example give xarray issue

    I tried running the base example on the main page, but I am getting the following error :

    Traceback (most recent call last): File "/Users/eghbalhosseini/anaconda3/envs/arch/lib/python3.7/site-packages/xarray/core/dataset.py", line 884, in _construct_dataarray variable = self._variables[name] KeyError: 'z'

    I am using : xarray V:0.12.0

    opened by eghbalhosseini 4
  • neural data meta CoreNLP parsing issues

    neural data meta CoreNLP parsing issues

    Boar: Listening to the tale the Lord of the Manor tried to discern which huntsman was telling him the truth and what the two huntsmen were thinking as they waited for him to make a -- a. is only parsed together with next sentence.

    MatchstickSeller: "Grandmother" cried the little one "O -- same as above.

    KingOfBirds: He wanted to shout "I am king!" so that the owl could hear him as he flew but thought better of it. -- split into two sentences.

    HighSchool: "Was that another one of those messages?" her mom asked who was driving the car. -- parsed into two sentences.

    "Haven't you reported it yet?" asked Jill Lucy's older sister who was sitting in the passenger seat. -- as above

    "No it's fine. I love it" said Lucy knowing that affording the phone had been no small thing for her mother. -- as above

    "How does it feel to be tall and lanky like a giraffe?" ... -- as above

    opened by mschrimpf 1
  • installing latest brain-score version

    installing latest brain-score version

    Hey Martin, while installing I ran into the kwargs issue for linear_regression(..) again. This can be fixed by installing the latest version of brain-score.

    opened by carina-kauf 0
  • Gt

    Gt

    Scripts used for various analyses for paper submission (backed up 06152020): language specificity analyses, surface projections, language ROI analyses, layer preference analyses, statistics

    opened by gretatuckute 0
Owner
Martin Schrimpf
Research in computational neuroscience & deep learning at MIT
Martin Schrimpf
GitHub https://doi.org/10.1073/pnas.2105646118
An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

Phil Wang 173 Dec 14, 2022
Official implementation for NIPS'17 paper: PredRNN: Recurrent Neural Networks for Predictive Learning Using Spatiotemporal LSTMs.

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning The predictive learning of spatiotemporal sequences aims to generate future

THUML: Machine Learning Group @ THSS 243 Dec 26, 2022
PyTorch Code of "Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics"

Memory In Memory Networks It is based on the paper Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spati

Yang Li 12 May 30, 2022
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

DeepHyper Team 214 Jan 8, 2023
Predictive AI layer for existing databases.

MindsDB is an open-source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning

MindsDB Inc 12.2k Jan 3, 2023
Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Documentation: https://raamana.github.io/neuropredict/ News As of v0.6, neuropredict now supports regression applications i.e. predicting continuous t

Pradeep Reddy Raamana 93 Nov 29, 2022
Predictive AI layer for existing databases.

MindsDB is an open-source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning

MindsDB Inc 3.2k Feb 12, 2021
Real-Time Multi-Contact Model Predictive Control via ADMM

Here, you can find the code for the paper 'Real-Time Multi-Contact Model Predictive Control via ADMM'. Code is currently being cleared up and optimize

null 17 Dec 28, 2022
A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

Website, Tutorials, and Docs    Uncertainty Toolbox A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualizatio

Uncertainty Toolbox 1.4k Dec 28, 2022
Predictive Maintenance LSTM

Predictive-Maintenance-LSTM - Predictive maintenance study for Complex case study, we've obtained failure causes by operational error and more deeply by design mistakes.

Amir M. Sadafi 1 Dec 31, 2021