BErt-like Neurophysiological Data Representation

Last update: Dec 23, 2022

Related tags

Overview

BENDR

BErt-like Neurophysiological Data Representation

This repository contains the source code for reproducing, or extending the BERT-like self-supervision pre-training for EEG data from the article:

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

To run these scripts, you will need to use the DN3 project. We will try to keep this updated so that it works with the latest DN3 release. If you are just looking for the BENDR model, and don't need to reproduce the article results per se, BENDR will be (or maybe already is if I forgot to update it here) integrated into DN3, in which case I would start there.

Currently, we recommend version 0.2. Feel free to open an issue if you are having any trouble.

More extensive instructions are upcoming, but in essence you will need to either:

a)  Download the TUEG dataset and pre-train new encoder and contextualizer weights, _or_
b)  Use the [pre-trained model weights](https://github.com/SPOClab-ca/BENDR/releases/tag/v0.1-alpha)

Once you have a pre-trained model:

1) Add the paths of the pre-trained weights to configs/downstream.yml
2) Edit paths to local copies of your datasets in configs/downstream_datasets.yml
3) Run downstream.sh

Comments

about the loss function

Very appreciate for your contribution.i am really interested in the self training in EEG. The only question is about calculating loss function. In your paper, The calculation of the denominator uses cosine similarity between the output of the transformer and the 20 distractors and the input of the transformer. However, in the code, the calculation of the denominator uses cosine similarity between the input of the transformer and the 20 distractors, and the output of the transformer. In other word, the output and the input switch positions. Are both the calculation approaches the same? Or why did you change the calculation approache in the code? Thanks!

opened by stickOverCarrot 2
About deploy downstream.yml and downstream_datasets.yml

Tranks for supplying your code. But when I follow your markdown, I meet some problems

This is my project files

This is my downstream.yml

This is my downstream_datasets.yml

opened by YoloEliwa 1
Pre-trained weights?

Not an issue per se, but you state the pre-trained weights for your paper are available in this repo, yet I have had a good look around and I haven't found them, nor a means of downloading them. Please can you let me know where I could find them? I'm really keen to try out this exciting architecture you've put together!

opened by SgtWhiskeyjack 1
result_tracking module

There's a reference that's in the module import: downstream.py from result_tracking import ThinkerwiseResultTracker that looks like some type of tracking code for experiments?

opened by bencten 1
dropout should change

Iteration: 4%|▍ | 13/330 [00:36<16:00, 3.03s/batches, bac=0.5, Accuracy=0.51, loss=0.695, lr=1.47e-6]D:\Anaconda\envs\LGG\lib\site-packages\torch\nn\functional.py:1338: UserWarning: dropout2d: Received a 3D input to dropout2d and assuming that channel-wise 1D dropout behavior is desired - input is interpreted as shape (N, C, L), where C is the channel dim. This behavior will change in a future release to interpret the input as one without a batch dimension, i.e. shape (C, H, W). To maintain the 1D channel-wise dropout behavior, please switch to using dropout1d instead. warnings.warn("dropout2d: Received a 3D input to dropout2d and assuming that channel-wise "

opened by zy2021314 0
A more detailed explanation

We need to use your code for research, may I ask when you can provide detailed explanation, because we have some difficulties in understanding the code without detailed explanation.

opened by EchizenMike 0
preload in downstream.yml

In the "downstream.yml" file, what is the function of the "preload"? What's mean if I specify "preload: True" or "preload: False"?

Thank you in advance

opened by frannfuri 0

Releases(v0.1-alpha)

v0.1-alpha(Feb 25, 2021)

This release mainly serves so that the weights used for fine-tuning in the article are available for download (and one can avoid pre-training if not needed).
Source code(tar.gz)
Source code(zip)
contextualizer.pt(584.30 MB)
encoder.pt(15.15 MB)

Owner

GitHub

Elementary is an open-source data reliability framework for modern data teams. The first module of the framework is data lineage.

Data lineage made simple, reliable, and automated. Effortlessly track the flow of data, understand dependencies and analyze impact. Features Visualiza

898 Jan 9, 2023

Hidden Markov Models in Python, with scikit-learn like API

hmmlearn hmmlearn is a set of algorithms for unsupervised learning and inference of Hidden Markov Models. For supervised learning learning of HMMs and

2.7k Jan 3, 2023

🧪 Panel-Chemistry - exploratory data analysis and build powerful data and viz tools within the domain of Chemistry using Python and HoloViz Panel.

???? ??. The purpose of the panel-chemistry project is to make it really easy for you to do DATA ANALYSIS and build powerful DATA AND VIZ APPLICATIONS within the domain of Chemistry using using Python and HoloViz Panel.

97 Dec 8, 2022

fds is a tool for Data Scientists made by DAGsHub to version control data and code at once.

Fast Data Science, AKA fds, is a CLI for Data Scientists to version control data and code at once, by conveniently wrapping git and dvc

359 Dec 22, 2022

Tuplex is a parallel big data processing framework that runs data science pipelines written in Python at the speed of compiled code

Tuplex is a parallel big data processing framework that runs data science pipelines written in Python at the speed of compiled code. Tuplex has similar Python APIs to Apache Spark or Dask, but rather than invoking the Python interpreter, Tuplex generates optimized LLVM bytecode for the given pipeline and input data set.

791 Jan 4, 2023

A data parser for the internal syncing data format used by Fog of World.

A data parser for the internal syncing data format used by Fog of World. The parser is not designed to be a well-coded library with good performance, it is more like a demo for showing the data structure.

40 Dec 12, 2022

Functional Data Analysis, or FDA, is the field of Statistics that analyses data that depend on a continuous parameter.

Functional Data Analysis Python package

Grupo de Aprendizaje Automático - Universidad Autónoma de Madrid

184 Dec 27, 2022

Fancy data functions that will make your life as a data scientist easier.

WhiteBox Utilities Toolkit: Tools to make your life easier Fancy data functions that will make your life as a data scientist easier. Installing To ins

3 Oct 3, 2022

A Big Data ETL project in PySpark on the historical NYC Taxi Rides data

Processing NYC Taxi Data using PySpark ETL pipeline Description This is an project to extract, transform, and load large amount of data from NYC Taxi

2 Dec 12, 2021

Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

2 Nov 20, 2021

Utilize data analytics skills to solve real-world business problems using Humana’s big data

Humana-Mays-2021-HealthCare-Analytics-Case-Competition- The goal of the project is to utilize data analytics skills to solve real-world business probl

1 Dec 27, 2021

Python data processing, analysis, visualization, and data operations

Python This is a Python data processing, analysis, visualization and data operations of the source code warehouse, book ISBN: 9787115527592 Descriptio

1 Jan 16, 2022

PrimaryBid - Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift

Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift This project is composed of two parts: Part1 and Part2

1 Jan 19, 2022

Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

Data Scientist Learning Plan Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

27 Nov 1, 2022

BErt-like Neurophysiological Data Representation

Related tags

Overview

BENDR

Comments

about the loss function

About deploy downstream.yml and downstream_datasets.yml

Pre-trained weights?

result_tracking module

dropout should change

A more detailed explanation

preload in downstream.yml