A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Last update: Jan 4, 2023

Overview

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-the-art vision and language models and has powered multiple research projects at Facebook AI Research. See full list of project inside or built on MMF here.

MMF is powered by PyTorch, allows distributed training and is un-opinionated, scalable and fast. Use MMF to bootstrap for your next vision and language multimodal research project by following the installation instructions. Take a look at list of MMF features here.

MMF also acts as starter codebase for challenges around vision and language datasets (The Hateful Memes, TextVQA, TextCaps and VQA challenges). MMF was formerly known as Pythia. The next video shows an overview of how datasets and models work inside MMF. Checkout MMF's video overview.

Installation

Follow installation instructions in the documentation.

Documentation

Learn more about MMF here.

Citation

If you use MMF in your work or use any models published in MMF, please cite:

@misc{singh2020mmf,
  author =       {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and
                 Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi},
  title =        {MMF: A multimodal framework for vision and language research},
  howpublished = {\url{https://github.com/facebookresearch/mmf}},
  year =         {2020}
}

License

MMF is licensed under BSD license available in LICENSE file

Comments

[feat] Add ViLT model
Stack from ghstack:

#1142

#1106

#1103

-> #1102

#1141

#1101

Add ViLT model and unit tests. ViLT model from megaVLT, using ViT from huggingface.

Differential Revision: D31117427
CLA Signed Merged
opened by Ryan-Qiyu-Jiang 49
[feat] Add ViLT image and text embeddings
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

-> #1101

Add ViT image and Bert text embedding encoders for ViLT model.

Differential Revision: D31117428
CLA Signed Merged
opened by Ryan-Qiyu-Jiang 46
[feat][PL][8/N] save config with checkpoint
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670421
CLA Signed Merged
opened by ytsheng 39
[feat][PL][6/N] pretrained state mapping for PL
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670417
CLA Signed
opened by ytsheng 37
[refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29888294
CLA Signed
opened by ytsheng 36
[feat] Add ViLT encoder
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

#1101

-> #1100

Add encoder for loading pretrained ViTModel weights from huggingface

Differential Revision: D31117425
CLA Signed
opened by Ryan-Qiyu-Jiang 33
[feat][PL][10/N] lightning distributed
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29981634
CLA Signed
opened by ytsheng 33
[feat][PL][7/N] trainer checkpoint from mmf to lightning update
Stack from ghstack:

#1035 [feat][PL][11/N] Inference with test

#1033 [feat][PL][10/N] lightning distributed

#1030 [refactor][PL][9/N] change the name of the _load_checkpoint to be more descriptive to contain test

#1024 [feat][PL][8/N] save config with checkpoint

#1023 [feat][PL][7/N] trainer checkpoint from mmf to lightning update

#1022 [feat][PL][6/N] pretrained state mapping for PL

Differential Revision: D29670422
CLA Signed
opened by ytsheng 33
[feat] Add ViT modules
Stack from ghstack:

#1142

#1106

#1103

#1102

#1141

#1101

#1100

-> #1099

Add modified huggingface ViT modules from megaVLT supporting attention masks

Differential Revision: D31117429
CLA Signed
opened by Ryan-Qiyu-Jiang 30
[feat] Add UNITER model wrapper
Stack from ghstack:

#1144

#1128

-> #1127

#1133

Add UNITER model to mmf registry with support for pretraining through yaml head configs.

Differential Revision: D31768457
CLA Signed
opened by Ryan-Qiyu-Jiang 28
Error during training Textvqa

❓ Questions and Help

Hi,

While I am trying the training code with m4c model, I am getting the following error,

2021-03-11T03:34:15 | mmf.utils.general: Total Parameters: 90850184. Trained Parameters: 90850184 2021-03-11T03:34:15 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 363, in next data = self._next_data() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data return self._process_data(data) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data data.reraise() File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch_utils.py", line 395, in reraise raise self.exc_type(msg) KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 231, in load image_id = int(split.split("")[-1]) ValueError: invalid literal for int() with base 10: 'train\7f14a505b6edcbc5'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataset.py", line 207, in getitem return self.datasets[dataset_idx][sample_idx] File "d:\project\new folder\mmf\mmf\datasets\builders\textvqa\dataset.py", line 100, in getitem features = self.features_db[idx] File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 91, in getitem return self.get(image_info) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 99, in get return self.from_path(feature_path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 107, in from_path features, infos = self._get_image_features_and_info(path) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 80, in _get_image_features_and_info image_feats, infos = self._read_features_and_info(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\features_database.py", line 65, in _read_features_and_info feature, info = feature_reader.read(feat_file) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 95, in read return self.feat_reader.read(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 158, in read image_info = self._load(image_feat_path) File "d:\project\new folder\mmf\mmf\datasets\databases\readers\feature_readers.py", line 238, in _load img_id_idx = self.image_id_indices[image_id] KeyError: b'train\7f14a505b6edcbc5'

When I tried with model = "Lorra", I am getting the below error,

2021-03-11T03:27:37 | mmf.utils.general: Total Parameters: 192497485. Trained Parameters: 192497485 2021-03-11T03:27:37 | mmf.trainers.core.training_loop: Starting training... Traceback (most recent call last): File "C:\Users\kvman\anaconda3\envs\mmf\Scripts\mmf_run-script.py", line 33, in sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')()) File "d:\project\new folder\mmf\mmf_cli\run.py", line 133, in run main(configuration, predict=predict) File "d:\project\new folder\mmf\mmf_cli\run.py", line 56, in main trainer.train() File "d:\project\new folder\mmf\mmf\trainers\mmf_trainer.py", line 132, in train self.training_loop() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 31, in training_loop self.run_training_epoch() File "d:\project\new folder\mmf\mmf\trainers\core\training_loop.py", line 74, in run_training_epoch for idx, batch in enumerate(self.train_loader): File "d:\project\new folder\mmf\mmf\datasets\multi_dataset_loader.py", line 213, in iter return iter(self.loaders[0]) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 291, in iter return _MultiProcessingDataLoaderIter(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\site-packages\torch\utils\data\dataloader.py", line 737, in init w.start() File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\Users\kvman\anaconda3\envs\mmf\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) BrokenPipeError: [Errno 32] Broken pipe

Kindly help me to resolve this issue.
needs more info triaged

opened by Mano2610 28

$errors while running pytests .\test$

errors while running pytests .\test

❓ Questions and Help

I successfully installed mmf according to the installation document.

conda create -n mmf python=3.7
conda activate mmf
git clone https://github.com/facebookresearch/mmf.git
cd mmf
pip install --editable .

but, in test step, I got below error messages. I goggled about libcudart.so and found out that it has to do with torchaudio. As reqirement.txt, my torchaudio version is 0.11.0. Anyone who knows to solve this problem?

Thank you very much. Minho

$ pytest ./tests/
ImportError while loading conftest '/home/mhlee/mmf/tests/conftest.py'.
tests/__init__.py:2: in <module>
    from mmf.utils.patch import patch_transformers
mmf/__init__.py:6: in <module>
    patch_transformers()
mmf/utils/patch.py:65: in patch_transformers
    f"transformers.models.{key}.{module}"
../.local/lib/python3.7/site-packages/transformers/models/speech_to_text/feature_extraction_speech_to_text.py:23: in <module>
    import torchaudio.compliance.kaldi as ta_kaldi
../.local/lib/python3.7/site-packages/torchaudio/__init__.py:1: in <module>
    from torchaudio import _extension  # noqa: F401
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:67: in <module>
    _init_extension()
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:61: in _init_extension
    _load_lib("libtorchaudio")
../.local/lib/python3.7/site-packages/torchaudio/_extension.py:51: in _load_lib
    torch.ops.load_library(path)
../.local/lib/python3.7/site-packages/torch/_ops.py:220: in load_library
    ctypes.CDLL(path)
/home/master/anaconda3/envs/mmf/lib/python3.7/ctypes/__init__.py:364: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

opened by cokemhlee 0

Bump terser from 5.10.0 to 5.16.1 in /website
Bumps terser from 5.10.0 to 5.16.1.

Changelog

Sourced from terser's changelog.

v5.16.1

Properly handle references in destructurings (const { [reference]: val } = ...)

Allow parsing of .#privatefield in nested classes

Do not evaluate operations that return large strings if that would make the output code larger

Make collapse_vars handle block scope correctly

Internal improvements: Typos (#1311), more tests, small-scale refactoring

v5.16.0

Disallow private fields in object bodies (#1011)

Parse #privatefield in object (#1279)

Compress #privatefield in object

v5.15.1

Fixed missing parentheses around optional chains

Avoid bare let or const as the bodies of if statements (#1253)

Small internal fixes (#1271)

Avoid inlining a class twice and creating two equivalent but !== classes.

v5.15.0

Basic support for ES2022 class static initializer blocks.

Add AudioWorkletNode constructor options to domprops list (#1230)

Make identity function inliner not inline id(...expandedArgs)

v5.14.2

Security fix for RegExps that should not be evaluated (regexp DDOS)

Source maps improvements (#1211)

Performance improvements in long property access evaluation (#1213)

v5.14.1

keep_numbers option added to TypeScript defs (#1208)

Fixed parsing of nested template strings (#1204)

v5.14.0

Switched to @jridgewell/source-map for sourcemap generation (#1190, #1181)

Fixed source maps with non-terminated segments (#1106)

Enabled typescript types to be imported from the package (#1194)

Extra DOM props have been added (#1191)

Delete the AST while generating code, as a means to save RAM

v5.13.1

Removed self-assignments (varname=varname) (closes #1081)

Separated inlining code (for inlining things into references, or removing IIFEs)

Allow multiple identifiers with the same name in var destructuring (eg var { a, a } = x) (#1176)

v5.13.0

... (truncated)

Commits

b90215a 5.16.1

319fe31 update changelog

d941a11 mark computed properties in for..of initializers as used. Closes #1238

635400e move AST_Scope#drop_unused into its own file

a192fac some cleanups

dcc3030 add shadowed variable test

4d37f39 traverse exported code while hoisting object properties. Closes #1307

1e99e5d update test

68461d4 properly detect nested classes. Closes #1313

b91e0ce docs: fix simple typo, concatentation -> concatenation (#1311)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump minimist from 1.2.5 to 1.2.7 in /website
Bumps minimist from 1.2.5 to 1.2.7.

Changelog

Sourced from minimist's changelog.

v1.2.7 - 2022-10-10

Commits

[meta] add auto-changelog 0ebf4eb

[actions] add reusable workflows e115b63

[eslint] add eslint; rules to enable later are warnings f58745b

[Dev Deps] switch from covert to nyc ab03356

[readme] rename and add badges 236f4a0

[meta] create FUNDING.yml; add funding in package.json 783a49b

[meta] use npmignore to autogenerate an npmignore file f81ece6

Only apps should have lockfiles 56cad44

[Dev Deps] update covert, tape; remove unnecessary tap 49c5f9f

[Tests] add aud in posttest 228ae93

[meta] add safe-publish-latest 01fc23f

[meta] update repo URLs 6b164c7

v1.2.6 - 2022-03-21

Commits

test from prototype pollution PR bc8ecee

isConstructorOrProto adapted from PR c2b9819

security notice for additional prototype pollution issue ef88b93

Commits

c590d75 v1.2.7

0ebf4eb [meta] add auto-changelog

e115b63 [actions] add reusable workflows

01fc23f [meta] add safe-publish-latest

f58745b [eslint] add eslint; rules to enable later are warnings

228ae93 [Tests] add aud in posttest

236f4a0 [readme] rename and add badges

ab03356 [Dev Deps] switch from covert to nyc

49c5f9f [Dev Deps] update covert, tape; remove unnecessary tap

783a49b [meta] create FUNDING.yml; add funding in package.json

Additional commits viewable in compare view

Maintainer changes

This version was pushed to npm by ljharb, a new releaser for minimist since your current version.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump prismjs from 1.26.0 to 1.29.0 in /website
Bumps prismjs from 1.26.0 to 1.29.0.

Release notes

Sourced from prismjs's releases.

v1.29.0

Release 1.29.0

v1.28.0

Release 1.28.0

v1.27.0

Release 1.27.0

Changelog

Sourced from prismjs's changelog.

1.29.0 (2022-08-23)

New components

BBj (#3511) 1134bdfc

BQN (#3515) 859f99a0

Cilk/C & Cilk/C++ (#3522) c8462a29

Gradle (#3443) 32119823

METAFONT (#3465) 2815f699

WGSL (#3455) 4c87d418

Updated components

AsciiDoc

Some regexes are too greedy (#3481) c4cbeeaa

Bash

Added "sh" alias (#3509) 6b824d47

Added support for parameters and the java and sysctl commands. (#3505) b9512b22

Added cargo command (#3488) 3e937137

BBj

Improve regexes (#3512) 0cad9ae5

CSS

Fixed @-rules not accounting for strings (#3438) 0d4b6cb6

CSS Extras

Added support for RebeccaPurple color (#3448) 646b2e0a

Hoon

Fixed escaped strings (#3473) 64642716

Java

Added support for constants (#3507) 342a0039

Markup

Fixed quotes in HTML attribute values (#3442) ca8eaeee

NSIS

Added missing commands (#3504) b0c2a9b4

Scala

Updated keywords to support Scala 3 (#3506) a090d063

SCSS

Fix casing in title of the scss lang (#3501) 2aed9ce7

Updated plugins

Line Highlight

Account for offset when clamping ranges (#3518) 098e3000

Ignore ranges outside of actual lines (#3475) 9a4e725b

Normalize Whitespace

Add configuration via attributes (#3467) 91dea0c8

Other

Added security policy (#3070) 05ee042a

Added list of maintainers (#3410) 866b302e

... (truncated)

Commits

59e5a34 1.29.0

cd080f2 Updated npmignore to include new MD files (#3534)

751664b Added PR stop notice (#3532)

248f6ab Added changelog for v1.29.0 (#3533)

098e300 Line Highlight: Account for offset when clamping ranges (#3518)

6b824d4 Bash: Added "sh" alias (#3509)

15272f7 Website: Added third-party tutorial for Pug template (#3459)

c8462a2 Cilk: Add support for Cilk (with C/C++) (#3522)

859f99a Added bqn language support (#3515)

0cad9ae BBj: Improve regexes (#3512)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump eventsource from 1.1.0 to 1.1.2 in /website
Bumps eventsource from 1.1.0 to 1.1.2.

Changelog

Sourced from eventsource's changelog.

1.1.2

Inline origin resolution, drops original dependency (#281 Espen Hovlandsdal)

1.1.1

Do not include authorization and cookie headers on redirect to different origin (#273 Espen Hovlandsdal)

Commits

0a8b85b 1.1.2

f99ae66 docs: update history for 1.1.2

06c9721 chore: rebuild polyfill

9494642 fix: inline origin resolution, drop original dependency (#281)

aa7a408 1.1.1

56d489e chore: rebuild polyfill

4a951e5 docs: update history for 1.1.1

f9f6416 fix: strip sensitive headers on redirect to different origin

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0
Bump follow-redirects from 1.14.7 to 1.15.2 in /website
Bumps follow-redirects from 1.14.7 to 1.15.2.

Commits

9655237 Release version 1.15.2 of the npm package.

6e2b86d Default to localhost if no host given.

449e895 Throw invalid URL error on relative URLs.

e30137c Use type functions.

76ea31f ternary operator syntax fix

84c00b0 HTTP header lines are separated by CRLF.

d28bcbf Create SECURITY.md (#202)

62a551c Release version 1.15.1 of the npm package.

7fe0779 Use for ... of.

948c30c Fix redirecting to relative URL when using proxy

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

CLA Signed dependencies javascript
opened by dependabot[bot] 0

Releases(v0.3.1)

v0.3.1(Aug 26, 2019)

Source code(tar.gz)
Source code(zip)
v0.3(Jun 26, 2019)
Features

Multi-tasking support: Multitasking over various datasets available in Pythia

Distributed training support

Better Customization Support: Use your custom losses, metrics, optimizers and lot of more

Standardized Trainer API: A standard trainer API to fit most of your use-cases, if not inherit and build your own trainer

Processors: Use processors to build out your datasets easily and without pain

SampleList and Sample: Use SampleList and Sample to have more granular control over what you pass and a single unified API for accessing attributes whether inside a dataset, a single sample or a batch

Feature Extraction: A new simple script to extract out features and related information from VQA MaskRCNN Benchmark

Registry: No need to manually load datasets and models anymore, registry takes care of loading your models, datasets and other classes at the fly. Think of registry as a singleton containing all that you need.

Tensorboard Logging: Tensorboard logging is now provided by default.

Configuration: Better hierarchal configuration system for better separation of concerns.

Checkpointing: Better control checkpointing and resuming

Logging: Better logging is provided now with eta, individual val losses and metrics. Just pass them back from your model and everything logs automatically

EvalAI Evaluation: Now, directly output JSON files that can be uploaded to EvalAI

Early Stopping

New Embeddings Support: FastText, GloVe, BERT etc.

Datasets

Following new datasets were added:

TextVQA

VizWiz

COCO Captioning

Preliminary GQA support

Models

LoRRA

Revamped Pythia

BUTD

Preliminary BAN Support

Note: There are a lot of breaking changes in the API from v0.1. Refer to the documentation to learn more on how to work with Pythia v0.3.
Source code(tar.gz)
Source code(zip)

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Related tags

Overview

Installation

Documentation

Citation

License

Comments

❓ Questions and Help

❓ Questions and Help

v5.16.1

v5.16.0

v5.15.1

v5.15.0

v5.14.2

v5.14.1

v5.14.0

v5.13.1

v5.13.0

v1.2.7 - 2022-10-10

Commits

v1.2.6 - 2022-03-21

Commits

v1.29.0

v1.28.0

v1.27.0

1.29.0 (2022-08-23)

New components

Updated components

Updated plugins

Other

Releases(v0.3.1)

v0.3.1(Aug 26, 2019)

v0.3(Jun 26, 2019)

Features

Datasets

Models

Owner

Facebook Research

FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision. ICCV 2021.

The code repository for EMNLP 2021 paper "Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization".

A modular, research-friendly framework for high-performance and inference of sequence models at many scales

Clinica is a software platform for clinical research studies involving patients with neurological and psychiatric diseases and the acquisition of multimodal data

Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

FEDn is an open-source, modular and ML-framework agnostic framework for Federated Machine Learning

A task-agnostic vision-language architecture as a step towards General Purpose Vision

MAGMA - a GPT-style multimodal model that can understand any combination of images and language

Detectron2 is FAIR's next-generation platform for object detection and segmentation.

ICLR 2021, Fair Mixup: Fairness via Interpolation

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

Fair Recommendation in Two-Sided Platforms

PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.

A FAIR dataset of TCV experimental results for validating edge/divertor turbulence models.

Code for Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022)

Regulatory Instruments for Fair Personalized Pricing.