A Repository of Community-Driven Natural Instructions

Overview

A Repository of Community-Driven Natural Instructions

TLDR; this repository maintains a community effort to create a large collection of tasks and their natural language definitions/instructions. We're looking for more contributions to make this data bigger! 🙌 We invite submission of new tasks to this benchmark by way of GitHub pull request, through September 15, 2021. The contributors with meaningful contribution to our tasks will be included as co-authors on a paper that will announce the benchmark as well as analysis/results on it.

Background

Why define tasks in natural language?

While the current dominant paradigm (supervised learning with task-specific labeled examples) has been successful in building task-specific models, such models can't generalize to unseen tasks; for example, a model that is supervised to solve questions cannot solve a classification task. We hypothesize that a model equipped with understanding and reasoning with natural language instructions should be able to generalize to any task that can be defined in terms of natural language.

Any empirical evidence that this might be true?

In our earlier effort, we built a smaller data (61 tasks) and observed that language models benefit from language instructions, i.e., their generalization to unseen tasks when they were provided with more instructions.
Also, generalization to unseen tasks improves as the model is trained on more tasks.

Why build this dataset?

We believe that our earlier work is just scratching the surface and there is probably so much that be studied in this setup. We hope to put together a much larger dataset that cover a wider range of reasoning abilities. We believe that this expanded dataset will serve as a useful playground for the community to study and build the next generation of AI/NLP models.

Task definitions

Each consists of input/output. For example, think of the task of sentiment classification:

  • Input: I thought the Spiderman animation was good, but the movie disappointed me.
  • Output: Mixed

Here is another example from the same task:

  • Input: The pumpkin was one of the worst that I've had in my life.
  • Output: Negative

Additionally, each ask contains a task definition:

Given a tweet, classify it into one of 4 categories: Positive, Negative, Neutral, or Mixed.

Overall, each tasks follows this schema:

Or if you're comfortable with json files, here is how it would look like:

{
  "Definition": "",
  "Positive Examples": [ { "input": "", "output": "",  "explanation": ""} ], 
  "Negative Examples": [ { "input": "", "output": "",  "explanation": ""} ],
  "Instances": [ { "input": "", "output": [""]} ],
}

How to contribute

We would appreciate any external contributions! 🙏

  • All submissions must be submitted via Github pull requests. These submissions will undergo a review before being merged.
  • Each task must contain contain a .json file that contains the task content. You can look inside the tasks/ directory for several examples.
    • Make sure that your json is human readable (use proper indentation; e.g., in Python: json.dumps(your_json_string, indent=4))
    • Make sure that you json file is not bigger than 50MB.
    • Make sure your task has no more 6.5k instances (input/output pairs).
    • Make sure to number your task json correctly (Look at the task number in the latest pull request, task number in your submission should be the next number). Make sure to include the source dataset name and the task category name while creating the json file name. You can use this format: taskabc__.json E.g. in task001_quoref_question_generation.json, the source dataset is 'quoref' and the task category is 'question generation. Note that, source need not necesserily be a dataset and can be a website e.g. leetcode. Finally, if you have created the json without any reference, use 'synthetic' in place of source.
    • Make sure to create a pull request after creating all possible tasks from a dataset. You should have one pull request per dataset.
    • If you're building your tasks based existing datasets and their crowdsourcing templates, see these guidelines.
  • Add your task to our list of tasks.
  • To make sure that your addition is formatted correctly, run the tests: > python src/test_all.py

If you have any questions or suggestions, please use the issues feature.

Frequently Asked Questions

Can I submit non-English tasks?

Yes! We welcome submission from any of the languages.

Can I create tasks without using crowdsourcing templates?

Yes! just make sure that the quality of instructions is good enough for a human to understand the task just based on instructions. You can take a different route than the guidelines.

What is the minimun number of instances I can have in my task?

Anything north of 100 is a safe number. The more, the merrier! Also, you should not have more than 6500 instances.

What do you mean by "meaningful contribution"?

If you're among top k contributors (say, k=25), or if you have contributed at least l tasks (say, l=10). Depending on the overall contributions, we will adjust these constants so that the number of authors don't exeed m (say, m=35).

My commits are not tied to my Github profie; Github does not show all my contributions.

Make sure that your email is set in your git environment and it is also mentioned in your github profile. See this and this.

Comments
  • Further update task domains & categories; Add Utility Scripts

    Further update task domains & categories; Add Utility Scripts

    • update domains and categories for tasks in mctao and cosmosqa
    • add script for auto adding domains for all task files of a given dataset
    • modify the test script to allow testing only a range of tasks (instead of ALL tasks)
    opened by garyhlai 24
  • Task 269-270: Counterfactual story generation

    Task 269-270: Counterfactual story generation

    Submitting the two tasks from issue #131 . First, task focuses on generating new story ending based on counterfactual context. Second, focuses on generating counterfactual context based on new and old endings such that it is counterfactual to initial context and justify the differences.

    opened by Maitreyapatel 21
  • Task 757-760: msr_sqa dataset

    Task 757-760: msr_sqa dataset

    Task 747-750: msr_sqa dataset

    Given a passage in tabular format, generate a question | Question generation Given a passage in tabular format and a question, generate a correct answer | Answer generation Given a passage in tabular format, generate an incorrect answer | Incorrect answer generation Given a passage in tabular format, generate a long text passage | Long text passage generation

    opened by sarnshreya 19
  • ATOMIC

    ATOMIC

    One can create tasks based on the atomic data, where the task definitions define the edges/relations in their dataset: https://allenai.org/data/atomic-2020

    task-suggestion 
    opened by danyaljj 17
  • Task 296-300: StoryCloze

    Task 296-300: StoryCloze

    Created 5 new tasks from the StoryCloze dataset (created 1 earlier)

    1. task296_StoryCloze_correct_end_answer_generation --> Given four sentences of five sentence story, select correct answer for last (fifth) sentence from the given option.
    2. task297_StoryCloze_incorrect_end_answer_generation --> Given four sentences of five sentence story, select incorrect answer for last (fifth) sentence from the given option.
    3. task298_StoryCloze_correct_end_classification --> Given four sentences of five sentence story and fifth sentence, classify whether the fifth sentence is proper or not by providing 'Yes' or 'No'.
    4. task299_StoryCloze_sentence_generation --> Given four sentences of five sentence story, provide the position and missing part (fifth sentence) of the story.
    5. task300_StoryCloze_order_generation --> Given five sentences of the story (sentences are shuffled), provide the correct order of the story.

    Please review.

    opened by Mirzyaaliii 16
  • Update Hierarchy for Task 059-081

    Update Hierarchy for Task 059-081

    • updated the domains & categories for task 059-081
    • files for task 063-064 are missing
    • I temporarily removed task 078 because the definition seems wrong (not matching with instance)
    • some nontrivial reorganization of the task hierarchy
    opened by garyhlai 14
  • Task 583-584 Universal Dependency English Parts of Speech Tagging

    Task 583-584 Universal Dependency English Parts of Speech Tagging

    Defined two tasks for Parts of Speech Tagging #94 using Universal Dependency English Dataset. Added an extra input "Position" to the ones suggested to distinguish words present multiple times within a sentence. Task 583: Universal Dependency English PoS tags coarse labels Task 583: Universal Dependency English PoS tags fine labels

    All PoS tags have equal number of instances ( unless very few examples present in original corpus for e.g. XX tag in fine labels, in which case all of its instances are added ).

    opened by mishra-sid 14
  • Tasks 747-749 and 614 from Glucose

    Tasks 747-749 and 614 from Glucose

    Hi, I created the first task and I will create a few more from this dataset (it has the potential for at least 10 tasks actually) paper: https://arxiv.org/pdf/2009.07758.pdf

    It'd be great if someone could review this, because I will create the other tasks in the same manner.

    opened by Palipoor 12
  • task 225-226: Stackoverflow english learner tasks

    task 225-226: Stackoverflow english learner tasks

    I have created two example tasks from Stack Overflow - English Language learner category. These tasks are very complex in terms of open ended, length of answers etc. Therefore, I think these should be a good addition.

    opened by Maitreyapatel 12
  • Data leakage with tasks 1295 and 1640

    Data leakage with tasks 1295 and 1640

    Task 1295 is a training task issued from the AdversarialQA dataset and Task 1640 is a testing task also issued from AdversarialQA. I would assume that this wasn't caught by automated checks because the URLs are different and there is a typo in the Source field of task 1640 where the value is "adverserial_qa"

    opened by axelmarmet 4
  • Dialog Tasks

    Dialog Tasks

    It looks like we don't have many dialog tasks, especially for response generation.

    This webpage contains a list of dialog tasks we can consider. I think we should include at least some of the popular tasks.

    My suggestion list: Dialog act classification - Switchboard corpus Dialog state tracking - Wizard-of-Oz, MultiWOZ Dialog generation - PERSONA-CHAT, DailyDilog++, DSTC7-challenge.

    task-suggestion 
    opened by yizhongw 2
  • Seemingly unrelated instances in tasks 1669 and 1670.

    Seemingly unrelated instances in tasks 1669 and 1670.

    The instances in tasks 1669 contain a lot of examples that don't seem related, or rather noisy. I'm not sure. The task is to "change male-gendered words and pronouns to female-gendered ones, or add words that are female-gendered" or the other way around (in the other task). But here are some examples from instances that are not doing that:

    		{
    			"input": "Hello Ardin, I am Tierny. Did you get his message?",
    			"output": [
    				"Hello Ardin, I am Tierny"
    			]
    		},
    		{
    			"input": "The place for apprenticeship was taken by Jerome Lamy",
    			"output": [
    				"The apprenticeship took place in Jerome Lamy"
    			]
    		},
    		{
    			"input": "i'm doing as well as a boy can. have you heard of the guy named sylvester denton?",
    			"output": [
    				"I'm doing as well as a boy can. Have you heard of Sylvia Denton?"
    			]
    		},
    		{
    			"input": "As men, I bet you wish we knew how she started her career in that field.",
    			"output": [
    				"as a man, I wish I knew how she started her career in that field."
    			]
    		}
    
    urgent 
    opened by Palipoor 1
  • Tasks 1285-1307: kpa dataset

    Tasks 1285-1307: kpa dataset

    These tasks have the same definitions with different domains. @ashok-arjun, Can you change the definitions? For example, we can have summary generation, argument generation, etc.

    urgent 
    opened by yeganehkordi 14
Releases(v2.7)
  • v2.7(Oct 10, 2022)

    What's Changed

    • fix typos by @manandey in https://github.com/allenai/natural-instructions/pull/770
    • Addressing data leakage by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/774
    • Fix a small typo in the tasks README by @aviaefrat in https://github.com/allenai/natural-instructions/pull/775

    New Contributors

    • @aviaefrat made their first contribution in https://github.com/allenai/natural-instructions/pull/775

    Full Changelog: https://github.com/allenai/natural-instructions/compare/v2.6...v2.7

    Source code(tar.gz)
    Source code(zip)
  • v2.6(Jul 1, 2022)

    What's Changed

    • Update tasks 200 - 600 based on human feedback by @Palipoor in https://github.com/allenai/natural-instructions/pull/761
    • Address feedbacks for tasks 600 - 800 by @Palipoor in https://github.com/allenai/natural-instructions/pull/762
    • Addressing crowdworker feedback by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/760
    • Update README.md by @swarooprm in https://github.com/allenai/natural-instructions/pull/763
    • Add standard evaluation setup by @yizhongw in https://github.com/allenai/natural-instructions/pull/764
    • Fix typo by @manandey in https://github.com/allenai/natural-instructions/pull/765
    • Fix arg name at rouge by @lkm2835 in https://github.com/allenai/natural-instructions/pull/766
    • Fix typos by @manandey in https://github.com/allenai/natural-instructions/pull/767
    • Updates for setting up the leaderboard by @yizhongw in https://github.com/allenai/natural-instructions/pull/768
    • Add description of how to produce reference file by @yizhongw in https://github.com/allenai/natural-instructions/pull/769

    New Contributors

    • @manandey made their first contribution in https://github.com/allenai/natural-instructions/pull/765
    • @lkm2835 made their first contribution in https://github.com/allenai/natural-instructions/pull/766

    Full Changelog: https://github.com/allenai/natural-instructions/compare/v2.5...v2.6

    Source code(tar.gz)
    Source code(zip)
  • v2.5(Apr 12, 2022)

    What's Changed

    • Update task json files with new sources, urls and categories. by @yizhongw in https://github.com/allenai/natural-instructions/pull/742
    • Add web nlg, personachat, quartz and wiqa by @Palipoor in https://github.com/allenai/natural-instructions/pull/740
    • Add a few lines to print the average number of instances by @Palipoor in https://github.com/allenai/natural-instructions/pull/743
    • Add missing tasks from prompt source by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/744
    • Add missing tasks from the prompt source by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/746
    • Fix a typo in the sources by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/749
    • Change order in the task 1296 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/747
    • Drop extra words from the definitions by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/750
    • Drop extra words from definitions by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/751
    • Update task1509_evalution_antonyms by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/753
    • Update task 575 and task 1599 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/756
    • Create a single file for crowd annotation using the tasks in test categories. by @yizhongw in https://github.com/allenai/natural-instructions/pull/758
    • Fix some lines that had extra columns in README by @Palipoor in https://github.com/allenai/natural-instructions/pull/754

    New Contributors

    • @yizhongw made their first contribution in https://github.com/allenai/natural-instructions/pull/742

    Full Changelog: https://github.com/allenai/natural-instructions/compare/v2.4...v2.5

    Source code(tar.gz)
    Source code(zip)
  • v2.4(Mar 26, 2022)

    What's Changed

    • Crowdworker evaluation of the tasks [Work in Progress] by @danyaljj in https://github.com/allenai/natural-instructions/pull/276
    • Revising categories: Tasks1-500 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/695
    • Remove a misleading phrase from instructions in tasks 702, 711 and 712 by @Palipoor in https://github.com/allenai/natural-instructions/pull/697
    • Add examples to task 697 and modify some instances to fix #661 by @Palipoor in https://github.com/allenai/natural-instructions/pull/698
    • Drop 1562 and 1563 because of vagueness reported in #685 by @Palipoor in https://github.com/allenai/natural-instructions/pull/699
    • Fix the imbalance issue in instances by @Palipoor in https://github.com/allenai/natural-instructions/pull/702
    • Task050 label imbalance bug fix by @swarooprm in https://github.com/allenai/natural-instructions/pull/704
    • Rename the duplicate task 119 by @Palipoor in https://github.com/allenai/natural-instructions/pull/708
    • Add English instances to task 265 by @Palipoor in https://github.com/allenai/natural-instructions/pull/707
    • Revising remaining tasks by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/705
    • Update test_all by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/710
    • Remove duplicate and incorrect domains for alt tasks and drop two tasks by @Palipoor in https://github.com/allenai/natural-instructions/pull/709
    • remove URLs from the tasks by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/711
    • Adding reasoning field - tasks1-50 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/712
    • Adding reasoning field- tasks51-200 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/713
    • Adding reasoning-tasks201-400 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/714
    • Adding reasoning- tasks401-700 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/715
    • Adding reasoning- tasks701-1014 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/716
    • Adding reasoning- tasks1015-1316 by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/717
    • Adding reasoning- tasks1317-end by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/718
    • Update test and hierarchy by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/720
    • Fix unicode characters by @Palipoor in https://github.com/allenai/natural-instructions/pull/723
    • Drop qualitative reasoning by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/724
    • Tasks1385-1390: ANLI, CB, HellaSwag, WSC by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/719
    • Script for updating categories, domains, and reasoning by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/722
    • Change definitions into lists of strings by @Palipoor in https://github.com/allenai/natural-instructions/pull/725
    • Move Mathematics and Ethical Judgement to reasoning by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/727
    • Drop factual reasoning by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/728
    • Fix map files by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/729
    • Winogrande tasks by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/730
    • Drop social commonsense reasoning and science branches by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/731
    • Merge tasks 288 and 1578 by @Palipoor in https://github.com/allenai/natural-instructions/pull/732
    • Merge all kpa tasks by @Palipoor in https://github.com/allenai/natural-instructions/pull/733
    • Update source and domain of aquamuse tasks by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/735
    • Add script to covert "prompt-source" tasks by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/734
    • Update convert_prompt.py by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/736
    • Add update_categories script by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/741
    • Create prompt_task_map.json by @yeganehkordi in https://github.com/allenai/natural-instructions/pull/737

    Full Changelog: https://github.com/allenai/natural-instructions/compare/v2.3...v2.4

    Source code(tar.gz)
    Source code(zip)
  • v2.3(Jan 26, 2022)

    More improvements: fixed bugs and improved the task category assignments.

    What's Changed

    • Fix for Issue #652. by @RushangKaria in https://github.com/allenai/natural-instructions-expansion/pull/680
    • Fix #488. Marked all pronouns in all instances. by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/676
    • Matres tasks enhancement by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/684
    • Update task1489_sarcasmdetection_tweet_classification.json by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/688
    • Tasks 1400-1425: Assigning categories, domains and addressing feedback by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/651
    • Tasks 1100-1200: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/683
    • Tasks 1550-1600: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/687
    • Tasks 1600-1726: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/690
    • Final version of task-hierarchy by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/691
    • Drop links from TriviaQA by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/689
    • Task 901-1000 Updated Files by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/693

    Full Changelog: https://github.com/allenai/natural-instructions-expansion/compare/2.2...v2.3

    Source code(tar.gz)
    Source code(zip)
  • 2.2(Dec 16, 2021)

    Fixing bugs and improving the task categorization.

    What's Changed

    • Fix the missing perspectives in task 738 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/646
    • Fixed human eval issues in tasks 1700+ and some grammar issues by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/644
    • Fixing issue #592 - Updating task 564 by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/636
    • Address feedback for tasks 1300 1400 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/638
    • Addressed Feedback for tasks 738-800 by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/647
    • Address feedbacks for tasks 1600 - 1657 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/641
    • Tasks 551-575: Assigning categories, domains, and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/594
    • Fixes Issue #577. by @RushangKaria in https://github.com/allenai/natural-instructions-expansion/pull/649
    • improve HateEval tasks by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/650
    • update tasks 1601 and 1602 to address issue #639 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/653
    • Update examples to Croatian / fix characters tasks 1626 - 1629 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/643
    • Tasks 1283-1300: Assigning categories, domains and addressing feedback by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/625
    • Update Categories and Domains for Task161-180 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/593
    • update categories and domains for task181-199 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/606
    • Tasks 1300-1325: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/659
    • Tasks 1425-1450: Assigning categories, domains and addressing feedback by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/656
    • fixed #524 by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/655
    • MMMLU task improvements by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/654
    • improve Jigsaw tasks by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/648
    • Tasks 1325-1350: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/660
    • Tasks 1200-1225: Assigning categories, domains and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/601
    • Added negative examples for MMMLU tasks. Fixes #569. by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/663
    • Removed incomplete instances from task667. Fixes #664. by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/665
    • Assigning Categories to Tasks 601 - 700 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/657
    • categories & domains for task300-321 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/666
    • Tasks 1350-1400: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/667
    • Tasks1000-1100: Assigning categories and domains by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/670
    • fixed explanation of task456 (fixes #581) by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/669
    • Task322-399 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/668
    • Updating Categories from Tasks 700-800 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/671
    • Updating categories Tasks 801-900 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/672
    • Tasks 1450-1500: Assigning categories, domains and addressing feedback by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/658
    • Tsk 1361: fix by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/674
    • Fix semeval by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/675
    • adding input language in definition assertion by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/678
    • Task463-500, 1500-1550 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/679
    • Fix the imbalanced labels in task 27 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/681
    • Fix imbalanced labels in tasks 200 and 202 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/682

    Full Changelog: https://github.com/allenai/natural-instructions-expansion/compare/v2.1...2.2

    Source code(tar.gz)
    Source code(zip)
  • v2.1(Nov 17, 2021)

    This release contains:

    • A collection of newly added tasks
    • A new hierarchy of task types and their domains
    • Lots of improvements and fixes to the existing tasks.

    We will continue to publish more releases as we improve the data and add more experiments. If you're wondering whether you can still contribute to the repo, the answer is a solid YES!

    What's Changed

    • Further update task domains & categories; Add Utility Scripts by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/436
    • Tasks 739-742: lhoestq dataset question and answer generation by @hanut1909 in https://github.com/allenai/natural-instructions-expansion/pull/317
    • Task 829-833: giga_fren, poem_sentiment and poleval2019_mt by @abhinawale12 in https://github.com/allenai/natural-instructions-expansion/pull/325
    • Tasks 877-881: kde4, schema_guided_dstc8 Datasets by @kashyap467 in https://github.com/allenai/natural-instructions-expansion/pull/334
    • Task 934 Turk Simplification by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/444
    • Tasks 1499-1504 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/461
    • Update zest tasks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/455
    • Address human feedback by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/447
    • Tasks600-606 by @Nikhitha0911 in https://github.com/allenai/natural-instructions-expansion/pull/285
    • corrections in the evalution task file by @atharva-naik in https://github.com/allenai/natural-instructions-expansion/pull/460
    • Tasks 1443 - 1446: Synthetic data by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/449
    • Tasks 1394-1397 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/416
    • Tasks 1361-1364 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/410
    • Tasks 1437-1442 : DoQA tasks by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/448
    • Tasks 1356-1360 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/409
    • Tasks906-909 DialogRE Answer Generation by @matthew-huff in https://github.com/allenai/natural-instructions-expansion/pull/341
    • Making the error message informative by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/466
    • Tasks 1365-1369 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/411
    • Second PR to Address human feedback 34, 35, 44, 45, 48, 49, 52-58, 167, 201-205 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/464
    • Task 1498 by @sharma121amit in https://github.com/allenai/natural-instructions-expansion/pull/456
    • Task 955 Rewrite simple English wikipedia sentences in more sophisticated English by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/354
    • Task 1505: root09 lexical semantic relation classification task added by @atharva-naik in https://github.com/allenai/natural-instructions-expansion/pull/470
    • Tasks910-916: Bianet and Imppres by @mirror3 in https://github.com/allenai/natural-instructions-expansion/pull/342
    • Tasks 766-769 craigslist_bargains, qed by @ritvik7 in https://github.com/allenai/natural-instructions-expansion/pull/321
    • Tasks 917-921: CoQA Dataset, code_x_glue Dataset by @ShashankDavalgi in https://github.com/allenai/natural-instructions-expansion/pull/343
    • Task 1308-1313 Amazon review by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/395
    • Address human feedback- tasks 301 - 332 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/469
    • Update categories and domains for tasks 029 - 058 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/459
    • Address human feedback Tasks 333 - 352 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/471
    • Tasks 1350-1355 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/408
    • A script to fix unicode characters in files by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/473
    • Tasks 1506 and 1507 (get minimal text span of dob of celebrity given bio) by @atharva-naik in https://github.com/allenai/natural-instructions-expansion/pull/472
    • Tasks 1334-1339 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/405
    • Tasks 1370-1377 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/412
    • Tasks 1378-1384 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/413
    • Addressing Human Feedback (Crowdworker Evaluation of Tasks) by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/476
    • Tasks 1340-1343 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/406
    • Tasks 1344-1349 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/407
    • Task 902-905: Deceptive Opinion Spam dataset, hate_speech_offensive by @meghpatel in https://github.com/allenai/natural-instructions-expansion/pull/340
    • Task 820-826: proto_qa & peixian/rtGender by @Mda233 in https://github.com/allenai/natural-instructions-expansion/pull/323
    • Tasks 970-1086: sherliic, prachathai67k and pib by @CodeHime in https://github.com/allenai/natural-instructions-expansion/pull/360
    • Tasks 960-967 by @ssingulu in https://github.com/allenai/natural-instructions-expansion/pull/357
    • Tasks 595-599: MOCHA and CUAD Datasets by @akhilkumargudipoodi in https://github.com/allenai/natural-instructions-expansion/pull/284
    • Correcting definition for Task 078 by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/485
    • Address human feedback on tasks 374 - 392 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/479
    • Address human feedback, tasks 397 - 400 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/480
    • Task 619-624 Ohsumed abd Onestop_english datasets by @papanisaicharan in https://github.com/allenai/natural-instructions-expansion/pull/300
    • Task 636-639: multi_voz_v22 dataset by @ayushkalani in https://github.com/allenai/natural-instructions-expansion/pull/304
    • Task 645 - 648: Wiki auto and winograd dataset by @khushal1996 in https://github.com/allenai/natural-instructions-expansion/pull/306
    • Tasks 676-684: OPP-115, HopeEDI, Ollie Datasets by @sskp-kaushik in https://github.com/allenai/natural-instructions-expansion/pull/315
    • Task 757-760: msr_sqa dataset by @sarnshreya in https://github.com/allenai/natural-instructions-expansion/pull/319
    • Tasks 838-842: cdt & para_pdt Datasets by @akhileshamara in https://github.com/allenai/natural-instructions-expansion/pull/327
    • Tasks 1218-1282: TED translation task for an additional 65 language directions by @davidstap in https://github.com/allenai/natural-instructions-expansion/pull/478
    • Task 1196-1217: ATOMIC by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/390
    • Tasks 857-861 by @ashutosh1608 in https://github.com/allenai/natural-instructions-expansion/pull/331
    • Tasks 872-876: opus_xhosanavy, emotion by @aklagoo in https://github.com/allenai/natural-instructions-expansion/pull/335
    • Tasks 886-889: Quail, Rotten Tomatoes and Go Emotions Dataset by @saianirud in https://github.com/allenai/natural-instructions-expansion/pull/337
    • task 868-871 SQL and MSMARCO by @ujjwalaananth in https://github.com/allenai/natural-instructions-expansion/pull/359
    • Update Hierarchy for Task 059-081 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/474
    • Task 894- 901: miam, freebase_qa by @karthikmuru in https://github.com/allenai/natural-instructions-expansion/pull/339
    • Tasks 968-969 : Xcopa by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/358
    • Tasks 922-926:Event2Mind and Coached_Conv_Pref dataset by @yashbhokare in https://github.com/allenai/natural-instructions-expansion/pull/344
    • Tasks 929-932 by @karannaik3797 in https://github.com/allenai/natural-instructions-expansion/pull/347
    • Task 956: leetcode 420 by @DZuoShi in https://github.com/allenai/natural-instructions-expansion/pull/355
    • Task 1161-1164 Coda19 by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/376
    • Task 1542 by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/500
    • Task 1166-1167 PennTreeBank and Brown Corpus Parts of Speech Tags by @mishra-sid in https://github.com/allenai/natural-instructions-expansion/pull/380
    • Tasks 1168-1185 : Xcopa by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/386
    • Tasks 1283-1284: Evaluating quality and Informativeness of System Generated Reference by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/393
    • Tasks 1323-1330 by @abhilashreddyy in https://github.com/allenai/natural-instructions-expansion/pull/402
    • Tasks 1407-1417: DART, ajgt_twitter_ar, and youtube_caption_corrections. by @RushangKaria in https://github.com/allenai/natural-instructions-expansion/pull/429
    • Revert "Tasks 1407-1417: DART, ajgt_twitter_ar, and youtube_caption_corrections." by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/512
    • Task1447-1453 and Tasks 1479-1497 Drug_Extraction by @Ishani-Mondal in https://github.com/allenai/natural-instructions-expansion/pull/450
    • 576 HW (1-5) by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/513
    • Task 1551 + Missing Task Numbers from Issue #482 by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/502
    • task_201_resolve by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/521
    • Address human feedback for tasks 400 - 420 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/499
    • Address feedbacks for tasks 428 - 452 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/506
    • Update test_all by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/427
    • Added task 1431-1436 by @shreeshiv in https://github.com/allenai/natural-instructions-expansion/pull/446
    • Update domains, categories, and address human feedbacks: Task 231-260 by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/484
    • Update categories, domains and addressing evaluation feedbacks for tasks 200-230 by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/463
    • renamed task 1511-1514 to 1582-1585 by @atharva-naik in https://github.com/allenai/natural-instructions-expansion/pull/527
    • 576 HW Tasks (part 2) by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/525
    • Edited filenames for PR #429 (Tasks 1407-1417: DART, ajgt_twitter_ar, and youtube_caption_corrections). by @RushangKaria in https://github.com/allenai/natural-instructions-expansion/pull/533
    • Tasks 1578-1581 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/511
    • Tasks 1557-1560 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/507
    • Tasks 1586-1589 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/515
    • Tasks 1590-1594 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/516
    • Tasks 1595-1598 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/517
    • Tasks 1662-1665 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/538
    • Tasks 1548-1549 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/531
    • Tasks 1599-1603 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/518
    • Tasks 1654-1657 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/536
    • Tasks 1604-1611 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/519
    • Tasks 1676-1678 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/541
    • Tasks 1703-1706 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/547
    • Task 1626 - 1631 by @AnshitaSinghBais in https://github.com/allenai/natural-instructions-expansion/pull/529
    • print contributed authors by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/557
    • address human feedback for tasks 477 - 500 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/556
    • Tasks 1719-1726 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/550
    • update categories & domains for task 082-100 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/520
    • Tasks 1685-1692 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/543
    • Tasks 1711-1714 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/549
    • Tasks 1645-1653 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/535
    • Tasks 1666-1670 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/539
    • 1637-1640 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/532
    • Tasks 1658-1661 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/537
    • Tasks 1612-1625 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/528
    • Address human feedback tasks 477 - 500 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/558
    • Address human feedback on tasks 600 - 618 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/561
    • Fixing Issue for 1361 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/566
    • Address human feedback for tasks 630 - 672 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/567
    • Address feedback for tasks 664 - 700 and fix a repeated misspelling across a lot of files. by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/568
    • Tasks 261-300: update categories, domains and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/565
    • Assigning categories for Task101-121 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/559
    • update task 122-140 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/571
    • fix xcopa_cause_effect_th by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/564
    • Tasks 500-525: Assigning categories, domains, and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/573
    • Fixed typos Tasks 570, 571 by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/590
    • Fixes Issue #582. by @RushangKaria in https://github.com/allenai/natural-instructions-expansion/pull/596
    • added language tags for tasks 938-954 by @tanay2001 in https://github.com/allenai/natural-instructions-expansion/pull/586
    • Update languages for tasks 927 - 1088 and address some human feedback. by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/595
    • Fix spacing problem for task 614 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/597
    • update categories & domains for task 141-160 by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/585
    • Fixing issue #489. Improvements to tasks 886-889 by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/598
    • Failing tests in the main branch by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/603
    • Update categories and domains for tasks 400-462 by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/475
    • Tasks 525-550: Assigning categories, domains, and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/587
    • Address feedback for tasks 1100 - 1200 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/605
    • fix agnews by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/609
    • Instructions fixed and human feedback addressed in mmmlu answer generation tasks (664-667, 685-737) by @pulkitverma25 in https://github.com/allenai/natural-instructions-expansion/pull/607
    • Tasks 576-600: Assigning categories, domains and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/600
    • Drop an outlier instance from task 264 by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/613
    • Update test_all.py test for single-output tasks by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/614
    • Downsampling by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/618
    • More downsampling. by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/620
    • Tasks 1226-1282: Assigning categories, domains, and addressing feedbacks by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/608
    • Downsampling skewed datasets by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/624
    • Task226 shuffle by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/627
    • print categories and domains at end of the logs. by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/632
    • Task 1148 ( maximum_ascii_value) : Edit Task definition to include ASCII information. by @mishra-sid in https://github.com/allenai/natural-instructions-expansion/pull/633
    • Output Language Tasks 939,941,943 by @tanay2001 in https://github.com/allenai/natural-instructions-expansion/pull/635
    • task1186_nne_hrngo_classification improvement by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/634
    • Fixes for Issue #583 by @CodeHime in https://github.com/allenai/natural-instructions-expansion/pull/628

    New Contributors

    • @hanut1909 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/317
    • @abhinawale12 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/325
    • @kashyap467 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/334
    • @Nikhitha0911 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/285
    • @matthew-huff made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/341
    • @sharma121amit made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/456
    • @mirror3 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/342
    • @ritvik7 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/321
    • @ShashankDavalgi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/343
    • @meghpatel made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/340
    • @Mda233 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/323
    • @CodeHime made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/360
    • @ssingulu made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/357
    • @akhilkumargudipoodi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/284
    • @papanisaicharan made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/300
    • @ayushkalani made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/304
    • @khushal1996 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/306
    • @sskp-kaushik made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/315
    • @sarnshreya made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/319
    • @akhileshamara made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/327
    • @ashutosh1608 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/331
    • @aklagoo made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/335
    • @saianirud made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/337
    • @ujjwalaananth made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/359
    • @karthikmuru made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/339
    • @yashbhokare made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/344
    • @karannaik3797 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/347
    • @DZuoShi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/355
    • @pulkitverma25 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/500
    • @abhilashreddyy made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/402
    • @RushangKaria made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/429
    • @Ishani-Mondal made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/450
    • @shreeshiv made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/446
    • @AnshitaSinghBais made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/529

    Full Changelog: https://github.com/allenai/natural-instructions-expansion/compare/v2.0...v2.1

    Source code(tar.gz)
    Source code(zip)
  • v2.0(Oct 18, 2021)

    We wrapped up our first public expansion effort on October 15, 2021. During this time, the community contributed over 1500 tasks!! 🎉 We will continue to publish more releases as we improve the data and add more experiments. If you're wondering whether you can still contribute to the repo, the answer is a solid YES!

    Note: v1 data is accessible here: https://instructions.apps.allenai.org/

    What's Changed

    • natural instructions-v1 by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/1
    • Updating Task Category in Readme by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/2
    • Repeat-copy task by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/4
    • Changed output format in examples to non-list, faq by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/9
    • Update README.md by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/12
    • task 73 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/11
    • Shailaja tasks by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/14
    • Added 4 tasks using SPLASH dataset and 1 using the CoNaLa dataset. by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/15
    • Update README.md: clarifying "meaningful contribution" by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/23
    • babi dataset task 1- 3 NI tasks (question generation, answer generation, identify supporting fact for QA) by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/22
    • Update README.md: commits not connected to Github profile by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/30
    • Adding tasks using examples from CoNaLa dataset. by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/29
    • Removing repititive examples. by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/40
    • Repeat instance fix by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/42
    • squad2.0 by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/32
    • Fixing several outstanding issues by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/38
    • Task103 facts to story by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/34
    • Updated files for task 102 and 103 by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/49
    • Update README.md by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/50
    • fixing issue #13, #43, #31, #24 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/53
    • removing randomization (issue #55) by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/56
    • Synthetic data by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/27
    • task108 abuse detection by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/41
    • ASSET two tasks 111 and 112 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/51
    • Synthetic frequency tasks by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/57
    • Task 105 sentence generation by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/62
    • Add task117_spl_translation_en_de task by @Mehrad0711 in https://github.com/allenai/natural-instructions-expansion/pull/66
    • task 104 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/35
    • Update crowdsourcing.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/69
    • Update task018_mctaco_temporal_reasoning_presence.json by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/67
    • Update task019_mctaco_temporal_reasoning_category.json by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/68
    • Task115 help classification by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/59
    • PIQA Dataset - Two subtasks by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/17
    • Task 67-72: AbductiveNLI by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/10
    • Task 119, 120 and 121: ZEST dataset by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/79
    • Task 65 -66: Time travel by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/8
    • Task 126-131: SCAN Dataset by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/81
    • Task 116 Com2sense task - Commonsense reasoning by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/60
    • Task 137-140: Detoxifying LMs Prompt Completion Dataset by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/84
    • Update README.md by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/86
    • Task 141-143: Odd Man Out Dataset by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/88
    • Task 151-154: Theory of Mind QA Dataset by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/91
    • Task 155: Count number of nouns/verbs in the given sentence by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/92
    • Add task 144 from SubjQA by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/89
    • Task110_logic2text by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/100
    • Task 156: CODAH Dataset by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/96
    • Task 122-125: New Conala tasks by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/80
    • Task 133-136: WinoWhy (ACL 2020 paper) by @colinzhaoust in https://github.com/allenai/natural-instructions-expansion/pull/83
    • Task 164-165: MCScript by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/105
    • Task166: ClariQ by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/106
    • Task 177: Para-nmt by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/110
    • Task 132: DAIS Dataset by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/82
    • Task 182: DuoRC dataset by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/115
    • Task 157: Count vowels consonants by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/97
    • Task 158 - 159: Frequency of words by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/98
    • Task 160 : Replace letter in sentence by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/99
    • Task 161-163: Count words that contain/start_with/end_with the given letter by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/101
    • Task 109 - Spam SMS Detection by @SavanDoshi in https://github.com/allenai/natural-instructions-expansion/pull/117
    • task 170, 191-192: HotpotQA by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/107
    • Tasks 179 - 181 from the paper "Effective Crowd-Annotation of Participants, Interventions, and Outcomes..." by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/114
    • Task 176 and Task 184 from dataset BREAK by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/109
    • Task 167-169: StrategyQA dataset by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/104
    • fix: typo spelling grammar by @slowy07 in https://github.com/allenai/natural-instructions-expansion/pull/125
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/126
    • Fixed unit test by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/124
    • Task 118-119: semeval math challenges by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/76
    • Task - 166 ClariQ Correction by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/134
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/139
    • Task 195-196: Sentiment140 by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/122
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/146
    • Task - 178: QuaRTz by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/111
    • Tasks 205-208: New Synthetic tasks by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/123
    • Add tasks 171-175 by @Mehrad0711 in https://github.com/allenai/natural-instructions-expansion/pull/108
    • adding tasks 145-150 by @liusiyi641 in https://github.com/allenai/natural-instructions-expansion/pull/90
    • Task 193-194: DuoRC by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/121
    • task 224 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/137
    • Changing the Expected Schema by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/147
    • Task209 stance detection by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/128
    • Task 210-212: logic2text by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/129
    • Task-223: QuaRTz by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/136
    • Task 227: ClariQ by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/141
    • Task 243- 245: Set operation tasks by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/148
    • Task 228-229: ARC by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/142
    • Task 239-242: TweetQA by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/145
    • Task 246-248: DREAM by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/149
    • Task 276 and 275 and 249 from enhanced wsc dataset by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/150
    • Task 268: CaseHOLD by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/155
    • Tasks 271-273: Europarl by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/157
    • Task 274: overruling by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/158
    • Task 283: DREAM by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/163
    • Task 287: CaseHOLD by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/166
    • task 290 : TellMyWhy by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/168
    • Task 301-303: ReCoRD by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/172
    • Task 281 Points of Correspondence detection by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/161
    • Task - 229: ARC (Corrected file) by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/181
    • task 225-226: Stackoverflow english learner tasks by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/140
    • Task 339: ReCoRD by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/186
    • Remove extra file by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/184
    • Update README.md by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/189
    • Task 183 add rhyming task by @ethankim00 in https://github.com/allenai/natural-instructions-expansion/pull/116
    • Task 184-190 SNLI Tasks, 197-204 MNLI tasks by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/120
    • task 282 : Scruples by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/162
    • tasks 250-253: more from SPL by @Mehrad0711 in https://github.com/allenai/natural-instructions-expansion/pull/151
    • tasks 254-263: SPL translation tasks by @Mehrad0711 in https://github.com/allenai/natural-instructions-expansion/pull/152
    • Task 277 - 280: StereoSet tasks by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/159
    • task 286 : OLID by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/165
    • Task 318-321: StereoSet by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/179
    • Task 380-381 from BoolQ by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/201
    • Task 348-349: Squad2.0 by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/190
    • Task 344-347 and Task 382: HybridQA by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/188
    • task 291 & 295 : semeval 2020 task4 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/169
    • task 322 - 328: Jigsaw by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/180
    • Add task 375 from paper "Exploring the Role of Argument Structure in Online Debate Persuasion" by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/198
    • Task 312-315: EuroParl (Swedish and English) by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/177
    • Task 394-396: PersianQA by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/210
    • Update test_all.py by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/213
    • Task 332: TellMeWhy by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/183
    • Task 379: AGNews Topic Classification by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/200
    • Task 363: SST2 by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/195
    • Task 391-393 from cod3s paper by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/209
    • Task 364 from "Towards Controllable Biases in Language Generation" by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/196
    • Task 312-315 (Corrected files) by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/214
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/218
    • Tasks 213-222 ROCStories by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/132
    • Tasks 284-285: IMDB by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/164
    • Tasks 288-289: Gigaword by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/167
    • Task 305-308: Jeopardy by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/175
    • Task 353-359: CaSiNo Dataset by @kushalchawla in https://github.com/allenai/natural-instructions-expansion/pull/193
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/221
    • Enhancements to README and test_all.py by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/224
    • Task 376-378 Word length tasks by @nrjvarshney in https://github.com/allenai/natural-instructions-expansion/pull/199
    • Task 316 - 317: CrowS-Pairs by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/178
    • Task 386-387 : Semeval2018 task3 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/204
    • Fix type keys by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/228
    • bringing the branch up to date by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/234
    • Update test_all.py: check the types of several recently added keys. by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/205
    • Updated FAQ by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/236
    • Update README.md: deadline update September -> October by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/256
    • Minor Edits to Task 137-140 by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/265
    • Tasks 230-238 IIRC Tasks by @aarunku5 in https://github.com/allenai/natural-instructions-expansion/pull/144
    • Task 296-300: StoryCloze by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/171
    • Task 304 and 401 from paper "Where’s My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution" by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/173
    • task 333 - 338: HateEval by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/185
    • Tasks 365-374 Synthetic Data and Tasks 517-520 by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/197
    • task 403 : Creak by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/223
    • Task 463-464: ParsiNLU-entailment by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/239
    • Task 465-466: ParsiNLU-QQP by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/240
    • Task 475: Yelp Polarity Classification by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/244
    • Task 383 and Task 456-459: MATRES by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/202
    • Task 460-462: Qasper by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/230
    • Task 467-468: ParsiNLU-reading_comprehension by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/241
    • Task 476-487: CLS Dataset - Multilingual Sentiment Classification of Product Reviews by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/245
    • Add tasks 513 and 514 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/261
    • Task512: Twitter Emotion Classification by @gkaramanolakis in https://github.com/allenai/natural-instructions-expansion/pull/260
    • Task 418-423 PerSent Dataset by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/229
    • Task 424-427: HindiEnglish Corpora by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/231
    • Tasks 573-578 air_dialog and curiosity_dialog instructions by @jayavardhan3112 in https://github.com/allenai/natural-instructions-expansion/pull/271
    • task 400 : PAWS by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/215
    • Tasks 607-610 by @arlenfan in https://github.com/allenai/natural-instructions-expansion/pull/287
    • Task 534 : farstail entailment by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/275
    • Task 352: CODA-19 by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/192
    • Task 329-331: GAP by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/182
    • Task 530-533 EuroParl Spanish dataset by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/274
    • Task 524-529: ParsiNLU-sentiment-analysis by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/272
    • Task 535-564 ALT & Discofuse Datasets by @PhaniRohithaKaza in https://github.com/allenai/natural-instructions-expansion/pull/269
    • Task 515-516 : Senteval by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/262
    • task 397-399 : Semeval2018 task1 by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/212
    • Task 360-362 : "Yes, and" response generation and classification. by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/194
    • Task 309-311: RACE by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/176
    • Task 615 Movieqa by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/297
    • Task 582 Natural Questions by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/278
    • Task 579-581 Socialiqa by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/277
    • Minor Edits To My Previously Merged Tasks by @eshaanpathak in https://github.com/allenai/natural-instructions-expansion/pull/313
    • Task 292-294: StoryCommonsense by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/170
    • Task 428-431 : Sent eval by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/232
    • Tasks 469-472: from MRQA and hasPart KB datasets by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/242
    • Task 473-474: ParsiNLU-multiple-choice by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/243
    • Task 521 - classification from QANTA by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/263
    • Tasks 640-644: esnli Dataset and refresd Dataset by @ilobo22 in https://github.com/allenai/natural-instructions-expansion/pull/305
    • task 340 - 343: WinoMT by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/187
    • task 350 - 351: WinoMT by @XudongOliverShen in https://github.com/allenai/natural-instructions-expansion/pull/191
    • Tasks 510-511: Reddit Tifu dataset by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/259
    • task 611 : Mutual by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/290
    • Task 616 CoLA by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/298
    • Task 269-270: Counterfactual story generation by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/156
    • Tasks 438-441: English-Gujarati Language by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/235
    • Task 405 from NarrativeQA by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/226
    • Task 522 News Editorials by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/267
    • add tasks 402 and 404 by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/225
    • Tasks 388-390: Torque (merge issue fix) by @shailaja183 in https://github.com/allenai/natural-instructions-expansion/pull/258
    • Tasks 747-749 and 614 from Glucose by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/296
    • Task 668: TLDR; summarizing paper abstracts by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/311
    • Task - 649: RACE by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/309
    • Task 650-663: ParsiNLU-translation by @yeganehkordi in https://github.com/allenai/natural-instructions-expansion/pull/310
    • Tasks 442-452 : opus_paracrawl and com_qa datasets by @Arutselvan in https://github.com/allenai/natural-instructions-expansion/pull/237
    • Tasks 664-667 and 685-737: Measuring Massive Multitask Language Understanding by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/308
    • task 827-828 : copa by @amirrezamirzaei in https://github.com/allenai/natural-instructions-expansion/pull/324
    • Task 933 Style Transfer (Simplify Sentences) by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/348
    • Task 617-618 Amazon Review by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/299
    • Task 612 Yoruba BBC headline by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/291
    • Task 843-849: financial_phrasebook, pubmed_QA Datasets. by @JalanshMunshi in https://github.com/allenai/natural-instructions-expansion/pull/328
    • Tasks 672-675 : Amazon/Yelp reviews and Google welformed query by @paik1 in https://github.com/allenai/natural-instructions-expansion/pull/314
    • Tasks 853-856: HIPPOCAMPUS and conv_ai_2 by @NeilFranks in https://github.com/allenai/natural-instructions-expansion/pull/330
    • Task 770-819: PAWS-X and PEC by @andersonjwan in https://github.com/allenai/natural-instructions-expansion/pull/322
    • Tasks 406-417 : Mickey tasks by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/227
    • Task 672, 750-754, 861-868: NumerSense Task by @tanay2001 in https://github.com/allenai/natural-instructions-expansion/pull/307
    • Tasks 629-635: dbpedia_14 and allegro_reviews datasets by @divya0627 in https://github.com/allenai/natural-instructions-expansion/pull/303
    • Tasks 834-837: Math Dataset & Viquiquad by @chiragvartak in https://github.com/allenai/natural-instructions-expansion/pull/326
    • Tasks 1087-1089 and 1146-1147 - Synthetic Data - Answer Generation by @Ravsehajsinghpuri in https://github.com/allenai/natural-instructions-expansion/pull/363
    • Tasks 453-455 : Swag Dataset by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/238
    • Task 761-765: app_review and emea by @vijaykumawat256 in https://github.com/allenai/natural-instructions-expansion/pull/320
    • Task 613 Politifact by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/293
    • Tasks 850-852: Synthetic data by @kurbster in https://github.com/allenai/natural-instructions-expansion/pull/329
    • Tasks 625-628 by @ananth-duggirala in https://github.com/allenai/natural-instructions-expansion/pull/302
    • Tasks 565-572 by @BhavyaSri1001 in https://github.com/allenai/natural-instructions-expansion/pull/270
    • Task 493-496: Amazon Polarity and Semeval by @SavanDoshi in https://github.com/allenai/natural-instructions-expansion/pull/253
    • Tasks 1090-1114: TED translation task for 25 language directions. by @davidstap in https://github.com/allenai/natural-instructions-expansion/pull/364
    • Task 432-437: alt (language translation) by @SavanDoshi in https://github.com/allenai/natural-instructions-expansion/pull/233
    • Task 585 Preposition by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/280
    • Task 738 from perspectrum by @Palipoor in https://github.com/allenai/natural-instructions-expansion/pull/316
    • Task 489-492: mwsc Dataset by @NSVPC in https://github.com/allenai/natural-instructions-expansion/pull/252
    • Tasks 264-266: Paper Reviews by @Mihir3009 in https://github.com/allenai/natural-instructions-expansion/pull/153
    • Tasks 743-746 EURLEX,YELP and ARITHMETIC by @RaghulRajM in https://github.com/allenai/natural-instructions-expansion/pull/318
    • Task 583-584 Universal Dependency English Parts of Speech Tagging by @mishra-sid in https://github.com/allenai/natural-instructions-expansion/pull/279
    • Task 586-590 Amazon food review by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/281
    • Tasks 1148-1153 by @Ravsehajsinghpuri in https://github.com/allenai/natural-instructions-expansion/pull/370
    • Task 1152-1159: BARD analogical reasoning dataset by @ashok-arjun in https://github.com/allenai/natural-instructions-expansion/pull/374
    • Task 669-671: AmbigQA by @Mirzyaaliii in https://github.com/allenai/natural-instructions-expansion/pull/312
    • Task hierarchy by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/361
    • Revert "Task hierarchy" by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/391
    • Create task-hierarchy.md in doc--In progress by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/389
    • Tasks 935-937: Defeasible reasoning task/dataset by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/352
    • Task 1314: Country Abbreviation by @sumantapatro in https://github.com/allenai/natural-instructions-expansion/pull/397
    • Task 384-385 Socialiqa(replaced) by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/379
    • Tasks 927-928 : style transfer by @tanay2001 in https://github.com/allenai/natural-instructions-expansion/pull/345
    • Tasks 938-954 Indic Glue by @tanay2001 in https://github.com/allenai/natural-instructions-expansion/pull/353
    • Tasks: 1130-1145 X-CSR commonsense multiple QA by @Maitreyapatel in https://github.com/allenai/natural-instructions-expansion/pull/368
    • Tasks 1315-136 and 1188-1194 by @Ravsehajsinghpuri in https://github.com/allenai/natural-instructions-expansion/pull/387
    • Updating the task hierarchy for two tasks by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/400
    • Task 497-509 : Scruples-Anecdote-Dilemmas-datasets by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/255
    • Tasks 890-893: GCWD and gap by @selvagmj in https://github.com/allenai/natural-instructions-expansion/pull/338
    • Task 1115-1129 ALT by @SavanDoshi in https://github.com/allenai/natural-instructions-expansion/pull/367
    • Task 591-594 SciQ dataset by @kuntalkumarpal in https://github.com/allenai/natural-instructions-expansion/pull/282
    • adding language indicators by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/419
    • flag repeated lines. by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/422
    • Fix file issues outside the tasks folder by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/424
    • Update test_all.py by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/425
    • tasks 1331-1333 by @Ravsehajsinghpuri in https://github.com/allenai/natural-instructions-expansion/pull/403
    • Verify language naming conventions by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/423
    • Task 1195: Disfluent to fluent sentence conversion by @ashok-arjun in https://github.com/allenai/natural-instructions-expansion/pull/388
    • Tasks 1398-1402 by @swarooprm in https://github.com/allenai/natural-instructions-expansion/pull/417
    • Tasks 1403-1406 by @Ravsehajsinghpuri in https://github.com/allenai/natural-instructions-expansion/pull/426
    • Task 1285-1307: Keypoint Matching - 23 topics by @ashok-arjun in https://github.com/allenai/natural-instructions-expansion/pull/394
    • Initial Task Hierarchy Revision & Auto Update Readme by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/428
    • Fix the inconsistencies of the definitions by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/432
    • Task 1418: bless lexical semantics task by @atharva-naik in https://github.com/allenai/natural-instructions-expansion/pull/433
    • Task 957-959: E2E by @ghlai9665 in https://github.com/allenai/natural-instructions-expansion/pull/356
    • Updating the tests: check if the tasks are sorted correctly. by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/434
    • Update README.md by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/439
    • Task 1419 - 1424 : MathQA by @mishra-sid in https://github.com/allenai/natural-instructions-expansion/pull/438
    • Task 1187 Politifact 2(Reviewed) by @cosmicishan in https://github.com/allenai/natural-instructions-expansion/pull/437
    • Task1186: natural-ness evaluation by @Sujan242 in https://github.com/allenai/natural-instructions-expansion/pull/371
    • Tasks 1425-1428 and 1317-1322 by @sumantapatro in https://github.com/allenai/natural-instructions-expansion/pull/398
    • Updating the readme [to be merged on October 15] by @danyaljj in https://github.com/allenai/natural-instructions-expansion/pull/381

    New Contributors

    • @danyaljj made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/1
    • @swarooprm made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/2
    • @amirrezamirzaei made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/11
    • @shailaja183 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/14
    • @kurbster made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/15
    • @Mihir3009 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/34
    • @nrjvarshney made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/27
    • @Maitreyapatel made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/41
    • @Palipoor made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/51
    • @Mirzyaaliii made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/62
    • @Mehrad0711 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/66
    • @kuntalkumarpal made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/17
    • @aarunku5 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/10
    • @yeganehkordi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/79
    • @eshaanpathak made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/81
    • @colinzhaoust made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/83
    • @SavanDoshi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/117
    • @slowy07 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/125
    • @liusiyi641 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/90
    • @ethankim00 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/116
    • @XudongOliverShen made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/159
    • @gkaramanolakis made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/200
    • @kushalchawla made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/193
    • @jayavardhan3112 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/271
    • @arlenfan made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/287
    • @cosmicishan made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/192
    • @PhaniRohithaKaza made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/269
    • @ilobo22 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/305
    • @Sujan242 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/311
    • @Arutselvan made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/237
    • @ghlai9665 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/348
    • @JalanshMunshi made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/328
    • @paik1 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/314
    • @NeilFranks made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/330
    • @andersonjwan made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/322
    • @tanay2001 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/307
    • @divya0627 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/303
    • @chiragvartak made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/326
    • @Ravsehajsinghpuri made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/363
    • @vijaykumawat256 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/320
    • @ananth-duggirala made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/302
    • @BhavyaSri1001 made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/270
    • @davidstap made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/364
    • @NSVPC made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/252
    • @RaghulRajM made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/318
    • @mishra-sid made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/279
    • @ashok-arjun made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/374
    • @sumantapatro made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/397
    • @selvagmj made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/338
    • @atharva-naik made their first contribution in https://github.com/allenai/natural-instructions-expansion/pull/433

    Full Changelog: https://github.com/allenai/natural-instructions-expansion/commits/v2.0

    Source code(tar.gz)
    Source code(zip)
ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

ALFRED A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han,

ALFRED 204 Dec 15, 2022
Flybirds - BDD-driven natural language automated testing framework, present by Trip Flight

Flybird | English Version 行为驱动开发(Behavior-driven development,缩写BDD),是一种软件过程的思想或者

Ctrip, Inc. 706 Dec 30, 2022
Repository for publicly available deep learning models developed in Rosetta community

trRosetta2 This package contains deep learning models and related scripts used by Baker group in CASP14. Installation Linux/Mac clone the package git

null 81 Dec 29, 2022
This repository contains a PyTorch implementation of "AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis".

AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis | Project Page | Paper | PyTorch implementation for the paper "AD-NeRF: Audio

null 551 Dec 29, 2022
Keras community contributions

keras-contrib : Keras community contributions Keras-contrib is deprecated. Use TensorFlow Addons. The future of Keras-contrib: We're migrating to tens

Keras 1.6k Dec 21, 2022
A community run, 5-day PyTorch Deep Learning Bootcamp

Deep Learning Winter School, November 2107. Tel Aviv Deep Learning Bootcamp : http://deep-ml.com. About Tel-Aviv Deep Learning Bootcamp is an intensiv

Shlomo Kashani. 1.3k Sep 4, 2021
RoBERTa Marathi Language model trained from scratch during huggingface 🤗 x flax community week

RoBERTa base model for Marathi Language (मराठी भाषा) Pretrained model on Marathi language using a masked language modeling (MLM) objective. RoBERTa wa

Nipun Sadvilkar 23 Oct 19, 2022
Graph-based community clustering approach to extract protein domains from a predicted aligned error matrix

Using a predicted aligned error matrix corresponding to an AlphaFold2 model , returns a series of lists of residue indices, where each list corresponds to a set of residues clustering together into a pseudo-rigid domain.

Tristan Croll 24 Nov 23, 2022
Python Implementation of algorithms in Graph Mining, e.g., Recommendation, Collaborative Filtering, Community Detection, Spectral Clustering, Modularity Maximization, co-authorship networks.

Graph Mining Author: Jiayi Chen Time: April 2021 Implemented Algorithms: Network: Scrabing Data, Network Construbtion and Network Measurement (e.g., P

Jiayi Chen 3 Mar 3, 2022
Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis. You write a high level configuration file specifying your in

Blue Collar Bioinformatics 917 Jan 3, 2023
ManimML is a project focused on providing animations and visualizations of common machine learning concepts with the Manim Community Library.

ManimML ManimML is a project focused on providing animations and visualizations of common machine learning concepts with the Manim Community Library.

null 259 Jan 4, 2023
A data-driven approach to quantify the value of classifiers in a machine learning ensemble.

Documentation | External Resources | Research Paper Shapley is a Python library for evaluating binary classifiers in a machine learning ensemble. The

Benedek Rozemberczki 188 Dec 29, 2022
Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

Peidong Liu(刘沛东) 54 Dec 17, 2022
Code for the paper Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration

IMAGINE: Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration This repo contains the code base of the paper Language as a Cog

Flowers Team 26 Dec 22, 2022
Code of paper "CDFI: Compression-Driven Network Design for Frame Interpolation", CVPR 2021

CDFI (Compression-Driven-Frame-Interpolation) [Paper] (Coming soon...) | [arXiv] Tianyu Ding*, Luming Liang*, Zhihui Zhu, Ilya Zharkov IEEE Conference

Tianyu Ding 95 Dec 4, 2022
Scalable, event-driven, deep-learning-friendly backtesting library

...Minimizing the mean square error on future experience. - Richard S. Sutton BTGym Scalable event-driven RL-friendly backtesting library. Build on

Andrew 922 Dec 27, 2022
Official PyTorch implementation of "IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos", CVPRW 2021

IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos Introduction This repo is official PyTorch implementatio

Gyeongsik Moon 29 Sep 24, 2022
Attention-driven Robot Manipulation (ARM) which includes Q-attention

Attention-driven Robotic Manipulation (ARM) This codebase is home to: Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation I

Stephen James 84 Dec 29, 2022
A data-driven maritime port simulator

PySeidon - A Data-Driven Maritime Port Simulator ?? Extendable and modular software for maritime port simulation. This software uses entity-component

null 6 Apr 10, 2022