Faced this issue when using pre-commit :

When I went to check the "pre-commit-config.yaml" file, it says that the repo for this hook is local.

Since black, flake8 and isort hooks passed, I commited the code with following command :

git commit -m "My french_conjugation_transformation" -n

It seems Spacy's tokenizer behaves differently when I run pytest -s --t=emojify and pytest -s --t=light --f=light.

For example, I added the following snippet in my generate() function:

print([str(t) for t in self.nlp(sentence)])

With input sentence "Apple is looking at buying U.K. startup for $132 billion."

pytest -s --t=emojify gives:

['Apple', 'is', 'looking', 'at', 'buying', 'U.K.', 'startup', 'for', '$', '132', 'billion', '.']

However, pytest -s --t=light --f=light gives:

['Apple', 'is', 'looking', 'at', 'buying', 'U.K.', 'startup', 'for', '$1', '32', 'billion.']

I use the fowling code to load spacy:

import spacy
from initialize import spacy_nlp
self.nlp = spacy_nlp if spacy_nlp else spacy.load("en_core_web_sm")

It looks very strange. Am I overlooking something?

For some strange reason, the build was failing for entity_mention_replacement_ner transformation, therefore, tests were temporarily removed, this PR just includes the tests again for entity_mention_replacement_ner.

This transformation adapts backtranslation to the task of NER by generating paraphrases of the contexts around the entity mention(s) using backtranslation. It can be used as a data augmentation strategy to improve the underlying sequence (NER) model as reported by Yaseen and Langer, 2021.

Contributors: Usama Yaseen ([email protected]), Stefan Langer ([email protected])

Affiliation: Siemens AG

This PR introduces the "OCR perturbation" transformation, which directly induces Optical Character Recognition (OCR) errors into the input text. It renders the input sentence as an image and recognizes the rendered text using an OCR engine.

This transformation replaces consistently financial amounts throughout a text. The replacement changes the amount, the writting format as well as the currency of the financial amount. The change is consistent in regard to:

• the modifier used to change all amounts of the same currency throughout the text.
  • e.g., the sentence I owe Fred € 20 and I need € 10 for the bus. might be changed to I owe Fred 2 906.37 Yen and I need 1 453.19 Yen for the bus.
• the modifier used to change the amounts so that new amounts are relatively close to the original amount.
• the rate used for a change of currency, reflecting the actual bank rate.

This filter allows filtering data based on the counts of named entities for more fine-grained analysis of text generation systems wrt named entities in the input. The PR includes test cases + Readme for more details. Thanks.

These augments convert causal sentences' direction and/or strength based on grammar rules and dependency trees. We introduce augments that amend both sentence and target (SentenceAndTargetOperation).

Hello,

This paraphraser is an enabler to use GPT-2 paraphrasers, originally trained by Krishna et al. for the paper Reformulating Unsupervised Style Transfer as Paraphrase Generation. Currently, I have integrated 6 different GPT-2 paraphrasers with different styles that I have tested and worked, but I might integrate more by uploading them to Huggingface (WIP).

The currently supported styles are:

• Shakespeare
• Switchboard (Conversation Speech)
• Tweets
• Bible
• Romantic poetry
• Basic

The paraphraser is quite similar to formality_change, but more diverse and general, and works for several different styles. There is a model for "Conversational Speech"-style which is similar to formality, which is the "Switchboard" version.

Model sizes

Each model is about 3.25 GB, so the testing will require a GPU if it has to be quick (as with any large language model).

A comment about the tests

Note that the tests currently only cover the Shakespeare model. I was unsure how to add different tests for different models. Perhaps subclasses for the different models is a better option than specifying the specific style, at least given the testing environment. Can I add instantiation arguments for the different tests somehow?

License

I was unsure how to handle the LICENSE from the original repository. Much of the code is based upon the original code (which I am not an author of) made by @martiansideofthemoon and his co-authors, but has been greatly modified. I therefore included the license (which is MIT), but please correct me if this is not necessary.

No new requirements

No new requirements were needed to be added; the current model and the code I adapted from the original repository was compatible with the requirements already existing in the repository.

(Edit, I accidentally clicked and submitted the PR before finishing what I was writing here).

This transformation augments the input sentence by randomly replacing colors.

For example,

I bought this pink shoes today! Isn't it pretty?

becomes

I bought this misty rose shoes today! Isn't it pretty?

This PR is for the changes discussed in #145. I thought removing it from the interface made the most sense, so we don't pass any redundant information and only set the seed for both random and numpy once in initialize.py. I've also used this to fix some remaining spacy.load issues in some of the filters/transformations so they are loaded only once in initialize.py.

Would be nice if you could test all things to make sure I didn't break anything.

Draft for now, since I think some tests need to be adjusted now since they used a different seed.

It does not appear possible to load the NumberToWord transformation after installing nlaugmenter.

https://github.com/GEM-benchmark/NL-Augmenter/blob/main/nlaugmenter/transformations/number-to-word/transformation.py

This is likely due to number-to-word breaking python's path loading.

from nlaugmenter.transformations.formality_change.transformation import Formal2Casual

OSError: prithivida/parrot_adequacy_on_BART is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

The model (prithivida/parrot_adequacy_on_BART) is indeed not available on huggingface anymore. Perhaps an acceptable alternative is to use prithivida/parrot_adequacy_model instead?

Thank you for your great work! It's super useful!

I have a suggestion for improvement - Some transformations are working with a "swap" principle. For example, in GenderSwap, if we had "sister" in the original sentence then it would be transformed to 'brother" and vice versa. There are scenarios when it's important to know what direction the transformation went, female to male or male to female. In my case for example, I want to compare the performances of my model on female/male sentences on inference time.

I really liked the way TenseTransformation works. You need to specify in the constructor what tense (past/present/future) you want to transform to. Maybe that could be applicable for other swap transformations?

Thanks again!

This transformation transfers the numbers written in Chinese to digits numbers, such as “一” (one) to 1, "二" (two) to 2, and "百" (a hundred) to 100. This transformation is a vital component of Chinese NLP systems. It would benefit all tasks which have a sentence/paragraph/document with numbers written in Capital Chinese as input like text classification, text generation, etc.

I'm creating a new PR as per the request in https://github.com/GEM-benchmark/NL-Augmenter/pull/274.

I also squashed all prior commits into one:

Renaming ambiguousfilter folder

Adding alphanumeric characters filter

Removing unused imports and dead code in alphanumeric filter, renaming class

Adding ambiguous characters filter

Fixing import error by renaming alphanumeric_filter.py -> filter.py

Adding keywords

Addressing reviewers comments

Adding necessary import

Making recommended changes