A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Overview



A Deep Learning NLP/NLU library by Intel® AI Lab

GitHub Website GitHub release

Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing

NLP Architect is an open source Python library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing and Natural Language Understanding Neural Networks.

Overview

NLP Architect is an NLP library designed to be flexible, easy to extend, allow for easy and rapid integration of NLP models in applications and to showcase optimized models.

Features:

  • Core NLP models used in many NLP tasks and useful in many NLP applications

  • Novel NLU models showcasing novel topologies and techniques

  • Optimized NLP/NLU models showcasing different optimization algorithms on neural NLP/NLU models

  • Model-oriented design:

    • Train and run models from command-line.
    • API for using models for inference in python.
    • Procedures to define custom processes for training, inference or anything related to processing.
    • CLI sub-system for running procedures
  • Based on optimized Deep Learning frameworks:

  • Essential utilities for working with NLP models - Text/String pre-processing, IO, data-manipulation, metrics, embeddings.

Installing NLP Architect

We recommend to install NLP Architect in a new python environment, to use python 3.6+ with up-to-date pip, setuptools and h5py.

Install using pip

Install core library only

pip install nlp-architect

Install from source (Github)

Includes core library, examples, solutions and tutorials:

git clone https://github.com/IntelLabs/nlp-architect.git
cd nlp-architect
pip install -e .  # install in developer mode

Running Examples and Solutions

To run provided examples and solutions please install the library with [all] flag which will install extra packages required. (requires installation from source)

pip install .[all]

Models

NLP models that provide best (or near) in class performance:

Natural Language Understanding (NLU) models that address semantic understanding:

Optimizing NLP/NLU models and misc. optimization techniques:

Solutions (End-to-end applications) using one or more models:

Documentation

Full library documentation of NLP models, algorithms, solutions and instructions on how to run each model can be found on our website.

NLP Architect library design philosophy

NLP Architect is a model-oriented library designed to showcase novel and different neural network optimizations. The library contains NLP/NLU related models per task, different neural network topologies (which are used in models), procedures for simplifying workflows in the library, pre-defined data processors and dataset loaders and misc utilities. The library is designed to be a tool for model development: data pre-process, build model, train, validate, infer, save or load a model.

The main design guidelines are:

  • Deep Learning framework agnostic
  • NLP/NLU models per task
  • Different topologies used in models
  • Showcase End-to-End applications (Solutions) utilizing one or more NLP Architect model
  • Generic dataset loaders, textual data processing utilities, and miscellaneous utilities that support NLP model development (loaders, text processors, io, metrics, etc.)
  • Procedures for defining processes for training, inference, optimization or any kind of elaborate script.
  • Pythonic API for using models for inference
  • Extensive model documentation and tutorials

Note

NLP Architect is an active space of research and development; Throughout future releases new models, solutions, topologies and framework additions and changes will be made. We aim to make sure all models run with Python 3.6+. We encourage researchers and developers to contribute their work into the library.

Citing

If you use NLP Architect in your research, please use the following citation:

@misc{izsak_peter_2018_1477518,
  title        = {NLP Architect by Intel AI Lab},
  month        = nov,
  year         = 2018,
  doi          = {10.5281/zenodo.1477518},
  url          = {https://doi.org/10.5281/zenodo.1477518}
}

Disclaimer

The NLP Architect is released as reference code for research purposes. It is not an official Intel product, and the level of quality and support may not be as expected from an official product. NLP Architect is intended to be used locally and has not been designed, developed or evaluated for production usage or web-deployment. Additional algorithms and environments are planned to be added to the framework. Feedback and contributions from the open source and NLP research communities are more than welcome.

Contact

Contact the NLP Architect development team through Github issues or email: [email protected]

Issues
  • Runtime errors when running with the ABSA model

    Runtime errors when running with the ABSA model

    I am trying to run the ABSA solution on my local Ubuntu 18.04 even with the provided train reviews dataset but getting the following error (using the exact instructions + virtualenv): 'file_name': ['tripadvisor_co_uk-travel_restaurant_reviews_sample_2000_train.csv']}}], 'references': []}: OSError('Unable to load model. Filepath is not an hdf5 file (or h5py is not available) or SavedModel.',) When I am trying to load a bunch of reviews, I get the following error:

    'references': []}: ParserError('Expected 2 fields in line 2, saw 3')

    Please advise how to fix it.

    bug 
    opened by stevesolun 10
  • Pip install failing from master

    Pip install failing from master

    Target objective:

    pip Install .

    Steps to objective:

    Successfully built nlp-architect spacy 2.0.18 has requirement numpy>=1.15.0, but you'll have numpy 1.14.5 which is incompatible. Installing collected packages: spacy, nlp-architect Killed

    Pull-Request related:

    opened by aribornstein 8
  • Any readme for BERT example?

    Any readme for BERT example?

    Target objective:

    Want help to run https://github.com/NervanaSystems/nlp-architect/blob/master/nlp_architect/procedures/transformers/glue.py

    Hi, I want to run the bert example newly added to the repo. Wondering is there any readme for running the example or any documentation for bert/quantized-bert related features? For instance, where to download model and 'labels.txt'?

    Thanks so much for your help in advance!

    opened by hkvision 7
  • bug: Error exporting QuantizedBert to ONNX

    bug: Error exporting QuantizedBert to ONNX

    When attempting to export a Q8BERT model to ONNX, TorchScript reports a runtime error.

    Since the quantized forward methods embed Tensors with grads from layer instance attributes rather than input parameters, exporting the model has to be done with explicit torch script tracing; however that creates a traced model with lacks fidelity with the python model, as branching behavior is not captured.

    However, attempting to create a script model with TorchScript, which should encode that behavior results in a missing config attribute on the BertModel superclass.

    To Reproduce Steps to reproduce the behavior:

    1. Train a Q8Bert model:
     nlp-train transformer_glue \
                --task_name mrpc \
                --model_name_or_path bert-base-uncased \
                --model_type quant_bert \
                --learning_rate 2e-5 \
                --output_dir $DATA_DIR \
                --data_dir $GLUE_DIR  \
                --evaluate_during_training \
                --do_lower_case \
                --per_gpu_train_batch_size 32 \
                --per_gpu_eval_batch_size 32 \
                --max_seq_length 128
    
    1. Load the 8bit model for inference
    import torch
    from nlp_architect.models.transformers.quantized_bert import QuantizedBertForSequenceClassification
    
    model = QuantizedBertForSequenceClassification.from_pretrained(configs.data_dir, from_8bit=True)
    device = torch.device("cuda")
    model.to(device)
    model.eval()
    
    
    1. Attempt to create the script model
    script_model = torch.jit.script(model)
    
    1. This produces the error:
    Traceback (most recent call last):
      File "convert-torch-q8bert-via-onnx-to-tensorflow.py", line 105, in <module>
        script_model = torch.jit.script(model)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1203, in script
        return torch.jit.torch.jit._recursive.recursive_script(obj)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
        return copy_to_script_module(mod, overload_stubs + stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
        torch.jit._create_methods_from_stubs(script_module, stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
        self._c._create_methods(self, defs, rcbs, defaults)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 195, in make_strong_submodule
        new_strong_submodule = recursive_script(module)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
        return copy_to_script_module(mod, overload_stubs + stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
        torch.jit._create_methods_from_stubs(script_module, stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
        self._c._create_methods(self, defs, rcbs, defaults)
    RuntimeError:
    module has no attribute 'config':
    at /home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/transformers/modeling_bert.py:675:15
            # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
            # ourselves in which case we just need to make it broadcastable to all heads.
            if attention_mask.dim() == 3:
                extended_attention_mask = attention_mask[:, None, :, :]
    
            # Provided a padding mask of dimensions [batch_size, seq_length]
            # - if the model is a decoder, apply a causal mask in addition to the padding mask
            # - if the model is an encoder, make the mask broadcastable to [batch_size, num_heads, seq_length, seq_length]
            if attention_mask.dim() == 2:
                if self.config.is_decoder:
                   ~~~~~~~~~~~ <--- HERE
                    batch_size, seq_length = input_shape
                    seq_ids = torch.arange(seq_length, device=device)
                    causal_mask = seq_ids[None, None, :].repeat(batch_size, seq_length, 1) <= seq_ids[None, :, None]
                    extended_attention_mask = causal_mask[:, None, :, :] * attention_mask[:, None, None, :]
                else:
                    extended_attention_mask = attention_mask[:, None, None, :]
    
            # Since attention_mask is 1.0 for positions we want to attend and 0.0 for
            # masked positions, this operation will create a tensor which is 0.0 for
    '__torch__.nlp_architect.models.transformers.quantized_bert.QuantizedBertModel.forward' is being compiled since it was called from '__torch__.nlp_architect.models.transformers.quantized_bert.QuantizedBertForSequenceClassification.forward'
    at /home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/transformers/modeling_bert.py:1014:8
        def forward(self, input_ids=None, attention_mask=None, token_type_ids=None,
                    position_ids=None, head_mask=None, inputs_embeds=None, labels=None):
    
            outputs = self.bert(input_ids,
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~...  <--- HERE
                                attention_mask=attention_mask,
                                token_type_ids=token_type_ids,
                                position_ids=position_ids,
                                head_mask=head_mask,
                                inputs_embeds=inputs_embeds)
    
            pooled_output = outputs[1]
    
            pooled_output = self.dropout(pooled_output)
    

    Expected behavior

    It should produce a script model without error

    **Environment setup: **

    • OS (Linux/Mac OS): Ubuntu Xenial
    • Python version: 3.7

    Additional context

    It may be possible to create pure helper quantized forward functions with explicit arguments which have the @torch.jit.script annotations which are called by the quantized layer forward methods

    Note that the HuggingFace transformers Bert model can be exported to ONNX.

    bug Stale 
    opened by kirstin-rhys 7
  • GPU training support

    GPU training support

    This is more of a question than issue but I wasn't able to find a forum to ask the question.

    I see installation instruction on enabling GPU supports. Yet, many algorithms (e.g. ner) do not specifically have a parameter for GPU enabling. Is it an issue easy to get around with or is it going to be a feature supported in the future?

    question 
    opened by DihuiLai 7
  • Problem running bist FileNotFoundError: [Errno 2] No such file or directory:  (...) params.json

    Problem running bist FileNotFoundError: [Errno 2] No such file or directory: (...) params.json

    I tried to run bist in your interface using the command :

    python3 server/serve.py --name bist

    I get the following error :

    25MB [00:28,  1.16s/MB]                                                                                                                                                                                     
    Download Complete
    Unzipping...
    Done.
    Traceback (most recent call last):
      File "serve.py", line 306, in <module>
        set_server_properties(app, args.name)
      File "serve.py", line 287, in set_server_properties
        service = Service(service_name)
      File "serve.py", line 143, in __init__
        self.service = self.load_service(service_name)
      File "serve.py", line 259, in load_service
        upload_service.load_model()
      File "/home/catalina/nlp-architect/nlp_architect/api/bist_parser_api.py", line 32, in load_model
        self.model = SpacyBISTParser()
      File "/home/catalina/nlp-architect/nlp_architect/pipelines/spacy_bist.py", line 48, in __init__
        self.bist_parser.load(bist_model if bist_model else SpacyBISTParser.pretrained)
      File "/home/catalina/nlp-architect/nlp_architect/models/bist_parser.py", line 117, in load
        with open(os.path.join(os.path.dirname(path), 'params.json'), 'r') as file:
    FileNotFoundError: [Errno 2] No such file or directory: '/home/catalina/nlp-architect/nlp_architect/pipelines/bist-pretrained/params.json'
    
    opened by Catadanna 7
  • nlp_architect command fails if dev req aren't installed

    nlp_architect command fails if dev req aren't installed

    Target objective:

    nlp_architect command fails if dev req aren't installed detect is libraries required are install and print a msg that states the error + how to install

    Steps to objective:

    • [ ] fix cmd.py
    • [ ]

    Pull-Request related:

    bug 
    opened by peteriz 7
  • Question about Seq2SeqIntentModel

    Question about Seq2SeqIntentModel

    This is used for slot tagging, and in the example you provide, the dataset is SNIPS and it is used for NER. Wondering why it is placed under intent_extraction and come with the name IntentModel?

    question Stale 
    opened by hkvision 6
  • How to improve bist model accuracy? :Bistmodel

    How to improve bist model accuracy? :Bistmodel

    I'm using spacybistparser with my own spacy and bist model trained using universal dependencies v2.0 dataset. My training follows the doc. I want to improve the accuracy of bistmodel. Currently, with default parameters, I train the bist model for 20iterations. The evaluation uas score is 87.8 percent. How can I improve it to 93.8 as mentioned in the docs? Is it possible with this english ud 2.0 dataset? Please help.

    question 
    opened by ravishpankar 6
  • question about Quantize BERT?

    question about Quantize BERT?

    opened by RyanHuangNLP 6
  • Bump pillow from 8.0.0 to 8.3.2 in /solutions/absa_solution

    Bump pillow from 8.0.0 to 8.3.2 in /solutions/absa_solution

    Bumps pillow from 8.0.0 to 8.3.2.

    Release notes

    Sourced from pillow's releases.

    8.3.2

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

    Security

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    Python 3.10 wheels

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    Fixed regressions

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

    Changes

    8.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.3.2 (2021-09-02)

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1 (2021-07-06)

    • Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

    • Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

    • Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

    8.3.0 (2021-07-01)

    • Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

    • Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

    • Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

    • De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

    • Replaced xml.etree.ElementTree #5565 [radarhere]

    ... (truncated)

    Commits
    • 8013f13 8.3.2 version bump
    • 23c7ca8 Update CHANGES.rst
    • 8450366 Update release notes
    • a0afe89 Update test case
    • 9e08eb8 Raise ValueError if color specifier is too long
    • bd5cf7d FLI tests for Oss-fuzz crash.
    • 94a0cf1 Fix 6-byte OOB read in FliDecode
    • cece64f Add 8.3.2 (2021-09-02) [CI skip]
    • e422386 Add release notes for Pillow 8.3.2
    • 08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump tensorflow from 1.15.2 to 2.5.1 in /solutions/absa_solution

    Bump tensorflow from 1.15.2 to 2.5.1 in /solutions/absa_solution

    Bumps tensorflow from 1.15.2 to 2.5.1.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.5.1

    Release 2.5.1

    This release introduces several vulnerability fixes:

    • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
    • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
    • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
    • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
    • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
    • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
    • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
    • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
    • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
    • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
    • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
    • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
    • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
    • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
    • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
    • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
    • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
    • Fixes a use after free in boosted trees creation (CVE-2021-37652)
    • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
    • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
    • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
    • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
    • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
    • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
    • Fixes a division by 0 in inplace operations (CVE-2021-37660)
    • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
    • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
    • Fixes a heap OOB in boosted trees (CVE-2021-37664)
    • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
    • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
    • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
    • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
    • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
    • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
    • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
    • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
    • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
    • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
    • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
    • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
    • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
    • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
    • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.5.1

    This release introduces several vulnerability fixes:

    • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
    • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
    • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
    • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
    • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
    • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
    • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
    • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
    • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
    • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
    • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
    • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
    • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
    • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
    • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
    • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
    • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
    • Fixes a use after free in boosted trees creation (CVE-2021-37652)
    • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
    • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
    • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse

    ... (truncated)

    Commits
    • 8222c1c Merge pull request #51381 from tensorflow/mm-fix-r2.5-build
    • d584260 Disable broken/flaky test
    • f6c6ce3 Merge pull request #51367 from tensorflow-jenkins/version-numbers-2.5.1-17468
    • 3ca7812 Update version numbers to 2.5.1
    • 4fdf683 Merge pull request #51361 from tensorflow/mm-update-relnotes-on-r2.5
    • 05fc01a Put CVE numbers for fixes in parentheses
    • bee1dc4 Update release notes for the new patch release
    • 47beb4c Merge pull request #50597 from kruglov-dmitry/v2.5.0-sync-abseil-cmake-bazel
    • 6f39597 Merge pull request #49383 from ashahab/abin-load-segfault-r2.5
    • 0539b34 Merge pull request #48979 from liufengdb/r2.5-cherrypick
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump tensorflow from 1.15.4 to 2.5.1

    Bump tensorflow from 1.15.4 to 2.5.1

    Bumps tensorflow from 1.15.4 to 2.5.1.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.5.1

    Release 2.5.1

    This release introduces several vulnerability fixes:

    • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
    • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
    • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
    • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
    • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
    • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
    • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
    • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
    • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
    • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
    • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
    • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
    • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
    • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
    • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
    • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
    • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
    • Fixes a use after free in boosted trees creation (CVE-2021-37652)
    • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
    • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
    • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)
    • Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)
    • Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)
    • Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)
    • Fixes a division by 0 in inplace operations (CVE-2021-37660)
    • Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)
    • Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)
    • Fixes a heap OOB in boosted trees (CVE-2021-37664)
    • Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)
    • Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)
    • Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)
    • Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)
    • Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)
    • Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)
    • Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)
    • Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)
    • Fixes a CHECK-fail in MapStage (CVE-2021-37673)
    • Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)
    • Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)
    • Fixes a division by 0 in most convolution operators (CVE-2021-37675)
    • Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)
    • Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)
    • Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.5.1

    This release introduces several vulnerability fixes:

    • Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)
    • Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)
    • Fixes a null pointer dereference in CompressElement (CVE-2021-37637)
    • Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)
    • Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)
    • Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)
    • Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)
    • Fixes a heap OOB in RaggedGather (CVE-2021-37641)
    • Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)
    • Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)
    • Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)
    • Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)
    • Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)
    • Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)
    • Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)
    • Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)
    • Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)
    • Fixes a use after free in boosted trees creation (CVE-2021-37652)
    • Fixes a division by 0 in ResourceGather (CVE-2021-37653)
    • Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)
    • Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)
    • Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse

    ... (truncated)

    Commits
    • 8222c1c Merge pull request #51381 from tensorflow/mm-fix-r2.5-build
    • d584260 Disable broken/flaky test
    • f6c6ce3 Merge pull request #51367 from tensorflow-jenkins/version-numbers-2.5.1-17468
    • 3ca7812 Update version numbers to 2.5.1
    • 4fdf683 Merge pull request #51361 from tensorflow/mm-update-relnotes-on-r2.5
    • 05fc01a Put CVE numbers for fixes in parentheses
    • bee1dc4 Update release notes for the new patch release
    • 47beb4c Merge pull request #50597 from kruglov-dmitry/v2.5.0-sync-abseil-cmake-bazel
    • 6f39597 Merge pull request #49383 from ashahab/abin-load-segfault-r2.5
    • 0539b34 Merge pull request #48979 from liufengdb/r2.5-cherrypick
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Performance issue in the definition of _inference, examples/memn2n_dialogue/memn2n_dialogue.py(P1)

    Performance issue in the definition of _inference, examples/memn2n_dialogue/memn2n_dialogue.py(P1)

    Hello, I found a performance issue in the difinition of_inference, examples/memn2n_dialogue/memn2n_dialogue.py, tf.nn.embedding_lookup will be calculated repeately during the program execution, resulting in reduced efficiency. So I think it should be created before the loop.

    The same issue exist in tf.reduce_sum in line 187 and 200.

    Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

    bug 
    opened by DLPerf 2
  • Why the number of data from restaurant and laptop is different from the paper?

    Why the number of data from restaurant and laptop is different from the paper?

    the paper: restaurant 5841 laptop 3845 but get restaurant 5844 laptop 3848

    question 
    opened by LemonDrinkTea 0
  • bug: ERROR: ResolutionImpossible

    bug: ERROR: ResolutionImpossible

    Describe the bug Installation error Model/procedure: $ pip install nlp-architect

    To Reproduce Steps to reproduce the behavior: $ pip install nlp-architect

    Expected behavior No Errors

    **Environment setup: **

    • OS (Linux/Mac OS): GNU/Linux
    • Python version: Python 3.9.6
    • Backend: pip

    Log:

    Collecting nlp-architect
      Using cached nlp_architect-0.5.4-py3-none-any.whl (308 kB)
    Collecting h5py
      Using cached h5py-3.3.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
    Collecting requests
      Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
         |████████████████████████████████| 62 kB 56 kB/s 
    Collecting gensim
      Using cached gensim-4.0.1-cp39-cp39-linux_x86_64.whl
    Collecting scipy
      Downloading scipy-1.7.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.4 MB)
         |████████████████████████████████| 28.4 MB 871 kB/s 
    Collecting six
      Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
    Collecting numpy==1.16.4
      Using cached numpy-1.16.4.zip (5.1 MB)
    Collecting dynet==2.1
      Using cached dyNET-2.1.tar.gz (454 kB)
        ERROR: Command errored out with exit status 1:
         command: /home/anks/STEALTH/ENV/absa_env/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-832t4ncx/dynet_ec03a5a10dc14ba6bb53d279dc810a9e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-832t4ncx/dynet_ec03a5a10dc14ba6bb53d279dc810a9e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-pj_foilz
             cwd: /tmp/pip-install-832t4ncx/dynet_ec03a5a10dc14ba6bb53d279dc810a9e/
        Complete output (5 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-install-832t4ncx/dynet_ec03a5a10dc14ba6bb53d279dc810a9e/setup.py", line 17, in <module>
            from Cython.Distutils import build_ext as _build_ext
        ModuleNotFoundError: No module named 'Cython'
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/22/8e/148ecf0ec4ffce667e84abd6aa134bbff1f73cbb99c92c198149f122889e/dyNET-2.1.tar.gz#sha256=d0f58aaf3926da24baba6e3e76cb3d090c8b6d359196ce138b11faa291b2ec07 (from https://pypi.org/simple/dynet/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    Collecting nlp-architect
      Using cached nlp_architect-0.5.3-py3-none-any.whl (301 kB)
      Using cached nlp_architect-0.5.2-py3-none-any.whl (978 kB)
    Collecting tqdm
      Using cached tqdm-4.61.2-py2.py3-none-any.whl (76 kB)
    Collecting wordfreq
      Downloading wordfreq-2.5.0.tar.gz (56.8 MB)
         |████████████████████████████████| 56.8 MB 50 kB/s 
    Collecting future
      Downloading future-0.18.2.tar.gz (829 kB)
         |████████████████████████████████| 829 kB 71 kB/s 
    Collecting python-dateutil<2.8.1
      Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)
         |████████████████████████████████| 226 kB 945 kB/s 
    Collecting ftfy
      Downloading ftfy-6.0.3.tar.gz (64 kB)
         |████████████████████████████████| 64 kB 226 kB/s 
    Collecting termcolor
      Downloading termcolor-1.1.0.tar.gz (3.9 kB)
    Collecting numpy<1.17
      Using cached numpy-1.16.6.zip (5.1 MB)
    Collecting hyperopt
      Downloading hyperopt-0.2.5-py2.py3-none-any.whl (965 kB)
         |████████████████████████████████| 965 kB 112 kB/s 
    Collecting pywikibot
      Downloading pywikibot-6.4.0.tar.gz (506 kB)
         |████████████████████████████████| 506 kB 422 kB/s 
    Collecting nltk
      Downloading nltk-3.6.2-py3-none-any.whl (1.5 MB)
         |████████████████████████████████| 1.5 MB 696 kB/s 
    Collecting tensorflow-hub
      Downloading tensorflow_hub-0.12.0-py2.py3-none-any.whl (108 kB)
         |████████████████████████████████| 108 kB 330 kB/s 
    Collecting nlp-architect
      Using cached nlp_architect-0.5.1-py3-none-any.whl (1.6 MB)
    Collecting pillow
      Downloading Pillow-8.3.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.0 MB)
         |████████████████████████████████| 3.0 MB 182 kB/s 
    Collecting regex==2018.02.21
      Downloading regex-2018.02.21.tar.gz (620 kB)
         |████████████████████████████████| 620 kB 475 kB/s 
    Collecting nlp-architect
      Downloading nlp_architect-0.5-py3-none-any.whl (1.2 MB)
         |████████████████████████████████| 1.2 MB 107 kB/s 
      Downloading nlp_architect-0.4.post3-py3-none-any.whl (1.1 MB)
         |████████████████████████████████| 1.1 MB 102 kB/s 
    Collecting spacy
      Downloading spacy-3.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.5 MB)
         |████████████████████████████████| 6.5 MB 857 kB/s 
    Collecting nlp-architect
      Downloading nlp_architect-0.4.post2-py3-none-any.whl (1.1 MB)
         |████████████████████████████████| 1.1 MB 70 kB/s 
    Collecting dynet==2.0.2
      Downloading dyNET-2.0.2.tar.gz (388 kB)
         |████████████████████████████████| 388 kB 557 kB/s 
        ERROR: Command errored out with exit status 1:
         command: /home/anks/STEALTH/ENV/absa_env/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-832t4ncx/dynet_eb19eec4e56248fdb3fef59af0805ab5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-832t4ncx/dynet_eb19eec4e56248fdb3fef59af0805ab5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-etz16c1n
             cwd: /tmp/pip-install-832t4ncx/dynet_eb19eec4e56248fdb3fef59af0805ab5/
        Complete output (5 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-install-832t4ncx/dynet_eb19eec4e56248fdb3fef59af0805ab5/setup.py", line 17, in <module>
            from Cython.Distutils import build_ext as _build_ext
        ModuleNotFoundError: No module named 'Cython'
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/01/10/cfc00197733dd7cf52d9e00c42017ebe7e818653980bccfe6f241ea5b79a/dyNET-2.0.2.tar.gz#sha256=d09f3799e3ad19572e9adec2784b0330a5d96bf6914a20858cd14b71e1482213 (from https://pypi.org/simple/dynet/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    Collecting nlp-architect
      Downloading nlp_architect-0.4.post1-py3-none-any.whl (1.1 MB)
         |████████████████████████████████| 1.1 MB 611 kB/s 
      Downloading nlp_architect-0.4-py3-none-any.whl (1.1 MB)
         |████████████████████████████████| 1.1 MB 190 kB/s 
      Downloading nlp_architect-0.3.1-py3-none-any.whl (1.0 MB)
         |████████████████████████████████| 1.0 MB 598 kB/s 
    Collecting pytest-mock
      Downloading pytest_mock-3.6.1-py3-none-any.whl (12 kB)
    ERROR: Cannot install nlp-architect==0.3.1, nlp-architect==0.4, nlp-architect==0.4.post1, nlp-architect==0.4.post2, nlp-architect==0.4.post3, nlp-architect==0.5, nlp-architect==0.5.1, nlp-architect==0.5.2, nlp-architect==0.5.3 and nlp-architect==0.5.4 because these package versions have conflicting dependencies.
    
    The conflict is caused by:
        nlp-architect 0.5.4 depends on dynet==2.1
        nlp-architect 0.5.3 depends on tensorflow==1.15.0
        nlp-architect 0.5.2 depends on dynet==2.1
        nlp-architect 0.5.1 depends on dynet==2.1
        nlp-architect 0.5 depends on dynet==2.1
        nlp-architect 0.4.post3 depends on dynet==2.1
        nlp-architect 0.4.post2 depends on dynet==2.0.2
        nlp-architect 0.4.post1 depends on dynet==2.0.2
        nlp-architect 0.4 depends on dynet==2.0.2
        nlp-architect 0.3.1 depends on dynet==2.0.2
    
    To fix this you could try to:
    1. loosen the range of package versions you've specified
    2. remove package versions to allow pip attempt to solve the dependency conflict
    
    ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
    
    
    bug 
    opened by AnkS4 0
  • Bump urllib3 from 1.25.11 to 1.26.5 in /solutions/absa_solution

    Bump urllib3 from 1.25.11 to 1.26.5 in /solutions/absa_solution

    Bumps urllib3 from 1.25.11 to 1.26.5.

    Release notes

    Sourced from urllib3's releases.

    1.26.5

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.4

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.3

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed bytes and string comparison issue with headers (Pull #2141)

    • Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme (Pull #2107)

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    1.26.2

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

    1.26.1

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

    1.26.0

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

    • Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning should opt-in explicitly by setting ssl_version=ssl.PROTOCOL_TLSv1_1 (Pull #2002) Starting in urllib3 v2.0: Connections that receive a DeprecationWarning will fail

    • Deprecated Retry options Retry.DEFAULT_METHOD_WHITELIST, Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST and Retry(method_whitelist=...) in favor of Retry.DEFAULT_ALLOWED_METHODS, Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT, and Retry(allowed_methods=...) (Pull #2000) Starting in urllib3 v2.0: Deprecated options will be removed

    ... (truncated)

    Changelog

    Sourced from urllib3's changelog.

    1.26.5 (2021-05-26)

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    1.26.4 (2021-03-15)

    • Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

    1.26.3 (2021-01-26)

    • Fixed bytes and string comparison issue with headers (Pull #2141)

    • Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme. (Pull #2107)

    1.26.2 (2020-11-12)

    • Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

    1.26.1 (2020-11-11)

    • Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

    1.26.0 (2020-11-10)

    • NOTE: urllib3 v2.0 will drop support for Python 2. Read more in the v2.0 Roadmap <https://urllib3.readthedocs.io/en/latest/v2-roadmap.html>_.

    • Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

    • Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning

    ... (truncated)

    Commits
    • d161647 Release 1.26.5
    • 2d4a3fe Improve performance of sub-authority splitting in URL
    • 2698537 Update vendored six to 1.16.0
    • 07bed79 Fix deprecation warnings for Python 3.10 ssl module
    • d725a9b Add Python 3.10 to GitHub Actions
    • 339ad34 Use pytest==6.2.4 on Python 3.10+
    • f271c9c Apply latest Black formatting
    • 1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
    • a891304 Release 1.26.4
    • 8d65ea1 Merge pull request from GHSA-5phf-pp7p-vc2r
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • question: [Quantization] Which files to change to make inference faster for Q8BERT?

    question: [Quantization] Which files to change to make inference faster for Q8BERT?

    I know from previous issues it is mentioned that that Q8BERT was just an experiment to measure the accuracy of quantized BERT model. But, given that the accuracy is good, what changed would need to be made to torch.nn.quantization file to replace the FP32 operations by INT8 operations?

    Replacing the FP32 Linear layers with the torch.nn.quantized.Linear should theoretically work since it will have optimized operations, but it doesn't. Same for other layers.

    If someone could just point out how to improve the inference speed (hints, tips, directions, code, anything), it would be helpful since the model's accuracy is really good and I would like to use it for downstream tasks. I don't mind even creating a PR once those changes are done so that it merges with the main repo.

    Thank you!

    question 
    opened by sarthaklangde 0
  • question: [Q8Bert experiment Setting]

    question: [Q8Bert experiment Setting]

    Hello, I read the Q8Bert paper and have tried to reproduce the experiment results. But, on some GLUE tasks ( e.g cola, mrpc ), the differences between the fp32 results and quantized ones are much larger than the differences reported in the paper. I tried sweeping initial learning rate but still the result was still far from the reported results.

    image

    So, I want to ask you if the experiment on Q8bert was done with default parameters set inside nlp-architect code as below.

    image

    If not, could you tell me the experiment setting.

    question 
    opened by daumkh402 2
  • improvement: distillation for TransformerSequenceClassifier models for GLUE tasks

    improvement: distillation for TransformerSequenceClassifier models for GLUE tasks

    Hi,

    I'm wondering if it would be easy to add support for knowledge distillation for the Transformers on GLUE tasks (i.e. the TransformerSequenceClassifier module).

    I see that the distillation loss has been implemented, and it's an option for the NeuralTagger which uses the TransformerTokenClassifier. Would it be easy to add distillation support for the GLUE models?

    Here's how I was envisioning implementing it, modeling off of the distillation implementation for the tagger models:

    1. TransformerSequenceClassifier’s train calls the base transformer model’s _train(). I would need to add a distiller argument to this function. This function would then handle distillation by loading in the teacher and the relevant arguments, just like the NeuralTagger
    2. in procedures/transformers/glue.py, i would need to add a do_kd_training function that adds distilation args. This function would need to create a teacher model from these args (loading in weights from the passed-in path), create a TeacherStudentDistill instance, and pass in this object as the distiller argument into the new _train() function.

    Does this seem about right? Are there any roadblocks you'd envision / why wasn't distillation implemented for the sequence classifier models to begin with?

    opened by rmovva 0
Releases(v0.5.5.1)
  • v0.5.5.1(Nov 17, 2020)

  • v0.5.4.1(Sep 16, 2020)

  • v0.5.4(Apr 12, 2020)

    • Improved ID-CNN and LSTM sequence tagger models accuracy
    • Added support for BILOU format for tagging tasks
    • Updated Transformer token tagging and neural taggers to save best and support BILOU format
    • Refactor pseudo/distillation procedures to reproduce results according to our paper
    Source code(tar.gz)
    Source code(zip)
  • v0.5(Aug 28, 2019)

    General features

    • Added PyTorch as a backend
    • New nlp_architect CLI for training/running/process models and scripts
    • New nlp_architect.procedures package for creating procedures/scripts for doing scripted work with NLP architect.
    • S3 URL caching of pre-trained models
    • Many additions/refactors/base classes reflecting new API (WIP)

    Transformer models

    • Integrated pytorch-transformers library into NLP Architect's model package.
    • Created a base transformer class for training transformer-based models
    • Added sequence/token classification models

    Optimization related models

    • Added a base transformer model for training quantized (8bit) BERT models
    • Added distillation process from Transformer models into CNN/LSTM based token classification models

    NLP related additions

    • Added GLUE benchmark tasks
    • Added CNN-LSTM and ID-CNN token classification models (pytorch backend)

    Documentation

    • Updated documentation website to reflect 0.5 release
    • Updated website with logo and matching style
    Source code(tar.gz)
    Source code(zip)
  • v0.4(Apr 2, 2019)

  • v0.3.1(Dec 17, 2018)

    Improvements

    • Installation process refactored and supports pip install nlp-architect (much more simple)
    • Moved server and solutions into main package
    • Many Cross-doc coreference model fixes and additions
    • Updates and fixed lots of documentation files

    Demos

    • Improved NLP Architect demos with a new design
    • Renamed nlp_architect demo to nlp_architect server
    Source code(tar.gz)
    Source code(zip)
  • v0.3(Nov 1, 2018)

    New Solution

    • Topics and Trend Analysis - extract topics and compare two temporal versions a corpus, highlighting hot and cold trends.

    New models

    • Sparse GNMT - A Tensorflow implementation of the GNMT model with sparsity and quantization operations integrated.
    • Semantic Relation Identification - Extract semantic relation types of two words or phrases using external resources.
    • Sieve-based Cross Document Coreference - A seive-based model for finding similar entities or events across different documents from the same domain.

    Improvements

    • Reading comprehension - added inference mode.
    • Sequential models - updated NER, IE, Chunker models to use tf.keras and added CNN-character based feature extractors and improved accuracy of all models.
    • CRF Layer - added native Tensorflow based CRF layer.
    • Word Sense Disambiguation - model updated to use tf.keras.
    • Demo UI - updated demo UI using AngularJS.
    • Installation - improved installation process and added support for CPU/MKL/GPU backends for Tensorflow.
    • NLP Architect cmd - added nlp_architect - a simple command initiator to handle maintenance tasks, see nlp_architect -h for the list of commands.
    • Lots of bug fixes and refactoring.
    Source code(tar.gz)
    Source code(zip)
  • v0.2(Aug 15, 2018)

    Release v0.2

    New Solution

    • Term Set Expansion - the task of expanding a given partial set of terms into a more complete set of terms that belong to the same semantic class. This solution demonstrates the usage of NLP Architect models (Word Chunker and NP2Vec) used in an application solution.

    New models

    • Unsupervised Crosslingual Embeddings model using a GAN to learn a mapping between languages - implemented in Tensorflow
    • Language Model (LM) using Temporal Convolution Network (TCN) - implemented in Tensorflow
    • Supervised Sentiment Classification - implemented in Keras

    Model improvements

    • Reading comprehension - refactored to use Tensorflow
    • End-to-end Memory Network for Goal Oriented Dialogue - refactored to use Tensorflow
    • Word Chunker - refactored to use tf.keras and use state-of-art model
    • NP semantic segmentation - refactored to use tf.keras
    • Updated CONLL2000, Amazon_Review, PTB, Fasttext, Wikitext-103 and Wikipedia-dump dataset loaders.

    New features

    • REST server refactored to use hug, new streamlined the UI and improved documentation. See updated documentation for further details.
    • Noun Phrase annotator plug-in for spaCy pipeline
    • Publications page with relevant material demonstrating the usage of NLP Architect
    • Tutorials page with a collection of Jupyter notebook tutorials using NLP Architect models
    Source code(tar.gz)
    Source code(zip)
  • 0.1(May 29, 2018)

    The current version of NLP Architect includes these features that we found interesting from both research perspectives and practical applications:

    • NLP core models that allow robust extraction of linguistic features for NLP workflow: for example, dependency parser (BIST) and NP chunker
    • NLU modules that provide best in class performance: for example, intent extraction (IE), name entity recognition (NER)
    • Modules that address semantic understanding: for example, colocations, most common word sense, NP embedding representation (e.g. NP2V)
    • Components instrumental for conversational AI: for example, ChatBot applications, including dialog system, sequence chunking and IE
    • End-to-end DL applications using new topologies: for example, Q&A, machine reading comprehension
    Source code(tar.gz)
    Source code(zip)
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

A Deep Learning NLP/NLU library by Intel® AI Lab Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing NLP Architect

Intel Labs 2.7k Oct 16, 2021
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

A Deep Learning NLP/NLU library by Intel® AI Lab Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing NLP Architect

Intel Labs 2.6k Feb 18, 2021
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

A Deep Learning NLP/NLU library by Intel® AI Lab Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing NLP Architect

Intel Labs 2.8k Oct 21, 2021
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 52.8k Oct 21, 2021
Mirco Ravanelli 2.1k Oct 18, 2021
Graph4nlp is the library for the easy use of Graph Neural Networks for NLP

Graph4NLP Graph4NLP is an easy-to-use library for R&D at the intersection of Deep Learning on Graphs and Natural Language Processing (i.e., DLG4NLP).

Graph4AI 1.1k Oct 22, 2021
Coreference resolution for English, German and Polish, optimised for limited training data and easily extensible for further languages

Coreferee Author: Richard Paul Hudson, msg systems ag 1. Introduction 1.1 The basic idea 1.2 Getting started 1.2.1 English 1.2.2 German 1.2.3 Polish 1

msg systems ag 103 Oct 14, 2021
PyTorch original implementation of Cross-lingual Language Model Pretraining.

XLM NEW: Added XLM-R model. PyTorch original implementation of Cross-lingual Language Model Pretraining. Includes: Monolingual language model pretrain

Facebook Research 2.5k Oct 15, 2021
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 40.9k Feb 18, 2021
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 52.8k Oct 22, 2021
Deep Learning for Natural Language Processing - Lectures 2021

This repository contains slides for the course "20-00-0947: Deep Learning for Natural Language Processing" (Technical University of Darmstadt, Summer term 2021).

null 94 Oct 18, 2021
The guide to tackle with the Text Summarization

The guide to tackle with the Text Summarization

Takahiro Kubo 1.1k Oct 20, 2021
State of the Art Natural Language Processing

Spark NLP: State of the Art Natural Language Processing Spark NLP is a Natural Language Processing library built on top of Apache Spark ML. It provide

John Snow Labs 2.4k Oct 22, 2021
State of the Art Natural Language Processing

Spark NLP: State of the Art Natural Language Processing Spark NLP is a Natural Language Processing library built on top of Apache Spark ML. It provide

John Snow Labs 1.9k Feb 18, 2021
Unsupervised Language Modeling at scale for robust sentiment classification

** DEPRECATED ** This repo has been deprecated. Please visit Megatron-LM for our up to date Large-scale unsupervised pretraining and finetuning code.

NVIDIA Corporation 1k Oct 17, 2021
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP

TextAttack ?? Generating adversarial examples for NLP models [TextAttack Documentation on ReadTheDocs] About • Setup • Usage • Design About TextAttack

QData 1.7k Oct 23, 2021
Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Amazon Web Services - Labs 1000 Apr 19, 2021
A Paper List for Speech Translation

Keyword: Speech Translation, Spoken Language Processing, Natural Language Processing

null 89 Oct 21, 2021
Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Sockeye This package contains the Sockeye project, an open-source sequence-to-sequence framework for Neural Machine Translation based on Apache MXNet

Amazon Web Services - Labs 1k Oct 13, 2021