DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Last update: Nov 14, 2022

Related tags

Overview

DeeBERT

This is the code base for the paper DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference.

Code in this repository is also available in the Huggingface Transformer repo (with minor modification for version compatibility). Check this page for models that we have trained in advance (the latest version of Huggingface Transformers Library is needed).

Installation

This repo is tested on Python 3.7.5, PyTorch 1.3.1, and Cuda 10.1. Using a virtulaenv or conda environemnt is recommended, for example:

conda install pytorch==1.3.1 torchvision cudatoolkit=10.1 -c pytorch

After installing the required environment, clone this repo, and install the following requirements:

git clone https://github.com/castorini/deebert
cd deebert
pip install -r ./requirements.txt
pip install -r ./examples/requirements.txt

Usage

There are four scripts in the scripts folder, which can be run from the repo root, e.g., scripts/train.sh.

In each script, there are several things to modify before running:

path to the GLUE dataset. Check this for more details.
path for saving fine-tuned models. Default: ./saved_models.
path for saving evaluation results. Default: ./plotting. Results are printed to stdout and also saved to npy files in this directory to facilitate plotting figures and further analyses.
model_type (bert or roberta)
model_size (base or large)
dataset (SST-2, MRPC, RTE, QNLI, QQP, or MNLI)

train.sh

This is for fine-tuning and evaluating models as in the original BERT paper.

train_highway.sh

This is for fine-tuning DeeBERT models.

eval_highway.sh

This is for evaluating each exit layer for fine-tuned DeeBERT models.

eval_entropy.sh

This is for evaluating fine-tuned DeeBERT models, given a number of different early exit entropy thresholds.

Citation

Please cite our paper if you find the repository useful:

@inproceedings{xin-etal-2020-deebert,
    title = "{D}ee{BERT}: Dynamic Early Exiting for Accelerating {BERT} Inference",
    author = "Xin, Ji  and
      Tang, Raphael  and
      Lee, Jaejun  and
      Yu, Yaoliang  and
      Lin, Jimmy",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.204",
    pages = "2246--2251",
}

Comments

Replication results

Ubuntu 18.10 Python 3.7.7 CUDA 10.1

train.sh (time 0.09) acc = 0.8676470588235294 f1 = 0.906896551724138

train_highway.sh (time 0.13) acc = 0.8676470588235294 f1 = 0.9072164948453608

eval_entropy.sh 0.0 Result: 0.9072164948453608 Eval time: 14.078018426895142 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 408} Expected saving 1.0

0.001 Result: 0.9072164948453608 Eval time: 13.798500061035156 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 408} Expected saving 1.0

0.005 Result: 0.9072164948453608 Eval time: 14.127072811126709 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 408} Expected saving 1.0

0.01 Result: 0.9072164948453608 Eval time: 13.872061491012573 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 408} Expected saving 1.0

0.05 Result: 0.9072164948453608 Eval time: 13.464979648590088 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 49, 11: 62, 12: 297} Expected saving 0.9673202614379085

0.1 Result: 0.9072164948453608 Eval time: 12.848443746566772 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 4, 7: 0, 8: 1, 9: 10, 10: 110, 11: 58, 12: 225} Expected saving 0.9313725490196079

0.15 Result: 0.9090909090909091 Eval time: 12.455684423446655 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 21, 7: 13, 8: 5, 9: 19, 10: 112, 11: 54, 12: 184} Expected saving 0.8884803921568627

0.2 Result: 0.9094017094017094 Eval time: 11.937445402145386 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 58, 7: 19, 8: 3, 9: 23, 10: 107, 11: 44, 12: 154} Expected saving 0.8402777777777778

0.3 Result: 0.8989898989898989 Eval time: 10.921518564224243 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 0, 5: 9, 6: 120, 7: 21, 8: 1, 9: 22, 10: 88, 11: 46, 12: 101} Expected saving 0.7589869281045751

0.4 Result: 0.8870151770657673 Eval time: 9.648621082305908 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 1, 5: 69, 6: 117, 7: 18, 8: 3, 9: 31, 10: 72, 11: 37, 12: 60} Expected saving 0.6795343137254902

0.5 Result: 0.8795986622073578 Eval time: 8.683350801467896 Exit layer counter {1: 0, 2: 0, 3: 0, 4: 61, 5: 105, 6: 86, 7: 19, 8: 5, 9: 27, 10: 43, 11: 31, 12: 31} Expected saving 0.5808823529411765

0.6 Result: 0.8516129032258065 Eval time: 6.725062131881714 Exit layer counter {1: 1, 2: 56, 3: 20, 4: 158, 5: 35, 6: 66, 7: 12, 8: 4, 9: 13, 10: 16, 11: 13, 12: 14} Expected saving 0.42483660130718953

0.7 Result: 0.8122270742358079 Eval time: 2.7605879306793213 Exit layer counter {1: 408, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 0} Expected saving 0.08333333333333333

opened by wongalvis14 5
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

Hi there I was using your model for my new classification task on Japanese text so I added the configuration needed for Japanese Bert I'm doing the training on the BertForSequenceClassifcation model and when I do validation I encountered a bug that said the if condition was done on a tensor with more than one value (that tensor is actually the highway entropy of a batch) so that threw a runtime error when compared with the early exit threshold Can you help me with this ? I'd appreciate your help on this problem Here is the snippet of the error code

opened by quanhoang288 3
Entropy calculation can be modified
When calculating entropy, dim=1 is better to be replaced with dim=-1, since num_labels is the last dimension of logits but not always the 2nd dimension (e.g., in Token Classification, num_labels is the 3rd dimension).

And what about using softmax directly?

def entropy(x): # my attempt x = torch.softmax(x, dim=-1) # softmax normalized prob distribution return -torch.sum(x*torch.log(x), dim=-1) # entropy calculation on probs: -\sum(p \ln(p))

def entropy(x): # original code exp_x = torch.exp(x) A = torch.sum(exp_x, dim=1) # sum of exp(x_i) B = torch.sum(x*exp_x, dim=1) # sum of x_i * exp(x_i) return torch.log(A) - B/A

It seems that the outputs are no different, and using softmax is generally more efficient than manual calculation.

Since DeeBERT is proposed for efficiency, it's better to choose the efficient way.

Mind if I open a PR for this? @ji-xin
opened by sbwww 1
Confusing code in class BertHighway in modeling_highway_bert.py
Greetings, DeeBERT is really a crucial and easy-to-understand achievement in BERT inference acceleration.

However, in transformers/modeling_highway_bert.py, the forward function of class BertHighway is a bit confusing. Your original code is as follows

def forward(self, encoder_outputs): # Pooler pooler_input = encoder_outputs[0] pooler_output = self.pooler(pooler_input) # "return" pooler_output # BertModel bmodel_output = (pooler_input, pooler_output) +encoder_outputs[1:] # "return" bodel_output # Dropout and classification pooled_output = bmodel_output[1] pooled_output = self.dropout(pooled_output) logits = self.classifier(pooled_output) return logits, pooled_output

I am not quite sure about introducing bmodel_output for it's not actually used.

It seems that pooler_input is sequence_output, and pooled_output is equivalent to pooler_output.

Is there any trick that should be noticed? Maybe the comments starting with "return" can be updated for more details.
opened by sbwww 1
Bump tensorflow from 2.0.0rc2 to 2.5.0rc0 in /docs
Bumps tensorflow from 2.0.0rc2 to 2.5.0rc0.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.0-rc0

Release 2.5.0

Major Features and Improvements

TPU embedding support

Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

tf.keras.metrics.AUC now support logit predictions.

Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.

tf.data:

tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.

tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).

tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.

Options returned by tf.data.Dataset.options() are no longer mutable.

tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().

tf.lite

Enabled the new MLIR-based quantization backend by default

The new backend is used for 8 bits full integer post-training quantization

The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)

Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change

tf.keras

Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.

tf.distribute

tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.

PluggableDevice

Third-party devices can now connect to TensorFlow modularly through StreamExecutor C API and PluggableDevice interface.

Add custom ops and kernels through kernel and op registration C API.

Register custom graph optimization passes with

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.6.0

Breaking Changes

tf.train.experimental.enable_mixed_precision_graph_rewrite is removed, as the API only works in graph mode and is not customizable. The function is still accessible under tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite, but it is recommended to use the Keras mixed precision API instead.

tf.lite:

Remove experimental.nn.dynamic_rnn, experimental.nn.TfLiteRNNCell and experimental.nn.TfLiteLSTMCell since they're no longer supported. It's recommended to just use keras lstm instead.

* *

Known Caveats

* * *

TF Core:

A longstanding bug in tf.while_loop, which caused it to execute sequentially, even when parallel_iterations>1, has now been fixed. However, the increased parallelism may result in increased memory use. Users who experience unwanted regressions should reset their while_loop's parallel_iterations value to 1, which is consistent with prior behavior.

Major Features and Improvements

* *

tf.keras:

tf.keras.utils.experimental.DatasetCreator now takes an optional tf.distribute.InputOptions for specific options when used with distribution.

Updates to Preprocessing layers API for consistency and clarity:

StringLookup and IntegerLookup default for mask_token changed to None. This matches the default masking behavior of Hashing and Embedding layers. To keep existing behavior, pass mask_token="" during layer creation.

... (truncated)

Commits

a8b6d5f Merge pull request #48222 from tensorflow/mm-fix-fileystem-on-r2.5

b9e31e6 Fix typo/logic bug in modular plugins.

158505e Switch TF filesystems to keep backwards compatibility.

96dfa5c Merge pull request #48107 from tensorflow/mihaimaruseac-patch-1

5f7fd89 Fix typo in setup.py

f8b5b9b Merge pull request #48093 from tensorflow/mihaimaruseac-patch-1

b84dac5 Update setup.py

b42047d Merge pull request #48091 from tensorflow-jenkins/version-numbers-2.5.0rc0-30114

1d4885b Update version numbers to 2.5.0-rc0

6af4297 Merge pull request #48082 from njeffrie:f1_depthwise

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tensorflow from 2.0.0 to 2.5.0
Bumps tensorflow from 2.0.0 to 2.5.0.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.0

Release 2.5.0

Major Features and Improvements

Support for Python3.9 has been added.

tf.data:

tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.

tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).

tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.

Options returned by tf.data.Dataset.options() are no longer mutable.

tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().

tf.lite

Enabled the new MLIR-based quantization backend by default

The new backend is used for 8 bits full integer post-training quantization

The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)

Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change

tf.keras

tf.keras.metrics.AUC now support logit predictions.

Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.

tf.distribute

tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.

Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.

TPU embedding support

Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

PluggableDevice

Third-party devices can now connect to TensorFlow as plug-ins through StreamExecutor C API. and PluggableDevice interface.

Add custom ops and kernels through kernel and op registration C API.

Register custom graph optimization passes with graph optimization C API.

oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.

They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.

We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.

TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

tf.keras:

Preprocessing layers API consistency changes:

StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.

IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.

TextVectorization default for pad_to_max_tokens switched to False.

CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.

Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.

Improvements to model saving/loading:

model.load_weights now accepts paths to saved models.

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.5.0

Breaking Changes

The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Known Caveats

Major Features and Improvements

TPU embedding support

Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

tf.keras.metrics.AUC now support logit predictions.

Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.

tf.data:

tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.

tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).

tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.

Options returned by tf.data.Dataset.options() are no longer mutable.

tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().

tf.lite

Enabled the new MLIR-based quantization backend by default

The new backend is used for 8 bits full integer post-training quantization

The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)

... (truncated)

Commits

a4dfb8d Merge pull request #49124 from tensorflow/mm-cherrypick-tf-data-segfault-fix-...

2107b1d Merge pull request #49116 from tensorflow-jenkins/version-numbers-2.5.0-17609

16b8139 Update snapshot_dataset_op.cc

86a0d86 Merge pull request #49126 from geetachavan1/cherrypicks_X9ZNY

9436ae6 Merge pull request #49128 from geetachavan1/cherrypicks_D73J5

6b2bf99 Validate that a and b are proper sparse tensors

c03ad1a Ensure validation sticks in banded_triangular_solve_op

12a6ead Merge pull request #49120 from geetachavan1/cherrypicks_KJ5M9

b67f5b8 Merge pull request #49118 from geetachavan1/cherrypicks_BIDTR

a13c0ad [tf.data][cherrypick] Fix snapshot segfault when using repeat and prefecth

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump urllib3 from 1.25.6 to 1.25.8
Bumps urllib3 from 1.25.6 to 1.25.8.

Release notes

Sourced from urllib3's releases.

1.25.8

Release: 1.25.8

1.25.7

No release notes provided.

Changelog

Sourced from urllib3's changelog.

1.25.8 (2020-01-20)

Drop support for EOL Python 3.4 (Pull #1774)

Optimize _encode_invalid_chars (Pull #1787)

1.25.7 (2019-11-11)

Preserve chunked parameter on retries (Pull #1715, Pull #1734)

Allow unset SERVER_SOFTWARE in App Engine (Pull #1704, Issue #1470)

Fix issue where URL fragment was sent within the request target. (Pull #1732)

Fix issue where an empty query section in a URL would fail to parse. (Pull #1732)

Remove TLS 1.3 support in SecureTransport due to Apple removing support (Pull #1703)

Commits

2a57bc5 Release 1.25.8 (#1788)

a2697e7 Optimize _encode_invalid_chars (#1787)

d2a5a59 Move IPv6 test skips in server fixtures

d44f0e5 Factorize test certificates serialization

84abc7f Generate IPV6 certificates using trustme

6a15b18 Run IPv6 Tornado server from fixture

4903840 Use trustme to generate IP_SAN cert

9971e27 Empty responses should have no lines.

62ef68e Use trustme to generate NO_SAN certs

fd2666e Use fixture to configure NO_SAN test certs

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump urllib3 from 1.25.3 to 1.25.8 in /docs
Bumps urllib3 from 1.25.3 to 1.25.8.

Release notes

Sourced from urllib3's releases.

1.25.8

Release: 1.25.8

1.25.7

No release notes provided.

1.25.6

Release: 1.25.6

1.25.5

Release: 1.25.5

1.25.4

Release: 1.25.4

Changelog

Sourced from urllib3's changelog.

1.25.8 (2020-01-20)

Drop support for EOL Python 3.4 (Pull #1774)

Optimize _encode_invalid_chars (Pull #1787)

1.25.7 (2019-11-11)

Preserve chunked parameter on retries (Pull #1715, Pull #1734)

Allow unset SERVER_SOFTWARE in App Engine (Pull #1704, Issue #1470)

Fix issue where URL fragment was sent within the request target. (Pull #1732)

Fix issue where an empty query section in a URL would fail to parse. (Pull #1732)

Remove TLS 1.3 support in SecureTransport due to Apple removing support (Pull #1703)

1.25.6 (2019-09-24)

Fix issue where tilde (~) characters were incorrectly percent-encoded in the path. (Pull #1692)

1.25.5 (2019-09-19)

Add mitigation for BPO-37428 affecting Python <3.7.4 and OpenSSL 1.1.1+ which caused certificate verification to be enabled when using cert_reqs=CERT_NONE. (Issue #1682)

1.25.4 (2019-09-19)

Propagate Retry-After header settings to subsequent retries. (Pull #1607)

Fix edge case where Retry-After header was still respected even when explicitly opted out of. (Pull #1607)

Remove dependency on rfc3986 for URL parsing.

Fix issue where URLs containing invalid characters within Url.auth would raise an exception instead of percent-encoding those characters.

... (truncated)

Commits

2a57bc5 Release 1.25.8 (#1788)

a2697e7 Optimize _encode_invalid_chars (#1787)

d2a5a59 Move IPv6 test skips in server fixtures

d44f0e5 Factorize test certificates serialization

84abc7f Generate IPV6 certificates using trustme

6a15b18 Run IPv6 Tornado server from fixture

4903840 Use trustme to generate IP_SAN cert

9971e27 Empty responses should have no lines.

62ef68e Use trustme to generate NO_SAN certs

fd2666e Use fixture to configure NO_SAN test certs

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump py from 1.8.0 to 1.10.0
Bumps py from 1.8.0 to 1.10.0.

Changelog

Sourced from py's changelog.

1.10.0 (2020-12-12)

Fix a regular expression DoS vulnerability in the py.path.svnwc SVN blame functionality (CVE-2020-29651)

Update vendored apipkg: 1.4 => 1.5

Update vendored iniconfig: 1.0.0 => 1.1.1

1.9.0 (2020-06-24)

Add type annotation stubs for the following modules:

py.error

py.iniconfig

py.path (not including SVN paths)

py.io

py.xml

There are no plans to type other modules at this time.

The type annotations are provided in external .pyi files, not inline in the code, and may therefore contain small errors or omissions. If you use py in conjunction with a type checker, and encounter any type errors you believe should be accepted, please report it in an issue.

1.8.2 (2020-06-15)

On Windows, py.path.locals which differ only in case now have the same Python hash value. Previously, such paths were considered equal but had different hashes, which is not allowed and breaks the assumptions made by dicts, sets and other users of hashes.

1.8.1 (2019-12-27)

Handle FileNotFoundError when trying to import pathlib in path.common on Python 3.4 (#207).

py.path.local.samefile now works correctly in Python 3 on Windows when dealing with symlinks.

Commits

e5ff378 Update CHANGELOG for 1.10.0

94cf44f Update vendored libs

5e8ded5 testing: comment out an assert which fails on Python 3.9 for now

afdffcc Rename HOWTORELEASE.rst to RELEASING.rst

2de53a6 Merge pull request #266 from nicoddemus/gh-actions

fa1b32e Merge pull request #264 from hugovk/patch-2

887d6b8 Skip test_samefile_symlink on pypy3 on Windows

e94e670 Fix test_comments() in test_source

fef9a32 Adapt test

4a694b0 Add GitHub Actions badge to README

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump pygments from 2.4.2 to 2.7.4 in /docs
Bumps pygments from 2.4.2 to 2.7.4.

Release notes

Sourced from pygments's releases.

2.7.4

Updated lexers:

Apache configurations: Improve handling of malformed tags (#1656)

CSS: Add support for variables (#1633, #1666)

Crystal (#1650, #1670)

Coq (#1648)

Fortran: Add missing keywords (#1635, #1665)

Ini (#1624)

JavaScript and variants (#1647 -- missing regex flags, #1651)

Markdown (#1623, #1617)

Shell

Lex trailing whitespace as part of the prompt (#1645)

Add missing in keyword (#1652)

SQL - Fix keywords (#1668)

Typescript: Fix incorrect punctuation handling (#1510, #1511)

Fix infinite loop in SML lexer (#1625)

Fix backtracking string regexes in JavaScript/TypeScript, Modula2 and many other lexers (#1637)

Limit recursion with nesting Ruby heredocs (#1638)

Fix a few inefficient regexes for guessing lexers

Fix the raw token lexer handling of Unicode (#1616)

Revert a private API change in the HTML formatter (#1655) -- please note that private APIs remain subject to change!

Fix several exponential/cubic-complexity regexes found by Ben Caller/Doyensec (#1675)

Fix incorrect MATLAB example (#1582)

Thanks to Google's OSS-Fuzz project for finding many of these bugs.

2.7.3

Updated lexers:

Ada (#1581)

HTML (#1615, #1614)

Java (#1594, #1586)

JavaScript (#1605, #1589, #1588)

JSON (#1569 -- this is a complete rewrite)

Lean (#1601)

LLVM (#1612)

Mason (#1592)

MySQL (#1555, #1551)

Rust (#1608)

Turtle (#1590, #1553)

Deprecated JsonBareObjectLexer, which is now identical to JsonLexer (#1600)

The ImgFormatter now calculates the exact character width, which fixes some issues with overlapping text (#1213, #1611)

... (truncated)

Changelog

Sourced from pygments's changelog.

Version 2.7.4

(released January 12, 2021)

Updated lexers:

Apache configurations: Improve handling of malformed tags (#1656)

CSS: Add support for variables (#1633, #1666)

Crystal (#1650, #1670)

Coq (#1648)

Fortran: Add missing keywords (#1635, #1665)

Ini (#1624)

JavaScript and variants (#1647 -- missing regex flags, #1651)

Markdown (#1623, #1617)

Shell

Lex trailing whitespace as part of the prompt (#1645)

Add missing in keyword (#1652)

SQL - Fix keywords (#1668)

Typescript: Fix incorrect punctuation handling (#1510, #1511)

Fix infinite loop in SML lexer (#1625)

Fix backtracking string regexes in JavaScript/TypeScript, Modula2 and many other lexers (#1637)

Limit recursion with nesting Ruby heredocs (#1638)

Fix a few inefficient regexes for guessing lexers

Fix the raw token lexer handling of Unicode (#1616)

Revert a private API change in the HTML formatter (#1655) -- please note that private APIs remain subject to change!

Fix several exponential/cubic-complexity regexes found by Ben Caller/Doyensec (#1675)

Fix incorrect MATLAB example (#1582)

Thanks to Google's OSS-Fuzz project for finding many of these bugs.

Version 2.7.3

(released December 6, 2020)

Updated lexers:

Ada (#1581)

HTML (#1615, #1614)

Java (#1594, #1586)

JavaScript (#1605, #1589, #1588)

JSON (#1569 -- this is a complete rewrite)

Lean (#1601)

LLVM (#1612)

Mason (#1592)

... (truncated)

Commits

4d555d0 Bump version to 2.7.4.

fc3b05d Update CHANGES.

ad21935 Revert "Added dracula theme style (#1636)"

e411506 Prepare for 2.7.4 release.

275e34d doc: remove Perl 6 ref

2e7e8c4 Fix several exponential/cubic complexity regexes found by Ben Caller/Doyensec

eb39c43 xquery: fix pop from empty stack

2738778 fix coding style in test_analyzer_lexer

02e0f09 Added 'ERROR STOP' to fortran.py keywords. (#1665)

c83fe48 support added for css variables (#1633)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump jinja2 from 2.10.1 to 2.11.3 in /docs
Bumps jinja2 from 2.10.1 to 2.11.3.

Release notes

Sourced from jinja2's releases.

2.11.3

This contains a fix for a speed issue with the urlize filter. urlize is likely to be called on untrusted user input. For certain inputs some of the regular expressions used to parse the text could take a very long time due to backtracking. As part of the fix, the email matching became slightly stricter. The various speedups apply to urlize in general, not just the specific input cases.

PyPI: https://pypi.org/project/Jinja2/2.11.3/

Changes: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-3

2.11.2

Changelog: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-2

2.11.1

This fixes an issue in async environment when indexing the result of an attribute lookup, like {{ data.items[1:] }}.

Changes: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-1

2.11.0

Changes: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-0

Blog: https://palletsprojects.com/blog/jinja-2-11-0-released/

Twitter: https://twitter.com/PalletsTeam/status/1221883554537230336

This is the last version to support Python 2.7 and 3.5. The next version will be Jinja 3.0 and will support Python 3.6 and newer.

2.10.3

Changes: http://jinja.palletsprojects.com/en/2.10.x/changelog/#version-2-10-3

2.10.2

Changes: http://jinja.palletsprojects.com/en/2.10.x/changelog/#version-2-10-2

Changelog

Sourced from jinja2's changelog.

Version 2.11.3

Released 2021-01-31

Improve the speed of the urlize filter by reducing regex backtracking. Email matching requires a word character at the start of the domain part, and only word characters in the TLD. :pr:1343

Version 2.11.2

Released 2020-04-13

Fix a bug that caused callable objects with __getattr__, like :class:~unittest.mock.Mock to be treated as a :func:contextfunction. :issue:1145

Update wordcount filter to trigger :class:Undefined methods by wrapping the input in :func:soft_str. :pr:1160

Fix a hang when displaying tracebacks on Python 32-bit. :issue:1162

Showing an undefined error for an object that raises AttributeError on access doesn't cause a recursion error. :issue:1177

Revert changes to :class:~loaders.PackageLoader from 2.10 which removed the dependency on setuptools and pkg_resources, and added limited support for namespace packages. The changes caused issues when using Pytest. Due to the difficulty in supporting Python 2 and :pep:451 simultaneously, the changes are reverted until 3.0. :pr:1182

Fix line numbers in error messages when newlines are stripped. :pr:1178

The special namespace() assignment object in templates works in async environments. :issue:1180

Fix whitespace being removed before tags in the middle of lines when lstrip_blocks is enabled. :issue:1138

:class:~nativetypes.NativeEnvironment doesn't evaluate intermediate strings during rendering. This prevents early evaluation which could change the value of an expression. :issue:1186

Version 2.11.1

Released 2020-01-30

Fix a bug that prevented looking up a key after an attribute ({{ data.items[1:] }}) in an async template. :issue:1141

... (truncated)

Commits

cf21539 release version 2.11.3

15ef8f0 Merge pull request #1343 from pallets/urlize-speedup

ef658dc speed up urlize matching

eeca0fe Merge pull request #1207 from mhansen/patch-1

2dd7691 Merge pull request #1209 from mhansen/patch-3

4892940 do_dictsort: update example ready to copy/paste

7db7d33 api.rst: bugfix in docs, import PackageLoader

9ec465b fix changelog header

737a4cd release version 2.11.2

179df6b Merge pull request #1190 from pallets/native-eval

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
The real inference acceleration is not GPU friendly---batch operator not supported

When inference with the highway early-exit given a batch B; when |B| = 1, the code is ok to run; when |B| > 1, the code can corrupt in the code of modeling_highway_bert.py due to if highway_entropy < self.early_exit_entropy[i]: conditional statement ; Only one sample can be inferred. Therefore, the inference is not GPU friendly.

opened by zhangzhenyu13 0
Adding Additional Transformer Models

Hello, I was running some experiments with DeeBERT and was wondering if there is any set of steps for adding new models beyond the ones currently included in the repository. I see that the RoBERTa and BERT models are preloaded in a custom setup so I was wondering if there was any process for adding our own models. Thanks!

opened by jonsaadfalcon 0