Easy-to-use library to boost AI inference leveraging state-of-the-art optimization techniques.

Related tags

Deep Learning machine-learning deep-learning compiler tensorflow inference pytorch computing tensorrt tvm onnx openvino huggingface
Overview

NEW RELEASE

How Nebullvm Works • Tutorials • Benchmarks • Installation • Get Started • Optimization Examples

Discord | Website | LinkedIn | Twitter

Nebullvm

nebullvm speeds up AI inference by 2-30x in just a few lines of code 🚀

How Nebullvm Works

This open-source library takes your AI model as input and outputs an optimized version that runs 2-30 times faster on your hardware. Nebullvm tests multiple optimization techniques (deep learning compilers, quantization, sparsity, distillation, and more) to identify the optimal way to execute your AI model on your specific hardware. The library can speed up your model 2 to 10 times without loss of performance, or up to 30 times if you specify that you are willing to trade off a self-defined amount of accuracy/precision for a super-low latency and a lighter model.

The goal of nebullvm is to let any developer benefit from the most advanced inference optimization techniques without having to spend countless hours understanding, installing, testing and debugging these powerful technologies.

The library aims to be:

☘️  Easy-to-use. It takes a few lines of code to install the library and optimize your models.

🔥  Framework agnostic. nebullvm supports the most widely used frameworks (PyTorch, TensorFlow, ONNX and Hugging Face, etc.) and provides as output an optimized version of your model with the same interface (PyTorch, TensorFlow, etc.).

💻  Deep learning model agnostic. nebullvm supports all the most popular architectures such as transformers, LSTMs, CNNs and FCNs.

🤖  Hardware agnostic. The library now works on most CPUs and GPUs and will soon support TPUs and other deep learning-specific ASICs.

🔑  Secure. Everything runs locally on your machine.

✨  Leveraging the best optimization techniques. There are many inference optimization techniques such as deep learning compilers, quantization, half precision or distillation, which are all meant to optimize the way your AI models run on your hardware. It would take a developer countless hours to install and test them on every model deployment. The library does that for you.

Do you like the concept? Leave a  ⭐  if you enjoy the project and join the Discord community where we chat about nebullvm and AI optimization. And happy acceleration  🚀 🚀

Benchmarks

We have tested nebullvm on popular AI models and hardware from leading vendors.

The table below shows the inference speedup provided by nebullvm. The speedup is calculated as the response time of the unoptimized model divided by the response time of the accelerated model, as an average over 100 experiments. As an example, if the response time of an unoptimized model was on average 600 milliseconds and after nebullvm optimization only 240 milliseconds, the resulting speedup is 2.5x times, meaning 150% faster inference.

A complete overview of the experiment and findings can be found on this page.

M1 Pro Intel Xeon AMD EPYC Nvidia T4
EfficientNetB0 23.3x 3.5x 2.7x 1.3x
EfficientNetB2 19.6x 2.8x 1.5x 2.7x
EfficientNetB6 19.8x 2.4x 2.5x 1.7x
Resnet18 1.2x 1.9x 1.7x 7.3x
Resnet152 1.3x 2.1x 1.5x 2.5x
SqueezeNet 1.9x 2.7x 2.0x 1.3x
Convnext tiny 3.2x 1.3x 1.8x 5.0x
Convnext large 3.2x 1.1x 1.6x 4.6x
GPT2 - 10 tokens 2.8x 3.2x 2.8x 3.8x
GPT2 - 1024 tokens - 1.7x 1.9x 1.4x
Bert - 8 tokens 6.4x 2.9x 4.8x 4.1x
Bert - 512 tokens 1.8x 1.3x 1.6x 3.1x
____________________ ____________ ____________ ____________ ____________

Overall, the library provides great results, with more than 2x acceleration in most cases and around 20x in a few applications. We can also observe that acceleration varies greatly across different hardware-model couplings, so we suggest you test nebullvm on your model and hardware to assess its full potential. You can find the instructions below.

Besides, across all scenarios, nebullvm is very helpful for its ease of use, allowing you to take advantage of inference optimization techniques without having to spend hours studying, testing and debugging these technologies.

Tutorials

We suggest testing the library on your AI models right away by following the installation instructions below. If you want to get a first feel of the library's capabilities or take a look at how nebullvm can be readily implemented in an AI workflow, we have built 3 tutorials and notebooks where the library can be tested on the most popular AI frameworks TensorFlow, PyTorch and Hugging Face.

  • Notebook: Accelerate fast.ai's Resnet34 with nebullvm
  • Notebook: Accelerate PyTorch YOLO with nebullvm
  • Notebook: Accelerate Hugging Face's GPT2 and BERT with nebullvm

Installation

[Click to expand] Step 1: Installation of nebullvm library

There are two ways to install nebullvm:

  1. Using PyPI. We suggest installing the library with pip to get the stable version of nebullvm
  2. From source code to get the latest features

Option 1A: Installation with PyPI (recommended)

The easiest way to install nebullvm is by using pip, running

pip install nebullvm

Option 1B: Source code installation

To install the source code you have to clone the directory on your local machine using git.

git clone https://github.com/nebuly-ai/nebullvm.git

Then, enter the repo and install nebullvm with pip.

cd nebullvm
pip install .
[Click to expand] Step 2: Installation of deep learning compilers

Now you need to install the compilers that the library leverages to create the optimized version of your models. We have built an auto-installer to install them automatically.

Option 2A: Installation at the first optimization run

The auto-installer is activated after you import nebullvm and perform your first optimization. You may run into import errors related to the deep learning compiler installation, but you can ignore these errors/warnings. It is also recommended restarting the python kernel between the auto-installation and the first optimization, otherwise not all compilers will be activated.

Option B: Installation before the first optimization run (recommended)

To avoid any problems, we strongly recommend running the auto-installation before performing the first optimization by running

python -c "import nebullvm"

You should ignore at this stage any import warning resulting from the previous command.

Option 2C: Selective installation of deep learning compilers

The library automatically installs all deep learning compilers it supports. In case you would be interested in bypassing the automatic installation, you can export the environment variable NO_COMPILER_INSTALLATION=1 by running

export NO_COMPILER_INSTALLATION=1

from your command line or adding

import os
os.environ["NO_COMPILER_INSTALLATION"] = "1"

in your python code before importing nebullvm for the first time.

Note that auto-installation of open-source compilers is done outside the nebullvm wheel. Installations of ApacheTVM and Openvino have been tested on macOS, linux distributions similar to Debian and CentOS.

The feature is still in an alpha version, so we expect that it may fail under untested circumstances.

Step 2-bis: Install TVM

Since the TVM compiler has to be installed from source code, its installation can take several minutes, or even hours, to complete. For this reason, we decided not to include it in the default automatic installer. However, if you want to squeeze the most performance out of your model on your machine, we highly recommend installing TVM as well. With nebullvm, installing TVM becomes very easy, just run

python -c "from nebullvm.installers.installers import install_tvm; install_tvm()"

and wait for the compiler to be installed! You can check that everything worked running

python -c "from tvm.runtime import Module"
[Click to expand] Possible installation issues

MacOS: the installation may fail on MacOS for MacBooks with the Apple Silicon chip, due to scipy compilation errors. The easy fix is to install scipy with another package manager such as conda (the Apple Silicon distribution of Mini-conda) and then install nebullvm. For any additional issues do not hesitate to open an issue or contact directly [email protected] by email.

Get Started

Nebullvm reduces the computation time of deep learning model inference by 2-30 times by testing multiple optimization techniques (deep learning compilers, quantization, half precision, distillation, and more) and identifying the optimal way to execute your AI model on your specific hardware.

Nebullvm can be deployed in two ways.

Option A: 2-10x acceleration, NO performance loss

If you choose this option, nebullvm will test multiple deep learning compilers (TensorRT, OpenVINO, ONNX Runtime, etc.) and identify the optimal way to compile your model on your hardware, increasing inference speed by 2-10 times without affecting the performance of your model.

Option B: 2-30x acceleration, supervised performance loss

Nebullvm is capable of speeding up inference by much more than 10 times in case you are willing to sacrifice a fraction of your model's performance. If you specify how much performance loss you are willing to sustain, nebullvm will push your model's response time to its limits by identifying the best possible blend of state-of-the-art inference optimization techniques, such as deep learning compilers, distillation, quantization, half precision, sparsity, etc.

Performance monitoring is accomplished using the perf_loss_ths (performance loss threshold), and the perf_metric for performance estimation.

Option B.1

When a predefined metric (e.g. “accuracy”) or a custom metric is passed as the perf_metric argument, the value of perf_loss_ths will be used as the maximum acceptable loss for the given metric evaluated on your datasets.

Options B.2 and B.3

When no perf_metric is provided as input, nebullvm calculates the performance loss using the default precision function. If the dataset is provided, the precision will be calculated on 100 sampled data (option B.2). Otherwise, the data will be randomly generated from the metadata provided as input, i.e. input_sizes and batch_size (option B.3).

[Click to expand] Options B.2 and B.3: Impact of perf_loss_ths on precision

The table below shows the impact of perf_loss_ths on the default metric "precision".

perf_loss_ths Expected behavior with the default “precision” metric
None or 0 No precision-reduction technique (distillation, quantization, half precision, sparsity, etc.) will be applied, as per Option A.
1 Nebullvm will accept the outcome of precision-reduction techniques only if the relative change of the smallest output logit is smaller than 1. This is usually correlated with a marginal drop in precision.
2 Nebullvm will accept a "riskier" output from precision-reduction techniques to achieve increased inference speed. This can usually have an impact on the accuracy of ~0.1%.
≥3 Aggressive precision reduction techniques are used to produce the lightest and fastest model possible. Accuracy drops depend on both model type and task type. A simple binary classification can still show accuracy drops around ~0.1%.

Optimization examples

[Click to expand] Optimization with PyTorch Here we present an example of optimizing a `pytorch` model with `nebullvm`: 
>>> # FOR EACH OPTION
>>> import torch
>>> import torchvision.models as models
>>> from nebullvm import optimize_torch_model
>>> model = models.efficientnet_b0()
>>> save_dir = "."
>>>
>>> # ONLY FOR OPTION A 
>>> bs, input_sizes = 1, [(3, 256, 256)]
>>> optimized_model = optimize_torch_model(
... model, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir
... )
>>>
>>> # ONLY FOR OPTION B.1
>>> dl = [((torch.randn(1, 3, 256, 256), ), 0)]
>>> perf_loss_ths = 0.1  # We can accept a drop in the loss function up to 10%
>>> optimized_model = optimize_torch_model(
... model, dataloader=dl, save_dir=save_dir, perf_loss_ths=perf_loss_ths, perf_metric="accuracy", 
... )
>>>
>>> # ONLY FOR OPTION B.2
>>> dl = [((torch.randn(1, 3, 256, 256), ), 0)]
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> optimized_model = optimize_torch_model(
... model, dataloader=dl, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # ONLY FOR OPTION B.3
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> bs, input_sizes = 1, [(3, 256, 256)]
>>> optimized_model = optimize_torch_model(
... model, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # FOR EACH OPTION
>>> x = torch.randn(bs, 3, 256, 256)
>>> res = optimized_model(x)

In the example above for options B.1 and B.2 we provided a dataset containing a single tuple (xs, y) where xs itself is a tuple containing all the inputs needed for the model. Note that for nebullvm input dataset should be in the format Sequence[Tuple[Tuple[Tensor, ...], TensorOrNone]]. The torch API also accept dataloaders as inputs, however the dataloader should return each batch as a tuple (xs, y) as described before.

[Click to expand] Optimization with TensorFlow 
>>> # FOR EACH OPTION
>>> import tensorflow as tf 
>>> from tensorflow.keras.applications.resnet50 import ResNet50
>>> from nebullvm import optimize_tf_model
>>> model = ResNet50()
>>> save_dir = "."
>>>
>>> # ONLY FOR OPTION A
>>> bs, input_sizes = 1, [(224, 224, 3)]
>>> optimized_model = optimize_tf_model(
... model, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir
... )
>>>
>>> # ONLY FOR OPTION B.1
>>> input_data = [((tf.random_normal_inizializer()(shape=(1, 224, 224, 3)), ), 0)]
>>> perf_loss_ths = 0.1  # We can accept a drop in the loss function up to 10%
>>> optimized_model = optimize_tf_model(
... model, dataset=input_data, save_dir=save_dir, perf_loss_ths=perf_loss_ths, perf_metric="accuracy", 
... )
>>>
>>> # ONLY FOR OPTION B.2
>>> input_data = [((tf.random_normal_inizializer()(shape=(1, 224, 224, 3)), ), 0)]
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> optimized_model = optimize_tf_model(
... model, dataset=input_data, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # ONLY FOR OPTION B.3
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> bs, input_sizes = 1, [(224, 224, 3)]
>>> optimized_model = optimize_tf_model(
... model, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # FOR EACH OPTION
>>> res = optimized_model(*optimized_model.get_inputs_example())
[Click to expand] Optimization with ONNX 
>>> # FOR EACH OPTION
>>> from nebullvm import optimize_torch_model
>>> import numpy as np
>>> model_path = "path-to-onnx-model"
>>> save_dir = "."
>>>
>>> # ONLY FOR OPTION A
>>> bs, input_sizes = 1, [(3, 256, 256)]
>>> optimized_model = optimize_onnx_model(
... model_path, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir
... )
>>>
>>> # ONLY FOR OPTION B.1
>>> data = [((np.random.randn(1, 3, 256, 256).astype(np.float32), ), 0)]
>>> perf_loss_ths = 0.1  # We can accept a drop in the loss function up to 10%
>>> optimized_model = optimize_onnx_model(
... model_path, data=data, save_dir=save_dir, perf_loss_ths=perf_loss_ths, perf_metric="accuracy", 
... )
>>>
>>> # ONLY FOR OPTION B.2
>>> data = [((np.random.randn(1, 3, 256, 256).astype(np.float32), ), 0)]
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> optimized_model = optimize_onnx_model(
... model_path, data=data, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # ONLY FOR OPTION B.3
>>> perf_loss_ths = 2  # Relative error on the smallest logits accepted
>>> bs, input_sizes = 1, [(3, 256, 256)]
>>> optimized_model = optimize_onnx_model(
... model_path, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir, perf_loss_ths=perf_loss_ths, 
... )
>>>
>>> # FOR EACH OPTION
>>> x = np.random.randn(1, 3, 256, 256).astype(np.float32)
>>> res = optimized_model(x)
[Click to expand] Optimization with Hugging Face

To make nebullvm work with huggingface we have changed the API slightly so that you can use the optimize_huggingface_model function to optimize your model. Below we show an example of how to accelerate GPT2 with nebullvm without loss of accuracy by leveraging only deep learning compilers (option A).

To make `nebullvm` work with `huggingface` we have changed the API slightly so that you can use the `optimize_huggingface_model` function to optimize your model.

`>>> from transformers import GPT2Tokenizer, GPT2Model
>>> from nebullvm.api.frontend.huggingface import optimize_huggingface_model
>>> tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
>>> model = GPT2Model.from_pretrained('gpt2')
>>> text = "Replace me by any text you'd like."
>>> encoded_input = tokenizer(text, return_tensors='pt')
>>> optimized_model = optimize_huggingface_model(
...     model=model,
...     tokenizer=tokenizer,
...     input_texts=[text],
...     batch_size=1,
...     max_input_sizes=[
...       tuple(value.size()[1:]) 
...       for value in encoded_input.values()
...     ],
...     save_dir=".",
...     extra_input_info=[{}, {"max_value": 1, "min_value": 0}],
... )
>>> res = optimized_model(**encoded_input)`

Set the number of threads per model

When running multiple replicas of the model in parallel, it would be useful for CPU-optimized algorithms to limit the number of threads to use for each model. In nebullvm, it is possible to set the maximum number of threads a single model can use with the environment variable NEBULLVM_THREADS_PER_MODEL. For instance, you can run

export NEBULLVM_THREADS_PER_MODEL = 2

for using just two CPU-threads per model at inference time and during optimization.

Supported frameworks

  • PyTorch
  • TensorFlow
  • Hugging Face

Supported deep learning compilers

  • OpenVINO
  • TensorRT
  • TVM
  • MLIR (Coming soon 🚀 )

Integration with other open-source libraries

Deep learning libraries

Repositories of the best tools for AI

Do you want to integrate nebullvm in your open-source library? Try it out and if you need support, do not hesitate to contact us at [email protected].

The community for AI acceleration

Do you want to meet nebullvm contributors and other developers who share the vision of an superfast and sustainable artificial intelligence? Or would you like to report bugs or improvement ideas for nebullvm? Join the community for AI acceleration on Discord!

Acknowledgments

Nebullvm was built by Nebuly, with a major contribution by Diego Fiori, as well as a lot of support from the community who submitted pull requests, provided very useful feedback, and opened issues.

Nebullvm builds on the outstanding work being accomplished by the open-source community and major hardware vendors on deep learning compilers.

How Nebullvm Works • Tutorials • Benchmarks • Installation • Get Started • Optimization Examples

Discord | Website | LinkedIn | Twitter

Comments
  • Issues while setting up Nebullvm on Ubuntu

    Issues while setting up Nebullvm on Ubuntu

    Hey! I was installing nebullvm on a conda environment. Faced few issues. Please let me know how can I solve them. I know we have a docker version also, but I am keeping that incase normal installation process doesn't work for me.

    Configuration - Conda environment, Python 3.8.

    As per the documentation, I used pip command to install nebullvm and then followed the next guide to install TVM mentioned in docs itself.

    1. By running the pip install nebullvm, it installs Tensorflow==2.7.3 (I see in previous issues that Tensorflow 2.8 is not supported yet). Tensorflow 2.7.3 requires numpy>=1.19.0 and onnxruntime-gpu requires numpy>=1.21.0. So, this means I can either use Tensorflow or Onnxruntime-gpu with current configuration. Screenshot from 2022-07-06 18-52-16

    2. While installing openvino, some error popped up. Attaching screenshots. Screenshot from 2022-07-06 18-54-53 Screenshot from 2022-07-06 18-55-45

    3. Next, I ran - python -c "from nebullvm.installers.installers import install_tvm; install_tvm()" and got ImportError : numpy.core.multiarray failed to import. I guess this is because of numpy version we are using.

    4. While tvm was being built, received following errors Screenshot from 2022-07-06 19-16-37

    Few discussions/Solutions :-

    1. I was following the documentation to install it, didn't see any particular Python version. I think we have some Python version on which this installation is very smooth and no such errors are encountered.
    2. We can update the documentation and mention the Python version on which nebullvm and tvm installs smoothly. 3. Please correct me if I am installing nebullvm incorrectly.
    3. I will create environment with Python 3.7 and try to setup nebullvm.
    opened by SahilChachra 10
  • [Bug] Metadata.read with FileNotFoundError

    [Bug] Metadata.read with FileNotFoundError

    According to #18, it will throw FileNotFoundError:

    >>> model_dir = 'g8q7g2i3pPk7QVwkr6zT1lV6x8mo3nUWOOQvT6oI'
>>> LearnerMetadata.read(model_dir).load_model(model_dir)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/chenshiyu/workspace/git/shiyu22/nebullvm/nebullvm/inference_learners/base.py", line 314, in read
    with open(path / cls.NAME, "r") as fin:
FileNotFoundError: [Errno 2] No such file or directory: 'g8q7g2i3pPk7QVwkr6zT1lV6x8mo3nUWOOQvT6oI/metadata.json'

    I think in the latest version, should run this loaded_model = LearnerMetadata.read(path + '/optimized_model').load_model(path), and I have submitted PR #83 with load() to fixed it, so we can load the optimized model:

    from nebullvm.inference_learners.base import LearnerMetadata

path = "path-to-directory-where-the-model-is-saved"
loaded_model = LearnerMetadata.load(path).load_model(path)
    opened by shiyu22 8
  • OpenVINO Compiler Installation in Ubuntu 22.04

    OpenVINO Compiler Installation in Ubuntu 22.04 

    (venv) isgursoy@isgursoy-VORKE:/opt/cloud/projects/akil/classification-engine$ python -m nebullvm.installers.auto_installer --frameworks torch onnx huggingface --compilers all
.
.
.
21/11/2022 06:29:10 PM [ INFO ] Trying to install openvino on the platform...
21/11/2022 06:29:12 PM [ WARNING ] Unable to install openvino on this platform. The compiler will be skipped.

    No other relevant messages.

    opened by isgursoy 5
  • meansure running time with median latency

    meansure running time with median latency

    Hi, I notice that nebullvm sometimes runs very slow on large models, and found that nebullvm currently uses mean latency to measure the running time of the optimized model in compute_optimized_running_time( optimized_model: BaseInferenceLearner, steps: int = 100). Could it be better if using median latency? Also, we could use an adaptive algorithm to reduce total running steps?

    opened by reiase 5
  • Why the time consumming is increased after using nebullvm in the GPT2 jupyter script?

    Why the time consumming is increased after using nebullvm in the GPT2 jupyter script?

    When I run the GPT2 jupyter script in https://github.com/nebuly-ai/nebullvm/blob/main/resources/notebooks/Accelerate-Hugging-Face-GPT2-and-BERT-with-nebullvm.ipynb, I have got the output below:

    Hello, I'm username! I've tested nebullvm on the following setup: Hardware: Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz CPU and Tesla V100-SXM2-32GB GPU. Model: GPT2 - HuggingFace Tokens: 10

    • Vanilla performance: 25.12ms
    • Optimized performance: 35.6ms
    • Speedup: 0.7x Tokens: 181
    • Vanilla performance: 73.49ms
    • Optimized performance: 71.9ms
    • Speedup: 1.0x

    This is my environment: CUDA Version: 11.2 Python 3.7.5 numpy 1.19.0 torch 1.11.0 onnxruntime 1.11.1 nvidia-tensorrt 8.4.0.6 GPU: Tesla V100

    This is all the information in the console after runing the scrpit:

    runfile('/data/nlp/li.ma/CPM-Generate-Pytorch/nebullvm_GPT_demo.py', wdir='/data/nlp/li.ma/CPM-Generate-Pytorch') Average response time for GPT2: (10 tokens): 25.121240615844727 ms Average response time for GPT2: (181 tokens): 73.49021196365356 ms RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd ImportError: numpy.core.multiarray failed to import The above exception was the direct cause of the following exception: SystemError: returned a result with an error set /usr/local/lib/python3.7/dist-packages/defusedxml/init.py:30: DeprecationWarning: defusedxml.cElementTree is deprecated, import from defusedxml.ElementTree instead. from . import cElementTree Using pad_token, but it is not set yet. Ignore MatMul due to non constant B: /[MatMul_113] Ignore MatMul due to non constant B: /[MatMul_138] Ignore MatMul due to non constant B: /[MatMul_311] Ignore MatMul due to non constant B: /[MatMul_336] Ignore MatMul due to non constant B: /[MatMul_509] Ignore MatMul due to non constant B: /[MatMul_534] Ignore MatMul due to non constant B: /[MatMul_707] Ignore MatMul due to non constant B: /[MatMul_732] Ignore MatMul due to non constant B: /[MatMul_905] Ignore MatMul due to non constant B: /[MatMul_930] Ignore MatMul due to non constant B: /[MatMul_1103] Ignore MatMul due to non constant B: /[MatMul_1128] Ignore MatMul due to non constant B: /[MatMul_1301] Ignore MatMul due to non constant B: /[MatMul_1326] Ignore MatMul due to non constant B: /[MatMul_1499] Ignore MatMul due to non constant B: /[MatMul_1524] Ignore MatMul due to non constant B: /[MatMul_1697] Ignore MatMul due to non constant B: /[MatMul_1722] Ignore MatMul due to non constant B: /[MatMul_1895] Ignore MatMul due to non constant B: /[MatMul_1920] Ignore MatMul due to non constant B: /[MatMul_2093] Ignore MatMul due to non constant B: /[MatMul_2118] Ignore MatMul due to non constant B: /[MatMul_2291] Ignore MatMul due to non constant B: /[MatMul_2316] 2022-05-24 11:16:42.711719365 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1429_ReduceMax 2022-05-24 11:16:42.711826854 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1429_ReduceMin 2022-05-24 11:16:42.725020430 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.43_ReduceMax 2022-05-24 11:16:42.725076130 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.43_ReduceMin 2022-05-24 11:16:42.725096945 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2018_ReduceMax 2022-05-24 11:16:42.725118940 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2018_ReduceMin 2022-05-24 11:16:42.725139125 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_394_ReduceMax 2022-05-24 11:16:42.725160207 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_394_ReduceMin 2022-05-24 11:16:42.725179652 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2454_ReduceMax 2022-05-24 11:16:42.725199205 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2454_ReduceMin 2022-05-24 11:16:42.725221431 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_833_ReduceMax 2022-05-24 11:16:42.725240934 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_833_ReduceMin 2022-05-24 11:16:42.725580809 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3214_ReduceMax 2022-05-24 11:16:42.725602648 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3214_ReduceMin 2022-05-24 11:16:42.725619617 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2293_ReduceMax 2022-05-24 11:16:42.725672329 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2293_ReduceMin 2022-05-24 11:16:42.725691544 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2455_ReduceMax 2022-05-24 11:16:42.725709226 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2455_ReduceMin 2022-05-24 11:16:42.725730943 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2417_ReduceMax 2022-05-24 11:16:42.725749796 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2417_ReduceMin 2022-05-24 11:16:42.725767675 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1745_ReduceMax 2022-05-24 11:16:42.725794756 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1745_ReduceMin 2022-05-24 11:16:42.726284695 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3216_ReduceMax 2022-05-24 11:16:42.726634945 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3216_ReduceMin 2022-05-24 11:16:42.726675637 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.39_ReduceMax 2022-05-24 11:16:42.726710786 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.39_ReduceMin 2022-05-24 11:16:42.728635870 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3259_ReduceMax 2022-05-24 11:16:42.728695197 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3259_ReduceMin 2022-05-24 11:16:42.728741807 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.119_ReduceMax 2022-05-24 11:16:42.728776025 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.119_ReduceMin 2022-05-24 11:16:42.728820008 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_411_ReduceMax 2022-05-24 11:16:42.728848003 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_411_ReduceMin 2022-05-24 11:16:42.728909427 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1216_ReduceMax 2022-05-24 11:16:42.728993414 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1216_ReduceMin 2022-05-24 11:16:42.729034381 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3219_ReduceMax 2022-05-24 11:16:42.729065276 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3219_ReduceMin 2022-05-24 11:16:42.729102523 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_998_ReduceMax 2022-05-24 11:16:42.729142896 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_998_ReduceMin 2022-05-24 11:16:42.729172489 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_745_ReduceMax 2022-05-24 11:16:42.729206179 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_745_ReduceMin 2022-05-24 11:16:42.729238932 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2453_ReduceMax 2022-05-24 11:16:42.729270579 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2453_ReduceMin 2022-05-24 11:16:42.729311674 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.191_ReduceMax 2022-05-24 11:16:42.729352025 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.191_ReduceMin 2022-05-24 11:16:42.729385139 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_453_ReduceMax 2022-05-24 11:16:42.729434303 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_453_ReduceMin 2022-05-24 11:16:42.729473471 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2022_ReduceMax 2022-05-24 11:16:42.729502892 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2022_ReduceMin 2022-05-24 11:16:42.729532305 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1560_ReduceMax 2022-05-24 11:16:42.729565870 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1560_ReduceMin 2022-05-24 11:16:42.729595267 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3001_ReduceMax 2022-05-24 11:16:42.729643676 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3001_ReduceMin 2022-05-24 11:16:42.729695220 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.75_ReduceMax 2022-05-24 11:16:42.729729601 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.75_ReduceMin 2022-05-24 11:16:42.729764271 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3253_ReduceMax 2022-05-24 11:16:42.729801913 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3253_ReduceMin 2022-05-24 11:16:42.729843200 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3256_ReduceMax 2022-05-24 11:16:42.729886452 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3256_ReduceMin 2022-05-24 11:16:42.729930466 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.71_ReduceMax 2022-05-24 11:16:42.729959024 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.71_ReduceMin 2022-05-24 11:16:42.729989811 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.87_ReduceMax 2022-05-24 11:16:42.730021286 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.87_ReduceMin 2022-05-24 11:16:42.730051087 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2196_ReduceMax 2022-05-24 11:16:42.730078110 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2196_ReduceMin 2022-05-24 11:16:42.730130610 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2548_ReduceMax 2022-05-24 11:16:42.730171916 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2548_ReduceMin 2022-05-24 11:16:42.730202144 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2532_ReduceMax 2022-05-24 11:16:42.730233303 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2532_ReduceMin 2022-05-24 11:16:42.730268977 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_491_ReduceMax 2022-05-24 11:16:42.730299408 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_491_ReduceMin 2022-05-24 11:16:42.730331042 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.167_ReduceMax 2022-05-24 11:16:42.730362034 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.167_ReduceMin 2022-05-24 11:16:42.730396715 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.139_ReduceMax 2022-05-24 11:16:42.730428105 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.139_ReduceMin 2022-05-24 11:16:42.730461962 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1924_ReduceMax 2022-05-24 11:16:42.730493382 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1924_ReduceMin 2022-05-24 11:16:42.730530299 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1142_ReduceMax 2022-05-24 11:16:42.730561732 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1142_ReduceMin 2022-05-24 11:16:42.731305425 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3296_ReduceMax 2022-05-24 11:16:42.731351437 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3296_ReduceMin 2022-05-24 11:16:42.731385630 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1180_ReduceMax 2022-05-24 11:16:42.731417700 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1180_ReduceMin 2022-05-24 11:16:42.731447254 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2785_ReduceMax 2022-05-24 11:16:42.731478607 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2785_ReduceMin 2022-05-24 11:16:42.731513880 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1255_ReduceMax 2022-05-24 11:16:42.731545336 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1255_ReduceMin 2022-05-24 11:16:42.731578068 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3258_ReduceMax 2022-05-24 11:16:42.731615346 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3258_ReduceMin 2022-05-24 11:16:42.731657916 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.23_ReduceMax 2022-05-24 11:16:42.731689191 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.23_ReduceMin 2022-05-24 11:16:42.731718118 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2620_ReduceMax 2022-05-24 11:16:42.731750255 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2620_ReduceMin 2022-05-24 11:16:42.731801297 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.3_ReduceMax 2022-05-24 11:16:42.731835508 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.3_ReduceMin 2022-05-24 11:16:42.731867357 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_670_ReduceMax 2022-05-24 11:16:42.731909135 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_670_ReduceMin 2022-05-24 11:16:42.731943637 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_492_ReduceMax 2022-05-24 11:16:42.731974344 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_492_ReduceMin 2022-05-24 11:16:42.732004121 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.39_ReduceMax 2022-05-24 11:16:42.732038104 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.39_ReduceMin 2022-05-24 11:16:42.732085793 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1510_ReduceMax 2022-05-24 11:16:42.732138324 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1510_ReduceMin 2022-05-24 11:16:42.732173234 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_814_ReduceMax 2022-05-24 11:16:42.732210203 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_814_ReduceMin 2022-05-24 11:16:42.732260671 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.19_ReduceMax 2022-05-24 11:16:42.732294998 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.19_ReduceMin 2022-05-24 11:16:42.732328105 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.23_ReduceMax 2022-05-24 11:16:42.732356049 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.23_ReduceMin 2022-05-24 11:16:42.732394123 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.23_ReduceMax 2022-05-24 11:16:42.732428118 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.23_ReduceMin 2022-05-24 11:16:42.732457833 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1088_ReduceMax 2022-05-24 11:16:42.732490766 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1088_ReduceMin 2022-05-24 11:16:42.732519858 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3199_ReduceMax 2022-05-24 11:16:42.732550731 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3199_ReduceMin 2022-05-24 11:16:42.732580471 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1002_ReduceMax 2022-05-24 11:16:42.732610079 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1002_ReduceMin 2022-05-24 11:16:42.732638359 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_432_ReduceMax 2022-05-24 11:16:42.732682523 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_432_ReduceMin 2022-05-24 11:16:42.732720310 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1981_ReduceMax 2022-05-24 11:16:42.732763698 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1981_ReduceMin 2022-05-24 11:16:42.732819275 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.39_ReduceMax 2022-05-24 11:16:42.732856522 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.39_ReduceMin 2022-05-24 11:16:42.732886188 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_3162_ReduceMax 2022-05-24 11:16:42.732913684 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_3162_ReduceMin 2022-05-24 11:16:42.732949775 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_490_ReduceMax 2022-05-24 11:16:42.732980844 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_490_ReduceMin 2022-05-24 11:16:42.733013047 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1720_ReduceMax 2022-05-24 11:16:42.733058880 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1720_ReduceMin 2022-05-24 11:16:42.733099172 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2494_ReduceMax 2022-05-24 11:16:42.733153138 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2494_ReduceMin 2022-05-24 11:16:42.733189719 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2233_ReduceMax 2022-05-24 11:16:42.733237123 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2233_ReduceMin 2022-05-24 11:16:42.733296009 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2483_ReduceMax 2022-05-24 11:16:42.733344885 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2483_ReduceMin 2022-05-24 11:16:42.733386667 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1174_ReduceMax 2022-05-24 11:16:42.733413581 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1174_ReduceMin 2022-05-24 11:16:42.733443021 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2471_ReduceMax 2022-05-24 11:16:42.733471936 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2471_ReduceMin 2022-05-24 11:16:42.733514602 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2451_ReduceMax 2022-05-24 11:16:42.733541155 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2451_ReduceMin 2022-05-24 11:16:42.733571666 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1978_ReduceMax 2022-05-24 11:16:42.733621565 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1978_ReduceMin 2022-05-24 11:16:42.733661554 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1538_ReduceMax 2022-05-24 11:16:42.733702014 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1538_ReduceMin 2022-05-24 11:16:42.733736660 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_709_ReduceMax 2022-05-24 11:16:42.733786020 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_709_ReduceMin 2022-05-24 11:16:42.733822075 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.235_ReduceMax 2022-05-24 11:16:42.733854267 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.235_ReduceMin 2022-05-24 11:16:42.733899838 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.227_ReduceMax 2022-05-24 11:16:42.733931280 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.227_ReduceMin 2022-05-24 11:16:42.733966369 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2485_ReduceMax 2022-05-24 11:16:42.734015804 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2485_ReduceMin 2022-05-24 11:16:42.734063435 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2235_ReduceMax 2022-05-24 11:16:42.734123940 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2235_ReduceMin 2022-05-24 11:16:42.734166677 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2709_ReduceMax 2022-05-24 11:16:42.734200459 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2709_ReduceMin 2022-05-24 11:16:42.734247417 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2766_ReduceMax 2022-05-24 11:16:42.734279950 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2766_ReduceMin 2022-05-24 11:16:42.734309105 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_981_ReduceMax 2022-05-24 11:16:42.734341544 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_981_ReduceMin 2022-05-24 11:16:42.734370958 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_835_ReduceMax 2022-05-24 11:16:42.734402362 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_835_ReduceMin 2022-05-24 11:16:42.734459236 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.55_ReduceMax 2022-05-24 11:16:42.734480882 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.55_ReduceMin 2022-05-24 11:16:42.734502519 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1669_ReduceMax 2022-05-24 11:16:42.734525424 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1669_ReduceMin 2022-05-24 11:16:42.734547851 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3042_ReduceMax 2022-05-24 11:16:42.734567650 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3042_ReduceMin 2022-05-24 11:16:42.734587036 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.7_ReduceMax 2022-05-24 11:16:42.734609837 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.7_ReduceMin 2022-05-24 11:16:42.734629078 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_451_ReduceMax 2022-05-24 11:16:42.734672365 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_451_ReduceMin 2022-05-24 11:16:42.734700323 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2804_ReduceMax 2022-05-24 11:16:42.734732758 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2804_ReduceMin 2022-05-24 11:16:42.734755826 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_960_ReduceMax 2022-05-24 11:16:42.734791013 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_960_ReduceMin 2022-05-24 11:16:42.734818848 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2599_ReduceMax 2022-05-24 11:16:42.734842563 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2599_ReduceMin 2022-05-24 11:16:42.734861880 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.27_ReduceMax 2022-05-24 11:16:42.734885888 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.27_ReduceMin 2022-05-24 11:16:42.734907710 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_3068_ReduceMax 2022-05-24 11:16:42.734937195 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_3068_ReduceMin 2022-05-24 11:16:42.734974712 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2162_ReduceMax 2022-05-24 11:16:42.734995801 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2162_ReduceMin 2022-05-24 11:16:42.735015284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2493_ReduceMax 2022-05-24 11:16:42.735053333 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2493_ReduceMin 2022-05-24 11:16:42.735094956 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2907_ReduceMax 2022-05-24 11:16:42.735156540 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2907_ReduceMin 2022-05-24 11:16:42.735183309 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2689_ReduceMax 2022-05-24 11:16:42.735207268 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2689_ReduceMin 2022-05-24 11:16:42.735227793 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_285_ReduceMax 2022-05-24 11:16:42.735253442 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_285_ReduceMin 2022-05-24 11:16:42.735278405 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2491_ReduceMax 2022-05-24 11:16:42.735313214 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2491_ReduceMin 2022-05-24 11:16:42.735346271 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_233_ReduceMax 2022-05-24 11:16:42.735363954 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_233_ReduceMin 2022-05-24 11:16:42.735383516 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3252_ReduceMax 2022-05-24 11:16:42.735410125 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3252_ReduceMin 2022-05-24 11:16:42.735439901 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_3255_ReduceMax 2022-05-24 11:16:42.735468893 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_3255_ReduceMin 2022-05-24 11:16:42.735497612 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.31_ReduceMax 2022-05-24 11:16:42.735516148 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.31_ReduceMin 2022-05-24 11:16:42.735537066 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1632_ReduceMax 2022-05-24 11:16:42.735555775 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1632_ReduceMin 2022-05-24 11:16:42.735582690 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1793_ReduceMax 2022-05-24 11:16:42.735616665 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1793_ReduceMin 2022-05-24 11:16:42.735639297 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3058_ReduceMax 2022-05-24 11:16:42.735661789 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3058_ReduceMin 2022-05-24 11:16:42.735686207 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1962_ReduceMax 2022-05-24 11:16:42.735732475 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1962_ReduceMin 2022-05-24 11:16:42.735757944 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.7_ReduceMax 2022-05-24 11:16:42.735792785 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.7_ReduceMin 2022-05-24 11:16:42.735816730 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2142_ReduceMax 2022-05-24 11:16:42.735835090 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2142_ReduceMin 2022-05-24 11:16:42.735856375 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2020_ReduceMax 2022-05-24 11:16:42.735878346 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2020_ReduceMin 2022-05-24 11:16:42.735902347 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.3_ReduceMax 2022-05-24 11:16:42.735924822 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.3_ReduceMin 2022-05-24 11:16:42.735947062 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.203_ReduceMax 2022-05-24 11:16:42.735970553 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.203_ReduceMin 2022-05-24 11:16:42.735994802 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1091_ReduceMax 2022-05-24 11:16:42.736013314 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1091_ReduceMin 2022-05-24 11:16:42.736034520 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2344_ReduceMax 2022-05-24 11:16:42.736057169 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2344_ReduceMin 2022-05-24 11:16:42.736076469 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.11_ReduceMax 2022-05-24 11:16:42.736095599 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.11_ReduceMin 2022-05-24 11:16:42.736130401 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2982_ReduceMax 2022-05-24 11:16:42.736168797 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2982_ReduceMin 2022-05-24 11:16:42.736195193 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.43_ReduceMax 2022-05-24 11:16:42.736218005 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.43_ReduceMin 2022-05-24 11:16:42.736240047 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1668_ReduceMax 2022-05-24 11:16:42.736260507 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1668_ReduceMin 2022-05-24 11:16:42.736282170 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.135_ReduceMax 2022-05-24 11:16:42.736304597 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.135_ReduceMin 2022-05-24 11:16:42.736332586 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1688_ReduceMax 2022-05-24 11:16:42.736355768 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1688_ReduceMin 2022-05-24 11:16:42.736378646 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_454_ReduceMax 2022-05-24 11:16:42.736414528 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_454_ReduceMin 2022-05-24 11:16:42.736438704 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_666_ReduceMax 2022-05-24 11:16:42.736456714 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_666_ReduceMin 2022-05-24 11:16:42.736477681 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1907_ReduceMax 2022-05-24 11:16:42.736497142 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1907_ReduceMin 2022-05-24 11:16:42.736516157 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1690_ReduceMax 2022-05-24 11:16:42.736536500 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1690_ReduceMin 2022-05-24 11:16:42.736555015 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_235_ReduceMax 2022-05-24 11:16:42.736576280 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_235_ReduceMin 2022-05-24 11:16:42.736596935 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3040_ReduceMax 2022-05-24 11:16:42.736620049 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3040_ReduceMin 2022-05-24 11:16:42.736649954 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2995_ReduceMax 2022-05-24 11:16:42.736687072 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2995_ReduceMin 2022-05-24 11:16:42.736715410 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2959_ReduceMax 2022-05-24 11:16:42.736733369 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2959_ReduceMin 2022-05-24 11:16:42.736762056 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.175_ReduceMax 2022-05-24 11:16:42.736786698 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.175_ReduceMin 2022-05-24 11:16:42.736805838 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2228_ReduceMax 2022-05-24 11:16:42.736846494 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2228_ReduceMin 2022-05-24 11:16:42.736876642 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2787_ReduceMax 2022-05-24 11:16:42.736896308 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2787_ReduceMin 2022-05-24 11:16:42.736916155 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.47_ReduceMax 2022-05-24 11:16:42.736939465 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.47_ReduceMin 2022-05-24 11:16:42.736959869 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1090_ReduceMax 2022-05-24 11:16:42.736982499 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1090_ReduceMin 2022-05-24 11:16:42.737004433 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2490_ReduceMax 2022-05-24 11:16:42.737040298 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2490_ReduceMin 2022-05-24 11:16:42.737073821 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.87_ReduceMax 2022-05-24 11:16:42.737095681 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.87_ReduceMin 2022-05-24 11:16:42.737126869 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_617_ReduceMax 2022-05-24 11:16:42.737150115 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_617_ReduceMin 2022-05-24 11:16:42.737176622 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.99_ReduceMax 2022-05-24 11:16:42.737199711 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.99_ReduceMin 2022-05-24 11:16:42.737236801 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1471_ReduceMax 2022-05-24 11:16:42.737273876 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1471_ReduceMin 2022-05-24 11:16:42.737300961 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_3000_ReduceMax 2022-05-24 11:16:42.737344727 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_3000_ReduceMin 2022-05-24 11:16:42.737376817 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.67_ReduceMax 2022-05-24 11:16:42.737399937 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.67_ReduceMin 2022-05-24 11:16:42.737423008 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query_ReduceMax 2022-05-24 11:16:42.737446255 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query_ReduceMin 2022-05-24 11:16:42.737468321 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1598_ReduceMax 2022-05-24 11:16:42.737490663 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1598_ReduceMin 2022-05-24 11:16:42.737514783 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.51_ReduceMax 2022-05-24 11:16:42.737537441 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.51_ReduceMin 2022-05-24 11:16:42.737561672 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.195_ReduceMax 2022-05-24 11:16:42.737584087 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.195_ReduceMin 2022-05-24 11:16:42.737608069 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_443_ReduceMax 2022-05-24 11:16:42.737646241 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_443_ReduceMin 2022-05-24 11:16:42.737672807 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.115_ReduceMax 2022-05-24 11:16:42.737694825 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.115_ReduceMin 2022-05-24 11:16:42.737717349 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.47_ReduceMax 2022-05-24 11:16:42.737741628 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.47_ReduceMin 2022-05-24 11:16:42.737772452 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1723_ReduceMax 2022-05-24 11:16:42.737808063 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1723_ReduceMin 2022-05-24 11:16:42.737839631 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2688_ReduceMax 2022-05-24 11:16:42.737859053 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2688_ReduceMin 2022-05-24 11:16:42.737877871 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2781_ReduceMax 2022-05-24 11:16:42.737895545 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2781_ReduceMin 2022-05-24 11:16:42.737929433 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.11_ReduceMax 2022-05-24 11:16:42.737953313 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.11_ReduceMin 2022-05-24 11:16:42.737975388 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2048_ReduceMax 2022-05-24 11:16:42.738004667 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2048_ReduceMin 2022-05-24 11:16:42.738027589 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.155_ReduceMax 2022-05-24 11:16:42.738049654 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.155_ReduceMin 2022-05-24 11:16:42.738069770 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1746_ReduceMax 2022-05-24 11:16:42.738092968 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1746_ReduceMin 2022-05-24 11:16:42.738117931 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2961_ReduceMax 2022-05-24 11:16:42.738136623 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2961_ReduceMin 2022-05-24 11:16:42.738165112 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_648_ReduceMax 2022-05-24 11:16:42.738192709 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_648_ReduceMin 2022-05-24 11:16:42.738212788 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.127_ReduceMax 2022-05-24 11:16:42.738237383 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.127_ReduceMin 2022-05-24 11:16:42.738260510 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_904_ReduceMax 2022-05-24 11:16:42.738283019 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_904_ReduceMin 2022-05-24 11:16:42.738306709 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1256_ReduceMax 2022-05-24 11:16:42.738329506 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1256_ReduceMin 2022-05-24 11:16:42.738352924 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1761_ReduceMax 2022-05-24 11:16:42.738371308 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1761_ReduceMin 2022-05-24 11:16:42.738391334 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2997_ReduceMax 2022-05-24 11:16:42.738427089 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2997_ReduceMin 2022-05-24 11:16:42.738457022 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.55_ReduceMax 2022-05-24 11:16:42.738486213 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.55_ReduceMin 2022-05-24 11:16:42.738506485 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.63_ReduceMax 2022-05-24 11:16:42.738530437 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.63_ReduceMin 2022-05-24 11:16:42.738553386 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1601_ReduceMax 2022-05-24 11:16:42.738571973 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1601_ReduceMin 2022-05-24 11:16:42.738592897 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2835_ReduceMax 2022-05-24 11:16:42.738615855 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2835_ReduceMin 2022-05-24 11:16:42.738636476 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.7_ReduceMax 2022-05-24 11:16:42.738661911 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.7_ReduceMin 2022-05-24 11:16:42.738684264 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2255_ReduceMax 2022-05-24 11:16:42.738715091 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2255_ReduceMin 2022-05-24 11:16:42.738739710 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2746_ReduceMax 2022-05-24 11:16:42.738773661 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2746_ReduceMin 2022-05-24 11:16:42.738802317 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.55_ReduceMax 2022-05-24 11:16:42.738822238 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.55_ReduceMin 2022-05-24 11:16:42.738844181 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_632_ReduceMax 2022-05-24 11:16:42.738863880 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_632_ReduceMin 2022-05-24 11:16:42.738886010 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.95_ReduceMax 2022-05-24 11:16:42.738907743 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.95_ReduceMin 2022-05-24 11:16:42.738928151 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_964_ReduceMax 2022-05-24 11:16:42.738962459 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_964_ReduceMin 2022-05-24 11:16:42.738990837 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1208_ReduceMax 2022-05-24 11:16:42.739026657 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1208_ReduceMin 2022-05-24 11:16:42.739066014 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2001_ReduceMax 2022-05-24 11:16:42.739089282 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2001_ReduceMin 2022-05-24 11:16:42.739117640 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.15_ReduceMax 2022-05-24 11:16:42.739141676 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.15_ReduceMin 2022-05-24 11:16:42.739164820 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3250_ReduceMax 2022-05-24 11:16:42.739192134 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3250_ReduceMin 2022-05-24 11:16:42.739229474 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1686_ReduceMax 2022-05-24 11:16:42.739247259 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1686_ReduceMin 2022-05-24 11:16:42.739269198 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.67_ReduceMax 2022-05-24 11:16:42.739292458 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.67_ReduceMin 2022-05-24 11:16:42.739318096 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_664_ReduceMax 2022-05-24 11:16:42.739336071 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_664_ReduceMin 2022-05-24 11:16:42.739355558 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_509_ReduceMax 2022-05-24 11:16:42.739389184 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_509_ReduceMin 2022-05-24 11:16:42.739412668 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3220_ReduceMax 2022-05-24 11:16:42.739432395 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3220_ReduceMin 2022-05-24 11:16:42.739451457 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2743_ReduceMax 2022-05-24 11:16:42.739486261 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2743_ReduceMin 2022-05-24 11:16:42.739520224 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_518_ReduceMax 2022-05-24 11:16:42.739543328 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_518_ReduceMin 2022-05-24 11:16:42.739574970 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.151_ReduceMax 2022-05-24 11:16:42.739594137 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.151_ReduceMin 2022-05-24 11:16:42.739614921 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_415_ReduceMax 2022-05-24 11:16:42.739636382 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_415_ReduceMin 2022-05-24 11:16:42.739656258 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2657_ReduceMax 2022-05-24 11:16:42.739675289 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2657_ReduceMin 2022-05-24 11:16:42.739695467 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.27_ReduceMax 2022-05-24 11:16:42.739717064 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.27_ReduceMin 2022-05-24 11:16:42.739738975 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.39_ReduceMax 2022-05-24 11:16:42.739761286 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.39_ReduceMin 2022-05-24 11:16:42.739783022 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1707_ReduceMax 2022-05-24 11:16:42.739820577 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1707_ReduceMin 2022-05-24 11:16:42.739844970 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1722_ReduceMax 2022-05-24 11:16:42.739878866 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1722_ReduceMin 2022-05-24 11:16:42.739919482 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1491_ReduceMax 2022-05-24 11:16:42.739943577 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1491_ReduceMin 2022-05-24 11:16:42.739964099 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1856_ReduceMax 2022-05-24 11:16:42.739983039 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1856_ReduceMin 2022-05-24 11:16:42.740010229 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.59_ReduceMax 2022-05-24 11:16:42.740034108 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.59_ReduceMin 2022-05-24 11:16:42.740058471 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_764_ReduceMax 2022-05-24 11:16:42.740089836 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_764_ReduceMin 2022-05-24 11:16:42.740119054 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_686_ReduceMax 2022-05-24 11:16:42.740139555 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_686_ReduceMin 2022-05-24 11:16:42.740175104 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3248_ReduceMax 2022-05-24 11:16:42.740202041 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3248_ReduceMin 2022-05-24 11:16:42.740233393 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1122_ReduceMax 2022-05-24 11:16:42.740252492 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1122_ReduceMin 2022-05-24 11:16:42.740273195 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2217_ReduceMax 2022-05-24 11:16:42.740312188 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2217_ReduceMin 2022-05-24 11:16:42.740337005 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2238_ReduceMax 2022-05-24 11:16:42.740370565 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2238_ReduceMin 2022-05-24 11:16:42.740399182 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.31_ReduceMax 2022-05-24 11:16:42.740417626 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.31_ReduceMin 2022-05-24 11:16:42.740438942 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.207_ReduceMax 2022-05-24 11:16:42.740461344 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.207_ReduceMin 2022-05-24 11:16:42.740485282 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.143_ReduceMax 2022-05-24 11:16:42.740507686 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.143_ReduceMin 2022-05-24 11:16:42.740530350 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.3_ReduceMax 2022-05-24 11:16:42.740551462 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.3_ReduceMin 2022-05-24 11:16:42.740572237 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1018_ReduceMax 2022-05-24 11:16:42.740593200 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1018_ReduceMin 2022-05-24 11:16:42.740635648 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2397_ReduceMax 2022-05-24 11:16:42.740660546 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2397_ReduceMin 2022-05-24 11:16:42.740682046 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2472_ReduceMax 2022-05-24 11:16:42.740717659 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2472_ReduceMin 2022-05-24 11:16:42.740741637 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_508_ReduceMax 2022-05-24 11:16:42.740760678 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_508_ReduceMin 2022-05-24 11:16:42.740782342 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.23_ReduceMax 2022-05-24 11:16:42.740816215 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.23_ReduceMin 2022-05-24 11:16:42.740844005 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.11_ReduceMax 2022-05-24 11:16:42.740862821 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.11_ReduceMin 2022-05-24 11:16:42.740882453 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_708_ReduceMax 2022-05-24 11:16:42.740918383 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_708_ReduceMin 2022-05-24 11:16:42.740944617 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1251_ReduceMax 2022-05-24 11:16:42.740962447 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1251_ReduceMin 2022-05-24 11:16:42.740981266 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.11_ReduceMax 2022-05-24 11:16:42.741016291 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.11_ReduceMin 2022-05-24 11:16:42.741040256 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_702_ReduceMax 2022-05-24 11:16:42.741075521 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_702_ReduceMin 2022-05-24 11:16:42.741102644 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_581_ReduceMax 2022-05-24 11:16:42.741129135 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_581_ReduceMin 2022-05-24 11:16:42.741150426 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value_ReduceMax 2022-05-24 11:16:42.741174533 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value_ReduceMin 2022-05-24 11:16:42.741193843 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights_ReduceMax 2022-05-24 11:16:42.741212366 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights_ReduceMin 2022-05-24 11:16:42.741237838 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1853_ReduceMax 2022-05-24 11:16:42.741261477 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1853_ReduceMin 2022-05-24 11:16:42.741280345 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_414_ReduceMax 2022-05-24 11:16:42.741305330 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_414_ReduceMin 2022-05-24 11:16:42.741326256 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1765_ReduceMax 2022-05-24 11:16:42.741347748 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1765_ReduceMin 2022-05-24 11:16:42.741372443 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2111_ReduceMax 2022-05-24 11:16:42.741391156 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2111_ReduceMin 2022-05-24 11:16:42.741412387 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2511_ReduceMax 2022-05-24 11:16:42.741436200 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2511_ReduceMin 2022-05-24 11:16:42.741456031 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1343_ReduceMax 2022-05-24 11:16:42.741477296 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1343_ReduceMin 2022-05-24 11:16:42.741504067 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_961_ReduceMax 2022-05-24 11:16:42.741538580 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_961_ReduceMin 2022-05-24 11:16:42.741565764 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_488_ReduceMax 2022-05-24 11:16:42.741583350 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_488_ReduceMin 2022-05-24 11:16:42.741602430 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_362_ReduceMax 2022-05-24 11:16:42.741624823 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_362_ReduceMin 2022-05-24 11:16:42.741646441 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2434_ReduceMax 2022-05-24 11:16:42.741668492 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2434_ReduceMin 2022-05-24 11:16:42.741692493 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2927_ReduceMax 2022-05-24 11:16:42.741712851 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2927_ReduceMin 2022-05-24 11:16:42.741731927 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1377_ReduceMax 2022-05-24 11:16:42.741750623 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1377_ReduceMin 2022-05-24 11:16:42.741771409 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.11_ReduceMax 2022-05-24 11:16:42.741795219 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.11_ReduceMin 2022-05-24 11:16:42.741823995 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2944_ReduceMax 2022-05-24 11:16:42.741847582 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2944_ReduceMin 2022-05-24 11:16:42.741866967 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2786_ReduceMax 2022-05-24 11:16:42.741890153 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2786_ReduceMin 2022-05-24 11:16:42.741914694 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_471_ReduceMax 2022-05-24 11:16:42.741935991 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_471_ReduceMin 2022-05-24 11:16:42.741989752 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2740_ReduceMax 2022-05-24 11:16:42.742039874 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2740_ReduceMin 2022-05-24 11:16:42.742069997 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2365_ReduceMax 2022-05-24 11:16:42.742095649 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2365_ReduceMin 2022-05-24 11:16:42.742129345 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2200_ReduceMax 2022-05-24 11:16:42.742150713 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2200_ReduceMin 2022-05-24 11:16:42.742171468 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_669_ReduceMax 2022-05-24 11:16:42.742196919 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_669_ReduceMin 2022-05-24 11:16:42.742220066 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3109_ReduceMax 2022-05-24 11:16:42.742242750 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3109_ReduceMin 2022-05-24 11:16:42.742263759 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3038_ReduceMax 2022-05-24 11:16:42.742282373 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3038_ReduceMin 2022-05-24 11:16:42.742308722 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.31_ReduceMax 2022-05-24 11:16:42.742334195 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.31_ReduceMin 2022-05-24 11:16:42.742357167 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2039_ReduceMax 2022-05-24 11:16:42.742389351 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2039_ReduceMin 2022-05-24 11:16:42.742505446 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1019_ReduceMax 2022-05-24 11:16:42.742538653 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1019_ReduceMin 2022-05-24 11:16:42.742562356 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.7_ReduceMax 2022-05-24 11:16:42.742584636 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.7_ReduceMin 2022-05-24 11:16:42.742604393 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.79_ReduceMax 2022-05-24 11:16:42.742625704 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.79_ReduceMin 2022-05-24 11:16:42.742645331 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1763_ReduceMax 2022-05-24 11:16:42.742662433 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1763_ReduceMin 2022-05-24 11:16:42.742682248 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1435_ReduceMax 2022-05-24 11:16:42.742702773 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1435_ReduceMin 2022-05-24 11:16:42.742722086 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.103_ReduceMax 2022-05-24 11:16:42.742745229 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.103_ReduceMin 2022-05-24 11:16:42.742767137 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1511_ReduceMax 2022-05-24 11:16:42.742789125 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1511_ReduceMin 2022-05-24 11:16:42.742811450 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.163_ReduceMax 2022-05-24 11:16:42.742834980 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.163_ReduceMin 2022-05-24 11:16:42.742864021 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2038_ReduceMax 2022-05-24 11:16:42.742885721 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2038_ReduceMin 2022-05-24 11:16:42.742908855 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1197_ReduceMax 2022-05-24 11:16:42.742942000 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1197_ReduceMin 2022-05-24 11:16:42.742981002 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_445_ReduceMax 2022-05-24 11:16:42.743019196 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_445_ReduceMin 2022-05-24 11:16:42.743045636 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_649_ReduceMax 2022-05-24 11:16:42.743067470 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_649_ReduceMin 2022-05-24 11:16:42.743091345 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2621_ReduceMax 2022-05-24 11:16:42.743116937 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2621_ReduceMin 2022-05-24 11:16:42.743140897 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2558_ReduceMax 2022-05-24 11:16:42.743175076 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2558_ReduceMin 2022-05-24 11:16:42.743198392 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1235_ReduceMax 2022-05-24 11:16:42.743230752 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1235_ReduceMin 2022-05-24 11:16:42.743255525 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Mul_218_ReduceMax 2022-05-24 11:16:42.743273649 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Mul_218_ReduceMin 2022-05-24 11:16:42.743293420 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2108_ReduceMax 2022-05-24 11:16:42.743315401 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2108_ReduceMin 2022-05-24 11:16:42.743335000 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1512_ReduceMax 2022-05-24 11:16:42.743354802 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1512_ReduceMin 2022-05-24 11:16:42.743386905 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2876_ReduceMax 2022-05-24 11:16:42.743401626 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2876_ReduceMin 2022-05-24 11:16:42.743419835 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.171_ReduceMax 2022-05-24 11:16:42.743435412 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.171_ReduceMin 2022-05-24 11:16:42.743452946 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2216_ReduceMax 2022-05-24 11:16:42.743468685 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2216_ReduceMin 2022-05-24 11:16:42.743487175 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1467_ReduceMax 2022-05-24 11:16:42.743519689 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1467_ReduceMin 2022-05-24 11:16:42.743546605 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_612_ReduceMax 2022-05-24 11:16:42.743561501 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_612_ReduceMin 2022-05-24 11:16:42.743577581 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2256_ReduceMax 2022-05-24 11:16:42.743594544 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2256_ReduceMin 2022-05-24 11:16:42.743612795 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2021_ReduceMax 2022-05-24 11:16:42.743631423 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2021_ReduceMin 2022-05-24 11:16:42.743649739 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2854_ReduceMax 2022-05-24 11:16:42.743672274 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2854_ReduceMin 2022-05-24 11:16:42.743691681 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1689_ReduceMax 2022-05-24 11:16:42.743711584 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1689_ReduceMin 2022-05-24 11:16:42.743730298 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1834_ReduceMax 2022-05-24 11:16:42.743749176 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1834_ReduceMin 2022-05-24 11:16:42.743767994 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.87_ReduceMax 2022-05-24 11:16:42.743787478 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.87_ReduceMin 2022-05-24 11:16:42.743803596 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.231_ReduceMax 2022-05-24 11:16:42.743818592 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.231_ReduceMin 2022-05-24 11:16:42.743835382 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1980_ReduceMax 2022-05-24 11:16:42.743866449 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1980_ReduceMin 2022-05-24 11:16:42.743889291 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3218_ReduceMax 2022-05-24 11:16:42.743908077 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3218_ReduceMin 2022-05-24 11:16:42.743925905 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1766_ReduceMax 2022-05-24 11:16:42.743946575 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1766_ReduceMin 2022-05-24 11:16:42.743971603 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1213_ReduceMax 2022-05-24 11:16:42.744006051 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1213_ReduceMin 2022-05-24 11:16:42.744027818 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2749_ReduceMax 2022-05-24 11:16:42.744054247 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2749_ReduceMin 2022-05-24 11:16:42.744074842 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_3131_ReduceMax 2022-05-24 11:16:42.744089383 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_3131_ReduceMin 2022-05-24 11:16:42.744113719 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_304_ReduceMax 2022-05-24 11:16:42.744133502 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_304_ReduceMin 2022-05-24 11:16:42.744149626 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3293_ReduceMax 2022-05-24 11:16:42.744164582 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_3293_ReduceMin 2022-05-24 11:16:42.744179914 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1983_ReduceMax 2022-05-24 11:16:42.744214005 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1983_ReduceMin 2022-05-24 11:16:42.744235975 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.3_ReduceMax 2022-05-24 11:16:42.744266242 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.3_ReduceMin 2022-05-24 11:16:42.744285567 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1000_ReduceMax 2022-05-24 11:16:42.744306218 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1000_ReduceMin 2022-05-24 11:16:42.744326045 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.19_ReduceMax 2022-05-24 11:16:42.744342033 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.19_ReduceMin 2022-05-24 11:16:42.744362524 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.211_ReduceMax 2022-05-24 11:16:42.744378340 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.211_ReduceMin 2022-05-24 11:16:42.744400881 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.31_ReduceMax 2022-05-24 11:16:42.744421290 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.31_ReduceMin 2022-05-24 11:16:42.744441450 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2277_ReduceMax 2022-05-24 11:16:42.744457772 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2277_ReduceMin 2022-05-24 11:16:42.744479245 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1473_ReduceMax 2022-05-24 11:16:42.744511623 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1473_ReduceMin 2022-05-24 11:16:42.744535373 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2433_ReduceMax 2022-05-24 11:16:42.744551340 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2433_ReduceMin 2022-05-24 11:16:42.744567061 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2738_ReduceMax 2022-05-24 11:16:42.744598973 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2738_ReduceMin 2022-05-24 11:16:42.744621691 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_743_ReduceMax 2022-05-24 11:16:42.744635969 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_743_ReduceMin 2022-05-24 11:16:42.744651613 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2303_ReduceMax 2022-05-24 11:16:42.744681127 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2303_ReduceMin 2022-05-24 11:16:42.744699299 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_741_ReduceMax 2022-05-24 11:16:42.744714080 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_741_ReduceMin 2022-05-24 11:16:42.744729022 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1474_ReduceMax 2022-05-24 11:16:42.744756020 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1474_ReduceMin 2022-05-24 11:16:42.744775265 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1975_ReduceMax 2022-05-24 11:16:42.744805092 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1975_ReduceMin 2022-05-24 11:16:42.744827355 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1178_ReduceMax 2022-05-24 11:16:42.744846295 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1178_ReduceMin 2022-05-24 11:16:42.744864506 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.35_ReduceMax 2022-05-24 11:16:42.744884255 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.35_ReduceMin 2022-05-24 11:16:42.744905782 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_357_ReduceMax 2022-05-24 11:16:42.744920786 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_357_ReduceMin 2022-05-24 11:16:42.744936446 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1465_ReduceMax 2022-05-24 11:16:42.744969604 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1465_ReduceMin 2022-05-24 11:16:42.745000373 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_231_ReduceMax 2022-05-24 11:16:42.745014524 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_231_ReduceMin 2022-05-24 11:16:42.745029923 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_953_ReduceMax 2022-05-24 11:16:42.745064165 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_953_ReduceMin 2022-05-24 11:16:42.745085662 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.63_ReduceMax 2022-05-24 11:16:42.745104326 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.63_ReduceMin 2022-05-24 11:16:42.745135985 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.91_ReduceMax 2022-05-24 11:16:42.745157330 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.91_ReduceMin 2022-05-24 11:16:42.745177518 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1176_ReduceMax 2022-05-24 11:16:42.745216376 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1176_ReduceMin 2022-05-24 11:16:42.745233101 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2742_ReduceMax 2022-05-24 11:16:42.745269057 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2742_ReduceMin 2022-05-24 11:16:42.745292467 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2236_ReduceMax 2022-05-24 11:16:42.745326324 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2236_ReduceMin 2022-05-24 11:16:42.745348183 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1028_ReduceMax 2022-05-24 11:16:42.745376013 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1028_ReduceMin 2022-05-24 11:16:42.745397020 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1855_ReduceMax 2022-05-24 11:16:42.745416206 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1855_ReduceMin 2022-05-24 11:16:42.745435392 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_668_ReduceMax 2022-05-24 11:16:42.745453433 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_668_ReduceMin 2022-05-24 11:16:42.745471546 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1212_ReduceMax 2022-05-24 11:16:42.745503824 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1212_ReduceMin 2022-05-24 11:16:42.745525528 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.71_ReduceMax 2022-05-24 11:16:42.745543874 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.71_ReduceMin 2022-05-24 11:16:42.745561737 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.15_ReduceMax 2022-05-24 11:16:42.745595582 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.15_ReduceMin 2022-05-24 11:16:42.745620414 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_963_ReduceMax 2022-05-24 11:16:42.745656376 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_963_ReduceMin 2022-05-24 11:16:42.745678615 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1468_ReduceMax 2022-05-24 11:16:42.745714263 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1468_ReduceMin 2022-05-24 11:16:42.745737087 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2998_ReduceMax 2022-05-24 11:16:42.745766832 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2998_ReduceMin 2022-05-24 11:16:42.745820515 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2813_ReduceMax 2022-05-24 11:16:42.745847724 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_2813_ReduceMin 2022-05-24 11:16:42.745874736 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.7_ReduceMax 2022-05-24 11:16:42.745898455 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.7_ReduceMin 2022-05-24 11:16:42.745917206 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2402_ReduceMax 2022-05-24 11:16:42.745933109 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2402_ReduceMin 2022-05-24 11:16:42.745973151 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3291_ReduceMax 2022-05-24 11:16:42.745988795 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3291_ReduceMin 2022-05-24 11:16:42.746008238 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2179_ReduceMax 2022-05-24 11:16:42.746029592 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2179_ReduceMin 2022-05-24 11:16:42.746047099 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1652_ReduceMax 2022-05-24 11:16:42.746062823 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1652_ReduceMin 2022-05-24 11:16:42.746078861 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2528_ReduceMax 2022-05-24 11:16:42.746092961 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2528_ReduceMin 2022-05-24 11:16:42.746115609 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.27_ReduceMax 2022-05-24 11:16:42.746150685 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.27_ReduceMin 2022-05-24 11:16:42.746172401 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2232_ReduceMax 2022-05-24 11:16:42.746204270 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2232_ReduceMin 2022-05-24 11:16:42.746230199 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1324_ReduceMax 2022-05-24 11:16:42.746250818 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1324_ReduceMin 2022-05-24 11:16:42.746267495 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3036_ReduceMax 2022-05-24 11:16:42.746281804 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3036_ReduceMin 2022-05-24 11:16:42.746305848 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states_ReduceMax 2022-05-24 11:16:42.746340563 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states_ReduceMin 2022-05-24 11:16:42.746366195 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_941_ReduceMax 2022-05-24 11:16:42.746383353 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_941_ReduceMin 2022-05-24 11:16:42.746409661 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2993_ReduceMax 2022-05-24 11:16:42.746439015 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2993_ReduceMin 2022-05-24 11:16:42.746462179 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1728_ReduceMax 2022-05-24 11:16:42.746493452 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1728_ReduceMin 2022-05-24 11:16:42.746515650 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2981_ReduceMax 2022-05-24 11:16:42.746532240 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2981_ReduceMin 2022-05-24 11:16:42.746547199 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.43_ReduceMax 2022-05-24 11:16:42.746570694 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.43_ReduceMin 2022-05-24 11:16:42.746589845 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_705_ReduceMax 2022-05-24 11:16:42.746632913 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_705_ReduceMin 2022-05-24 11:16:42.746654397 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1726_ReduceMax 2022-05-24 11:16:42.746687524 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1726_ReduceMin 2022-05-24 11:16:42.746714491 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1815_ReduceMax 2022-05-24 11:16:42.746739105 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1815_ReduceMin 2022-05-24 11:16:42.746757861 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.7_ReduceMax 2022-05-24 11:16:42.746778259 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.7_ReduceMin 2022-05-24 11:16:42.746794128 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2275_ReduceMax 2022-05-24 11:16:42.746814501 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2275_ReduceMin 2022-05-24 11:16:42.746837352 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2549_ReduceMax 2022-05-24 11:16:42.746864088 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2549_ReduceMin 2022-05-24 11:16:42.746885635 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1470_ReduceMax 2022-05-24 11:16:42.746918364 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1470_ReduceMin 2022-05-24 11:16:42.746941009 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1600_ReduceMax 2022-05-24 11:16:42.746965418 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1600_ReduceMin 2022-05-24 11:16:42.746986408 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.23_ReduceMax 2022-05-24 11:16:42.747006075 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.23_ReduceMin 2022-05-24 11:16:42.747023995 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3021_ReduceMax 2022-05-24 11:16:42.747042797 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3021_ReduceMin 2022-05-24 11:16:42.747058351 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_747_ReduceMax 2022-05-24 11:16:42.747075178 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_747_ReduceMin 2022-05-24 11:16:42.747091240 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3128_ReduceMax 2022-05-24 11:16:42.747116692 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3128_ReduceMin 2022-05-24 11:16:42.747135118 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2510_ReduceMax 2022-05-24 11:16:42.747158158 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2510_ReduceMin 2022-05-24 11:16:42.747180522 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1706_ReduceMax 2022-05-24 11:16:42.747196697 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1706_ReduceMin 2022-05-24 11:16:42.747217626 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1196_ReduceMax 2022-05-24 11:16:42.747234316 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1196_ReduceMin 2022-05-24 11:16:42.747252937 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.71_ReduceMax 2022-05-24 11:16:42.747271364 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.71_ReduceMin 2022-05-24 11:16:42.747286896 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2325_ReduceMax 2022-05-24 11:16:42.747308732 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2325_ReduceMin 2022-05-24 11:16:42.747324447 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.183_ReduceMax 2022-05-24 11:16:42.747347859 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.183_ReduceMin 2022-05-24 11:16:42.747366144 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2943_ReduceMax 2022-05-24 11:16:42.747381688 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2943_ReduceMin 2022-05-24 11:16:42.747396490 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3041_ReduceMax 2022-05-24 11:16:42.747414564 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_3041_ReduceMin 2022-05-24 11:16:42.747433802 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_236_ReduceMax 2022-05-24 11:16:42.747454580 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_236_ReduceMin 2022-05-24 11:16:42.747471277 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_540_ReduceMax 2022-05-24 11:16:42.747491480 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_540_ReduceMin 2022-05-24 11:16:42.747517122 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.27_ReduceMax 2022-05-24 11:16:42.747536052 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.27_ReduceMin 2022-05-24 11:16:42.747553037 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.47_ReduceMax 2022-05-24 11:16:42.747572391 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.47_ReduceMin 2022-05-24 11:16:42.747588870 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_706_ReduceMax 2022-05-24 11:16:42.747618607 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_706_ReduceMin 2022-05-24 11:16:42.747661039 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3198_ReduceMax 2022-05-24 11:16:42.747677299 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3198_ReduceMin 2022-05-24 11:16:42.747693425 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.63_ReduceMax 2022-05-24 11:16:42.747711431 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.63_ReduceMin 2022-05-24 11:16:42.747726640 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1783_ReduceMax 2022-05-24 11:16:42.747746117 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1783_ReduceMin 2022-05-24 11:16:42.747761491 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2710_ReduceMax 2022-05-24 11:16:42.747777079 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2710_ReduceMin 2022-05-24 11:16:42.747792231 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1273_ReduceMax 2022-05-24 11:16:42.747808752 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1273_ReduceMin 2022-05-24 11:16:42.747824257 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2726_ReduceMax 2022-05-24 11:16:42.747839178 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2726_ReduceMin 2022-05-24 11:16:42.747857596 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2230_ReduceMax 2022-05-24 11:16:42.747890834 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2230_ReduceMin 2022-05-24 11:16:42.747912369 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1973_ReduceMax 2022-05-24 11:16:42.747943800 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1973_ReduceMin 2022-05-24 11:16:42.747966487 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.123_ReduceMax 2022-05-24 11:16:42.747984538 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.123_ReduceMin 2022-05-24 11:16:42.748002949 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1215_ReduceMax 2022-05-24 11:16:42.748036794 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1215_ReduceMin 2022-05-24 11:16:42.748058317 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3237_ReduceMax 2022-05-24 11:16:42.748079777 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3237_ReduceMin 2022-05-24 11:16:42.748101870 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_925_ReduceMax 2022-05-24 11:16:42.748125204 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_925_ReduceMin 2022-05-24 11:16:42.748154855 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1941_ReduceMax 2022-05-24 11:16:42.748170618 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1941_ReduceMin 2022-05-24 11:16:42.748186966 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.55_ReduceMax 2022-05-24 11:16:42.748206232 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.55_ReduceMin 2022-05-24 11:16:42.748224541 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key_ReduceMax 2022-05-24 11:16:42.748244230 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key_ReduceMin 2022-05-24 11:16:42.748262349 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_323_ReduceMax 2022-05-24 11:16:42.748280755 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_323_ReduceMin 2022-05-24 11:16:42.748298218 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.39_ReduceMax 2022-05-24 11:16:42.748314186 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.39_ReduceMin 2022-05-24 11:16:42.748331629 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1253_ReduceMax 2022-05-24 11:16:42.748346125 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1253_ReduceMin 2022-05-24 11:16:42.748364767 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1725_ReduceMax 2022-05-24 11:16:42.748399310 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_1725_ReduceMin 2022-05-24 11:16:42.748421429 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2363_ReduceMax 2022-05-24 11:16:42.748437795 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2363_ReduceMin 2022-05-24 11:16:42.748456404 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.83_ReduceMax 2022-05-24 11:16:42.748476748 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.83_ReduceMin 2022-05-24 11:16:42.748493023 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2727_ReduceMax 2022-05-24 11:16:42.748526945 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2727_ReduceMin 2022-05-24 11:16:42.748550429 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2531_ReduceMax 2022-05-24 11:16:42.748573543 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2531_ReduceMin 2022-05-24 11:16:42.748595339 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_996_ReduceMax 2022-05-24 11:16:42.748610080 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_996_ReduceMin 2022-05-24 11:16:42.748626200 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2765_ReduceMax 2022-05-24 11:16:42.748648687 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2765_ReduceMin 2022-05-24 11:16:42.748674785 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Reshape_3297_ReduceMax 2022-05-24 11:16:42.748699977 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Reshape_3297_ReduceMin 2022-05-24 11:16:42.748718280 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3090_ReduceMax 2022-05-24 11:16:42.748738336 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_3090_ReduceMin 2022-05-24 11:16:42.748757244 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.35_ReduceMax 2022-05-24 11:16:42.748777226 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.35_ReduceMin 2022-05-24 11:16:42.748796508 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_409_ReduceMax 2022-05-24 11:16:42.748810681 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_409_ReduceMin 2022-05-24 11:16:42.748827539 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.35_ReduceMax 2022-05-24 11:16:42.748846310 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.35_ReduceMin 2022-05-24 11:16:42.748872586 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2875_ReduceMax 2022-05-24 11:16:42.748891936 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2875_ReduceMin 2022-05-24 11:16:42.748910071 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2194_ReduceMax 2022-05-24 11:16:42.748924893 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2194_ReduceMin 2022-05-24 11:16:42.748943313 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_698_ReduceMax 2022-05-24 11:16:42.748979747 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_698_ReduceMin 2022-05-24 11:16:42.749000829 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_393_ReduceMax 2022-05-24 11:16:42.749016400 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_393_ReduceMin 2022-05-24 11:16:42.749032172 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_957_ReduceMax 2022-05-24 11:16:42.749065810 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_957_ReduceMin 2022-05-24 11:16:42.749087608 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2580_ReduceMax 2022-05-24 11:16:42.749112498 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2580_ReduceMin 2022-05-24 11:16:42.749132133 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.31_ReduceMax 2022-05-24 11:16:42.749151751 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.31_ReduceMin 2022-05-24 11:16:42.749168541 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_3167_ReduceMax 2022-05-24 11:16:42.749185973 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_3167_ReduceMin 2022-05-24 11:16:42.749214515 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_470_ReduceMax 2022-05-24 11:16:42.749240302 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_470_ReduceMin 2022-05-24 11:16:42.749264919 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1463_ReduceMax 2022-05-24 11:16:42.749297863 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1463_ReduceMin 2022-05-24 11:16:42.749319313 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2963_ReduceMax 2022-05-24 11:16:42.749338663 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2963_ReduceMin 2022-05-24 11:16:42.749357496 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1508_ReduceMax 2022-05-24 11:16:42.749371973 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1508_ReduceMin 2022-05-24 11:16:42.749388275 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2803_ReduceMax 2022-05-24 11:16:42.749403823 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2803_ReduceMin 2022-05-24 11:16:42.749422939 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2873_ReduceMax 2022-05-24 11:16:42.749443599 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2873_ReduceMin 2022-05-24 11:16:42.749458885 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.43_ReduceMax 2022-05-24 11:16:42.749478051 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.43_ReduceMin 2022-05-24 11:16:42.749496585 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_919_ReduceMax 2022-05-24 11:16:42.749510565 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_919_ReduceMin 2022-05-24 11:16:42.749526329 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2271_ReduceMax 2022-05-24 11:16:42.749540046 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2271_ReduceMin 2022-05-24 11:16:42.749555802 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3295_ReduceMax 2022-05-24 11:16:42.749573230 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3295_ReduceMin 2022-05-24 11:16:42.749592239 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_887_ReduceMax 2022-05-24 11:16:42.749615786 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_887_ReduceMin 2022-05-24 11:16:42.749632479 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_237_ReduceMax 2022-05-24 11:16:42.749652293 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_237_ReduceMin 2022-05-24 11:16:42.749668111 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2366_ReduceMax 2022-05-24 11:16:42.749683252 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_2366_ReduceMin 2022-05-24 11:16:42.749700166 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_450_ReduceMax 2022-05-24 11:16:42.749732130 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_450_ReduceMin 2022-05-24 11:16:42.749753257 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1382_ReduceMax 2022-05-24 11:16:42.749771442 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1382_ReduceMin 2022-05-24 11:16:42.749787033 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1977_ReduceMax 2022-05-24 11:16:42.749820033 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1977_ReduceMin 2022-05-24 11:16:42.749842106 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_263_ReduceMax 2022-05-24 11:16:42.749868054 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_263_ReduceMin 2022-05-24 11:16:42.749888120 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1939_ReduceMax 2022-05-24 11:16:42.749902623 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1939_ReduceMin 2022-05-24 11:16:42.749918735 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.15_ReduceMax 2022-05-24 11:16:42.749936313 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.15_ReduceMin 2022-05-24 11:16:42.749963357 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1434_ReduceMax 2022-05-24 11:16:42.749985969 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1434_ReduceMin 2022-05-24 11:16:42.750004104 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1528_ReduceMax 2022-05-24 11:16:42.750022453 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1528_ReduceMin 2022-05-24 11:16:42.750040979 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_253_ReduceMax 2022-05-24 11:16:42.750056823 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_253_ReduceMin 2022-05-24 11:16:42.750073747 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1179_ReduceMax 2022-05-24 11:16:42.750093637 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1179_ReduceMin 2022-05-24 11:16:42.750123143 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_687_ReduceMax 2022-05-24 11:16:42.750159605 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_687_ReduceMin 2022-05-24 11:16:42.750181408 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2530_ReduceMax 2022-05-24 11:16:42.750200002 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2530_ReduceMin 2022-05-24 11:16:42.750218340 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2526_ReduceMax 2022-05-24 11:16:42.750232609 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2526_ReduceMin 2022-05-24 11:16:42.750248070 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_903_ReduceMax 2022-05-24 11:16:42.750263925 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_903_ReduceMin 2022-05-24 11:16:42.750279586 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.27_ReduceMax 2022-05-24 11:16:42.750299471 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.27_ReduceMin 2022-05-24 11:16:42.750325117 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2016_ReduceMax 2022-05-24 11:16:42.750339284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2016_ReduceMin 2022-05-24 11:16:42.750356352 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1767_ReduceMax 2022-05-24 11:16:42.750371375 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1767_ReduceMin 2022-05-24 11:16:42.750393054 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.79_ReduceMax 2022-05-24 11:16:42.750410976 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.79_ReduceMin 2022-05-24 11:16:42.750429216 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2488_ReduceMax 2022-05-24 11:16:42.750463500 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2488_ReduceMin 2022-05-24 11:16:42.750491520 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1944_ReduceMax 2022-05-24 11:16:42.750510657 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1944_ReduceMin 2022-05-24 11:16:42.750533934 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1961_ReduceMax 2022-05-24 11:16:42.750550526 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1961_ReduceMin 2022-05-24 11:16:42.750565958 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.111_ReduceMax 2022-05-24 11:16:42.750580667 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.111_ReduceMin 2022-05-24 11:16:42.750596647 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_700_ReduceMax 2022-05-24 11:16:42.750628103 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_700_ReduceMin 2022-05-24 11:16:42.750649251 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_325_ReduceMax 2022-05-24 11:16:42.750667968 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_325_ReduceMin 2022-05-24 11:16:42.750685100 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.19_ReduceMax 2022-05-24 11:16:42.750703085 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.19_ReduceMin 2022-05-24 11:16:42.750720639 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_413_ReduceMax 2022-05-24 11:16:42.750740668 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_413_ReduceMin 2022-05-24 11:16:42.750757903 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.91_ReduceMax 2022-05-24 11:16:42.750773611 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.91_ReduceMin 2022-05-24 11:16:42.750793516 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.23_ReduceMax 2022-05-24 11:16:42.750811292 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.23_ReduceMin 2022-05-24 11:16:42.750828277 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_377_ReduceMax 2022-05-24 11:16:42.750843879 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_377_ReduceMin 2022-05-24 11:16:42.750869480 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1784_ReduceMax 2022-05-24 11:16:42.750895179 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1784_ReduceMin 2022-05-24 11:16:42.750914994 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.19_ReduceMax 2022-05-24 11:16:42.750932674 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output.19_ReduceMin 2022-05-24 11:16:42.750949983 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2000_ReduceMax 2022-05-24 11:16:42.750976281 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2000_ReduceMin 2022-05-24 11:16:42.750999045 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_773_ReduceMax 2022-05-24 11:16:42.751025284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_773_ReduceMin 2022-05-24 11:16:42.751045110 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.71_ReduceMax 2022-05-24 11:16:42.751064160 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.71_ReduceMin 2022-05-24 11:16:42.751079367 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1305_ReduceMax 2022-05-24 11:16:42.751098246 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1305_ReduceMin 2022-05-24 11:16:42.751121718 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.15_ReduceMax 2022-05-24 11:16:42.751138435 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.15_ReduceMin 2022-05-24 11:16:42.751155121 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_448_ReduceMax 2022-05-24 11:16:42.751189067 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_448_ReduceMin 2022-05-24 11:16:42.751210831 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2294_ReduceMax 2022-05-24 11:16:42.751238509 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_2294_ReduceMin 2022-05-24 11:16:42.751264616 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3275_ReduceMax 2022-05-24 11:16:42.751286340 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3275_ReduceMin 2022-05-24 11:16:42.751310244 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1637_ReduceMax 2022-05-24 11:16:42.751326689 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1637_ReduceMin 2022-05-24 11:16:42.751350637 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.59_ReduceMax 2022-05-24 11:16:42.751370670 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.59_ReduceMin 2022-05-24 11:16:42.751393482 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.247_ReduceMax 2022-05-24 11:16:42.751412217 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.247_ReduceMin 2022-05-24 11:16:42.751429823 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2110_ReduceMax 2022-05-24 11:16:42.751450483 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_2110_ReduceMin 2022-05-24 11:16:42.751467797 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_703_ReduceMax 2022-05-24 11:16:42.751498819 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_703_ReduceMin 2022-05-24 11:16:42.751520335 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1529_ReduceMax 2022-05-24 11:16:42.751547890 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1529_ReduceMin 2022-05-24 11:16:42.751567776 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_867_ReduceMax 2022-05-24 11:16:42.751583119 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_867_ReduceMin 2022-05-24 11:16:42.751599926 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1579_ReduceMax 2022-05-24 11:16:42.751620045 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1579_ReduceMin 2022-05-24 11:16:42.751636316 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_559_ReduceMax 2022-05-24 11:16:42.751658894 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_559_ReduceMin 2022-05-24 11:16:42.751681336 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.215_ReduceMax 2022-05-24 11:16:42.751700533 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.215_ReduceMin 2022-05-24 11:16:42.751719618 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_3130_ReduceMax 2022-05-24 11:16:42.751739229 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_3130_ReduceMin 2022-05-24 11:16:42.751758049 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2618_ReduceMax 2022-05-24 11:16:42.751781788 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2618_ReduceMin 2022-05-24 11:16:42.751804773 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_447_ReduceMax 2022-05-24 11:16:42.751839216 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_447_ReduceMin 2022-05-24 11:16:42.751860508 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.7_ReduceMax 2022-05-24 11:16:42.751875553 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.7_ReduceMin 2022-05-24 11:16:42.751891529 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2449_ReduceMax 2022-05-24 11:16:42.751906422 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2449_ReduceMin 2022-05-24 11:16:42.751922547 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.39_ReduceMax 2022-05-24 11:16:42.751938124 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.39_ReduceMin 2022-05-24 11:16:42.751955826 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.223_ReduceMax 2022-05-24 11:16:42.751974192 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.223_ReduceMin 2022-05-24 11:16:42.751992963 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1892_ReduceMax 2022-05-24 11:16:42.752009068 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1892_ReduceMin 2022-05-24 11:16:42.752025142 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_795_ReduceMax 2022-05-24 11:16:42.752045099 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_795_ReduceMin 2022-05-24 11:16:42.752066580 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_578_ReduceMax 2022-05-24 11:16:42.752086761 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_578_ReduceMin 2022-05-24 11:16:42.752109512 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_923_ReduceMax 2022-05-24 11:16:42.752130166 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_923_ReduceMin 2022-05-24 11:16:42.752148657 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1945_ReduceMax 2022-05-24 11:16:42.752164999 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1945_ReduceMin 2022-05-24 11:16:42.752183795 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1210_ReduceMax 2022-05-24 11:16:42.752217006 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1210_ReduceMin 2022-05-24 11:16:42.752238541 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.75_ReduceMax 2022-05-24 11:16:42.752256742 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.75_ReduceMin 2022-05-24 11:16:42.752273003 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2070_ReduceMax 2022-05-24 11:16:42.752293680 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2070_ReduceMin 2022-05-24 11:16:42.752311350 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1923_ReduceMax 2022-05-24 11:16:42.752332014 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1923_ReduceMin 2022-05-24 11:16:42.752351116 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1050_ReduceMax 2022-05-24 11:16:42.752370530 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1050_ReduceMin 2022-05-24 11:16:42.752391727 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2199_ReduceMax 2022-05-24 11:16:42.752413339 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2199_ReduceMin 2022-05-24 11:16:42.752431252 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1431_ReduceMax 2022-05-24 11:16:42.752446191 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_1431_ReduceMin 2022-05-24 11:16:42.752461545 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1219_ReduceMax 2022-05-24 11:16:42.752485780 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1219_ReduceMin 2022-05-24 11:16:42.752506324 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_763_ReduceMax 2022-05-24 11:16:42.752522420 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_763_ReduceMin 2022-05-24 11:16:42.752540392 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1274_ReduceMax 2022-05-24 11:16:42.752567516 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1274_ReduceMin 2022-05-24 11:16:42.752586657 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_580_ReduceMax 2022-05-24 11:16:42.752605601 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_580_ReduceMin 2022-05-24 11:16:42.752623973 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.107_ReduceMax 2022-05-24 11:16:42.752642486 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.107_ReduceMin 2022-05-24 11:16:42.752661124 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_872_ReduceMax 2022-05-24 11:16:42.752677284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_872_ReduceMin 2022-05-24 11:16:42.752694394 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.19_ReduceMax 2022-05-24 11:16:42.752728845 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.19_ReduceMin 2022-05-24 11:16:42.752754675 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3059_ReduceMax 2022-05-24 11:16:42.752782938 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3059_ReduceMin 2022-05-24 11:16:42.752804705 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.79_ReduceMax 2022-05-24 11:16:42.752824974 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.79_ReduceMin 2022-05-24 11:16:42.752842559 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_836_ReduceMax 2022-05-24 11:16:42.752862509 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_836_ReduceMin 2022-05-24 11:16:42.752883378 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_221_ReduceMax 2022-05-24 11:16:42.752902032 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_221_ReduceMin 2022-05-24 11:16:42.752929456 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_726_ReduceMax 2022-05-24 11:16:42.752947415 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_726_ReduceMin 2022-05-24 11:16:42.752965311 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.87_ReduceMax 2022-05-24 11:16:42.752983849 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.87_ReduceMin 2022-05-24 11:16:42.753004104 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2706_ReduceMax 2022-05-24 11:16:42.753018529 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2706_ReduceMin 2022-05-24 11:16:42.753034427 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2672_ReduceMax 2022-05-24 11:16:42.753050124 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2672_ReduceMin 2022-05-24 11:16:42.753079330 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.31_ReduceMax 2022-05-24 11:16:42.753121446 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.31_ReduceMin 2022-05-24 11:16:42.753143789 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1346_ReduceMax 2022-05-24 11:16:42.753164047 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_1346_ReduceMin 2022-05-24 11:16:42.753181091 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1490_ReduceMax 2022-05-24 11:16:42.753205740 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1490_ReduceMin 2022-05-24 11:16:42.753243316 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1236_ReduceMax 2022-05-24 11:16:42.753262827 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1236_ReduceMin 2022-05-24 11:16:42.753281001 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_222_ReduceMax 2022-05-24 11:16:42.753299248 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_222_ReduceMin 2022-05-24 11:16:42.753316780 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3004_ReduceMax 2022-05-24 11:16:42.753339389 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3004_ReduceMin 2022-05-24 11:16:42.753359231 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1452_ReduceMax 2022-05-24 11:16:42.753390399 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1452_ReduceMin 2022-05-24 11:16:42.753410889 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2276_ReduceMax 2022-05-24 11:16:42.753433881 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2276_ReduceMin 2022-05-24 11:16:42.753452420 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_725_ReduceMax 2022-05-24 11:16:42.753481474 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_725_ReduceMin 2022-05-24 11:16:42.753502775 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.179_ReduceMax 2022-05-24 11:16:42.753524056 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.179_ReduceMin 2022-05-24 11:16:42.753543228 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attention_mask_ReduceMax 2022-05-24 11:16:42.753557946 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attention_mask_ReduceMin 2022-05-24 11:16:42.753577672 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_921_ReduceMax 2022-05-24 11:16:42.753595485 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_921_ReduceMin 2022-05-24 11:16:42.753612164 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1257_ReduceMax 2022-05-24 11:16:42.753628489 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1257_ReduceMin 2022-05-24 11:16:42.753645988 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.243_ReduceMax 2022-05-24 11:16:42.753667355 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.243_ReduceMin 2022-05-24 11:16:42.753686724 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2965_ReduceMax 2022-05-24 11:16:42.753702743 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2965_ReduceMin 2022-05-24 11:16:42.753721229 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.239_ReduceMax 2022-05-24 11:16:42.753741866 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.239_ReduceMin 2022-05-24 11:16:42.753759553 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.31_ReduceMax 2022-05-24 11:16:42.753778778 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.31_ReduceMin 2022-05-24 11:16:42.753796595 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_746_ReduceMax 2022-05-24 11:16:42.753814838 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_746_ReduceMin 2022-05-24 11:16:42.753832378 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2178_ReduceMax 2022-05-24 11:16:42.753849870 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_2178_ReduceMin 2022-05-24 11:16:42.753865869 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1943_ReduceMax 2022-05-24 11:16:42.753885673 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1943_ReduceMin 2022-05-24 11:16:42.753904284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1069_ReduceMax 2022-05-24 11:16:42.753923158 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_1069_ReduceMin 2022-05-24 11:16:42.753940478 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output_ReduceMax 2022-05-24 11:16:42.753959449 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_output_ReduceMin 2022-05-24 11:16:42.753976916 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.15_ReduceMax 2022-05-24 11:16:42.753995191 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.15_ReduceMin 2022-05-24 11:16:42.754015180 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.219_ReduceMax 2022-05-24 11:16:42.754035321 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.219_ReduceMin 2022-05-24 11:16:42.754053115 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2704_ReduceMax 2022-05-24 11:16:42.754067576 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2704_ReduceMin 2022-05-24 11:16:42.754082937 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1127_ReduceMax 2022-05-24 11:16:42.754102085 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_1127_ReduceMin 2022-05-24 11:16:42.754126117 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.15_ReduceMax 2022-05-24 11:16:42.754146078 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output key.15_ReduceMin 2022-05-24 11:16:42.754163712 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1984_ReduceMax 2022-05-24 11:16:42.754184404 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1984_ReduceMin 2022-05-24 11:16:42.754206024 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_955_ReduceMax 2022-05-24 11:16:42.754239815 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_955_ReduceMin 2022-05-24 11:16:42.754261211 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1414_ReduceMax 2022-05-24 11:16:42.754281232 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1414_ReduceMin 2022-05-24 11:16:42.754299021 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2964_ReduceMax 2022-05-24 11:16:42.754318067 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2964_ReduceMin 2022-05-24 11:16:42.754343233 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1283_ReduceMax 2022-05-24 11:16:42.754369294 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Split_1283_ReduceMin 2022-05-24 11:16:42.754390323 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2089_ReduceMax 2022-05-24 11:16:42.754413825 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Transpose_2089_ReduceMin 2022-05-24 11:16:42.754432052 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_942_ReduceMax 2022-05-24 11:16:42.754465211 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_942_ReduceMin 2022-05-24 11:16:42.754490971 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_326_ReduceMax 2022-05-24 11:16:42.754507304 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Div_326_ReduceMin 2022-05-24 11:16:42.754524541 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1001_ReduceMax 2022-05-24 11:16:42.754545777 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1001_ReduceMin 2022-05-24 11:16:42.754563410 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2487_ReduceMax 2022-05-24 11:16:42.754597103 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_2487_ReduceMin 2022-05-24 11:16:42.754618445 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.159_ReduceMax 2022-05-24 11:16:42.754640235 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.159_ReduceMin 2022-05-24 11:16:42.754658396 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.39_ReduceMax 2022-05-24 11:16:42.754691655 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output hidden_states.39_ReduceMin 2022-05-24 11:16:42.754713717 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.131_ReduceMax 2022-05-24 11:16:42.754729160 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.131_ReduceMin 2022-05-24 11:16:42.754745340 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2745_ReduceMax 2022-05-24 11:16:42.754781690 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Tanh_2745_ReduceMin 2022-05-24 11:16:42.754803957 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.35_ReduceMax 2022-05-24 11:16:42.754819295 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.35_ReduceMin 2022-05-24 11:16:42.754836339 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.35_ReduceMax 2022-05-24 11:16:42.754855468 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.35_ReduceMin 2022-05-24 11:16:42.754873963 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2147_ReduceMax 2022-05-24 11:16:42.754890820 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2147_ReduceMin 2022-05-24 11:16:42.754908827 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1159_ReduceMax 2022-05-24 11:16:42.754926631 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_1159_ReduceMin 2022-05-24 11:16:42.754943562 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.147_ReduceMax 2022-05-24 11:16:42.754966537 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.147_ReduceMin 2022-05-24 11:16:42.755001989 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1413_ReduceMax 2022-05-24 11:16:42.755018405 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1413_ReduceMin 2022-05-24 11:16:42.755034136 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.51_ReduceMax 2022-05-24 11:16:42.755049476 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output input.51_ReduceMin 2022-05-24 11:16:42.755065252 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1433_ReduceMax 2022-05-24 11:16:42.755084495 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1433_ReduceMin 2022-05-24 11:16:42.755101968 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_958_ReduceMax 2022-05-24 11:16:42.755147669 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_958_ReduceMin 2022-05-24 11:16:42.755169075 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3020_ReduceMax 2022-05-24 11:16:42.755189333 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3020_ReduceMin 2022-05-24 11:16:42.755226074 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2783_ReduceMax 2022-05-24 11:16:42.755241597 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2783_ReduceMin 2022-05-24 11:16:42.755277645 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3003_ReduceMax 2022-05-24 11:16:42.755312822 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_3003_ReduceMin 2022-05-24 11:16:42.755335744 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2198_ReduceMax 2022-05-24 11:16:42.755354216 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2198_ReduceMin 2022-05-24 11:16:42.755376696 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2239_ReduceMax 2022-05-24 11:16:42.755403830 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_2239_ReduceMin 2022-05-24 11:16:42.755425323 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1718_ReduceMax 2022-05-24 11:16:42.755460255 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1718_ReduceMin 2022-05-24 11:16:42.755481217 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_254_ReduceMax 2022-05-24 11:16:42.755510726 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_254_ReduceMin 2022-05-24 11:16:42.755534812 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3236_ReduceMax 2022-05-24 11:16:42.755550479 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_3236_ReduceMin 2022-05-24 11:16:42.755568128 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.23_ReduceMax 2022-05-24 11:16:42.755587696 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.23_ReduceMin 2022-05-24 11:16:42.755604220 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_486_ReduceMax 2022-05-24 11:16:42.755619500 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_486_ReduceMin 2022-05-24 11:16:42.755635174 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2273_ReduceMax 2022-05-24 11:16:42.755649300 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Sqrt_2273_ReduceMin 2022-05-24 11:16:42.755664852 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.43_ReduceMax 2022-05-24 11:16:42.755680789 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output attn_weights.43_ReduceMin 2022-05-24 11:16:42.755697451 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1729_ReduceMax 2022-05-24 11:16:42.755723444 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1729_ReduceMin 2022-05-24 11:16:42.755744813 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_924_ReduceMax 2022-05-24 11:16:42.755762874 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_924_ReduceMin 2022-05-24 11:16:42.755785351 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1684_ReduceMax 2022-05-24 11:16:42.755800885 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1684_ReduceMin 2022-05-24 11:16:42.755816911 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1345_ReduceMax 2022-05-24 11:16:42.755837860 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::MatMul_1345_ReduceMin 2022-05-24 11:16:42.755855546 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_431_ReduceMax 2022-05-24 11:16:42.755872351 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_431_ReduceMin 2022-05-24 11:16:42.755891299 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.187_ReduceMax 2022-05-24 11:16:42.755910419 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.187_ReduceMin 2022-05-24 11:16:42.755928646 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.15_ReduceMax 2022-05-24 11:16:42.755947082 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.15_ReduceMin 2022-05-24 11:16:42.755968283 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2652_ReduceMax 2022-05-24 11:16:42.755985591 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_2652_ReduceMin 2022-05-24 11:16:42.756005301 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2708_ReduceMax 2022-05-24 11:16:42.756026653 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2708_ReduceMin 2022-05-24 11:16:42.756052280 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.83_ReduceMax 2022-05-24 11:16:42.756072249 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output query.83_ReduceMin 2022-05-24 11:16:42.756093686 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2912_ReduceMax 2022-05-24 11:16:42.756119052 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Shape_2912_ReduceMin 2022-05-24 11:16:42.756142721 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1451_ReduceMax 2022-05-24 11:16:42.756159066 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1451_ReduceMin 2022-05-24 11:16:42.756182542 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.63_ReduceMax 2022-05-24 11:16:42.756201629 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output query.63_ReduceMin 2022-05-24 11:16:42.756218970 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1158_ReduceMax 2022-05-24 11:16:42.756235524 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_1158_ReduceMin 2022-05-24 11:16:42.756258096 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.199_ReduceMax 2022-05-24 11:16:42.756276119 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output input.199_ReduceMin 2022-05-24 11:16:42.756293667 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1397_ReduceMax 2022-05-24 11:16:42.756310247 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_1397_ReduceMin 2022-05-24 11:16:42.756327927 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2748_ReduceMax 2022-05-24 11:16:42.756363284 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_2748_ReduceMin 2022-05-24 11:16:42.756384442 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.47_ReduceMax 2022-05-24 11:16:42.756402797 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.47_ReduceMin 2022-05-24 11:16:42.756423286 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3276_ReduceMax 2022-05-24 11:16:42.756440005 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Reshape_3276_ReduceMin 2022-05-24 11:16:42.756457702 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.79_ReduceMax 2022-05-24 11:16:42.756476194 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output value.79_ReduceMin 2022-05-24 11:16:42.756494455 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1887_ReduceMax 2022-05-24 11:16:42.756510437 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1,1} for output onnx::Add_1887_ReduceMin 2022-05-24 11:16:42.756526956 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3182_ReduceMax 2022-05-24 11:16:42.756543001 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Shape_3182_ReduceMin 2022-05-24 11:16:42.756560856 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1506_ReduceMax 2022-05-24 11:16:42.756575821 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Add_1506_ReduceMin 2022-05-24 11:16:42.756594666 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1218_ReduceMax 2022-05-24 11:16:42.756629927 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1,1} for output onnx::Mul_1218_ReduceMin 2022-05-24 11:16:42.756651463 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_980_ReduceMax 2022-05-24 11:16:42.756678942 [W:onnxruntime:, execution_frame.cc:806 VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,1} for output onnx::Gemm_980_ReduceMin Model Optimizer arguments: Common parameters: - Path to the Input Model: /tmp/tmp3ifqoe73/temp.onnx - Path for generated IR: /tmp/tmp3ifqoe73 - IR output name: temp - Log level: ERROR - Batch: Not specified, inherited from the model - Input layers: input_0,input_1 - Output layers: Not specified, inherited from the model - Input shapes: (1, 181),(1, 181) - Source layout: Not specified - Target layout: Not specified - Layout: Not specified - Mean values: Not specified - Scale values: Not specified - Scale factor: Not specified - Precision of IR: FP32 - Enable fusing: True - User transformations: Not specified - Reverse input channels: False - Enable IR generation for fixed input shape: False - Use the transformations config file: None Advanced parameters: - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR: False - Force the usage of new Frontend of Model Optimizer for model conversion into IR: False OpenVINO runtime found in: /usr/local/lib/python3.7/dist-packages/openvino OpenVINO runtime version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 Model Optimizer version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 [ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] XML file: /tmp/tmp3ifqoe73/temp.xml [ SUCCESS ] BIN file: /tmp/tmp3ifqoe73/temp.bin [ SUCCESS ] Total execution time: 4.05 seconds. [ SUCCESS ] Memory consumed: 1062 MB. It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub* [ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11. Find more information about API v2.0 and IR v11 at https://docs.openvino.ai Model Optimizer arguments: Common parameters: - Path to the Input Model: /tmp/tmp3ifqoe73/temp.onnx - Path for generated IR: /tmp/tmp3ifqoe73 - IR output name: temp - Log level: ERROR - Batch: Not specified, inherited from the model - Input layers: input_0,input_1 - Output layers: Not specified, inherited from the model - Input shapes: (1, 181),(1, 181) - Source layout: Not specified - Target layout: Not specified - Layout: Not specified - Mean values: Not specified - Scale values: Not specified - Scale factor: Not specified - Precision of IR: FP16 - Enable fusing: True - User transformations: Not specified - Reverse input channels: False - Enable IR generation for fixed input shape: False - Use the transformations config file: None Advanced parameters: - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR: False - Force the usage of new Frontend of Model Optimizer for model conversion into IR: False OpenVINO runtime found in: /usr/local/lib/python3.7/dist-packages/openvino OpenVINO runtime version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 Model Optimizer version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 [ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] XML file: /tmp/tmp3ifqoe73/temp.xml [ SUCCESS ] BIN file: /tmp/tmp3ifqoe73/temp.bin [ SUCCESS ] Total execution time: 5.63 seconds. [ SUCCESS ] Memory consumed: 1062 MB. It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub* [ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11. Find more information about API v2.0 and IR v11 at https://docs.openvino.ai Model Optimizer arguments: Common parameters: - Path to the Input Model: /tmp/tmp3ifqoe73/temp.onnx - Path for generated IR: /tmp/tmp3ifqoe73 - IR output name: temp - Log level: ERROR - Batch: Not specified, inherited from the model - Input layers: input_0,input_1 - Output layers: Not specified, inherited from the model - Input shapes: (1, 181),(1, 181) - Source layout: Not specified - Target layout: Not specified - Layout: Not specified - Mean values: Not specified - Scale values: Not specified - Scale factor: Not specified - Precision of IR: FP32 - Enable fusing: True - User transformations: Not specified - Reverse input channels: False - Enable IR generation for fixed input shape: False - Use the transformations config file: None Advanced parameters: - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR: False - Force the usage of new Frontend of Model Optimizer for model conversion into IR: False OpenVINO runtime found in: /usr/local/lib/python3.7/dist-packages/openvino OpenVINO runtime version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 Model Optimizer version: 2022.1.0-7019-cdb9bec7210-releases/2022/1 [ SUCCESS ] Generated IR version 11 model. [ SUCCESS ] XML file: /tmp/tmp3ifqoe73/temp.xml [ SUCCESS ] BIN file: /tmp/tmp3ifqoe73/temp.bin [ SUCCESS ] Total execution time: 4.47 seconds. [ SUCCESS ] Memory consumed: 1062 MB. It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub* [ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11. Find more information about API v2.0 and IR v11 at https://docs.openvino.ai Average response time for GPT2 (10 tokens): 35.60269355773926 ms Average response time for GPT2 (181 tokens): 71.89674377441406 ms

    Hello, I'm username! I've tested nebullvm on the following setup: Hardware: Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz CPU and Tesla V100-SXM2-32GB GPU. Model: GPT2 - HuggingFace Tokens: 10

    • Vanilla performance: 25.12ms
    • Optimized performance: 35.6ms
    • Speedup: 0.7x Tokens: 181
    • Vanilla performance: 73.49ms
    • Optimized performance: 71.9ms
    • Speedup: 1.0x
    opened by mali19064 5
  • Different results with same data

    Different results with same data

    Hi everyone. I'm having some troubles getting consistency throughout the same inputs tensor.. Hw Sw overview:

    • Ubuntu 18.04
    • TensorRT-8.4.0.6
    • llvm 14.0
    • pip installed torch, onnx tensorflow latest stable
    • cuda 11.6
    • cudnn 8.4.0
    • nvidia 3070tx
    • Intel(R) Core(TM) i7-9700E CPU @ 2.60GHz

    The optimization that i'm doing as the samples:

    
import torch
import torchvision.models as models
from nebullvm import optimize_torch_model
import time
import numpy as np

model = models.vgg19()
bs, input_sizes = 1, [(3, 224, 224)]
save_dir = "."
optimized_model = optimize_torch_model( model, batch_size=bs, input_sizes=input_sizes, save_dir=save_dir)

x = torch.rand((1, *input_sizes[0]))

with torch.no_grad():
    res = optimized_model(x)[0]

    every iteration with the same vector give different results.. Sometimes there are NaN values and sometimes they change based on odd or even number of iterations. Example res[0][:10]:

    tensor([1.4013e-45, 4.5877e-41, 1.4013e-45,        nan, 1.4013e-45, 7.0065e-45, 1.4013e-45, 1.4013e-44, 2.9427e-44, 1.4013e-44])
tensor([3.6121e-11, 4.5877e-41, 3.6121e-11, 4.5877e-41, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00])
tensor([ 1.4013e-45,  0.0000e+00, -1.8365e+00,  0.0000e+00,  3.3631e-44, 0.0000e+00,  3.0810e-17,  4.5877e-41,  1.4013e-45,  4.5877e-41])
tensor([ 1.4013e-45,  0.0000e+00, -1.8370e+00,  0.0000e+00,  1.4013e-45, 0.0000e+00, -1.8370e+00,  0.0000e+00,  3.3631e-44,  0.0000e+00])

    Same x vector different results, sometimes about 40 order of magnitude. I'm getting something wrong? The inference time for the vgg19 it's incredible but I'm scared the something under the hood it's not working properly :

    %%timeit
res = optimized_model(x)[0]
145 µs ± 48 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
    bug 
    opened by UnibsMatt 5
  • AttributeError: module 'torch.nn.functional' has no attribute 'mish'

    AttributeError: module 'torch.nn.functional' has no attribute 'mish'

    Hi,

    i installed nebullvm , and when i import the package

    from nebullvm import optimize_model

    execution gives the error AttributeError: module 'torch.nn.functional' has no attribute 'mish'

    thanks in advance

    opened by AouatifZ 4
  • optimize_model - Information about the best performing optim method

    optimize_model - Information about the best performing optim method

    Hi,

    Thank you for this easy to use library. I'm using successfully using the optimize_model() function as per your notebook. I would like to know at the end, which optim method led to the optimized_model. Is there any way to have this information from the returned InferenceLearner object?

    Thank you!

    opened by pyvandenbussche 4
  • Error when installing on fresh conda env and apple macos silicon m1

    Error when installing on fresh conda env and apple macos silicon m1

    Hi,

    On an apple m1, I made sure to install tensorflow 2.7 on a new conda virtual environment:

    % conda install -c apple tensorflow-deps==2.7.0
% pip install tensorflow-macos==2.7.0
% pip install tensorflow-metal

    But when I tried pip install nebullvm I get this error:

    ERROR: Cannot install nebullvm==0.1.0, nebullvm==0.1.1, nebullvm==0.1.2, nebullvm==0.2.0, nebullvm==0.2.1 and nebullvm==0.2.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    nebullvm 0.2.2 depends on tensorflow<2.8.0 and >=2.7.0
    nebullvm 0.2.1 depends on tensorflow<2.8.0 and >=2.7.0
    nebullvm 0.2.0 depends on tensorflow<2.8.0 and >=2.7.0
    nebullvm 0.1.2 depends on tensorflow<2.8.0 and >=2.7.0
    nebullvm 0.1.1 depends on tensorflow>=2.7.0
    nebullvm 0.1.0 depends on tensorflow~=2.7.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

    What am I missing ? Thanks

    opened by scalastic 4
  • Could this benefit 'TensorFlow Lite for MicroControllers' models

    Could this benefit 'TensorFlow Lite for MicroControllers' models

    Models on microcontrollers ( eg. RPi Pico - ARM based) are very hardware constrained and could benefit greatly from this. And i know it's possible to convert a TF model to a TF-Lite one.

    Could 'nebullvm' be applied to the "TF lite for Microcontroller" flow to improve interference and/or is this a supported use-case already?

    I don't see TF Lite supported currently.

    enhancement 
    opened by NicoJuicy 4
  • numpy API version 0xe vs 0xd

    numpy API version 0xe vs 0xd

    When trying to run the sample code provided in the readme I got an error

    RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd Traceback (most recent call last): File "test_pytorch.py", line 3, in from nebullvm import optimize_torch_model File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/init.py", line 1, in from nebullvm.api.frontend.torch import optimize_torch_model # noqa F401 File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/api/frontend/torch.py", line 8, in from nebullvm.converters import ONNXConverter File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/converters/init.py", line 1, in from .converters import ONNXConverter # noqa F401 File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/converters/converters.py", line 5, in import tensorflow as tf File "<PATH_TO_ENV>/lib/python3.8/site-packages/tensorflow/init.py", line 37, in from tensorflow.python.tools import module_util as _module_util File "<PATH_TO_ENV>/lib/python3.8/site-packages/tensorflow/python/init.py", line 37, in from tensorflow.python.eager import context File "<PATH_TO_ENV>/lib/python3.8/site-packages/tensorflow/python/eager/context.py", line 35, in from tensorflow.python.client import pywrap_tf_session File "<PATH_TO_ENV>/lib/python3.8/site-packages/tensorflow/python/client/pywrap_tf_session.py", line 19, in from tensorflow.python.client._pywrap_tf_session import * ImportError: SystemError: <built-in method contains of dict object at 0x7f7393496680> returned a result with an error set

    when searching about 0xe and 0xd versions of numpy, this page said to try and upgrade which I did after which the error changed and uninstalled the latest version of numpy and said that it could not find openvino:

    <PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/inference_learners/openvino.py:26: UserWarning: No valid OpenVino installation has been found. Trying to re-install it from source. warnings.warn( Collecting openvino-dev Using cached openvino_dev-2021.4.2-3976-py3-none-any.whl (6.2 MB) Collecting numpy<1.20,>=1.16.6 Using cached numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB) Collecting editdistance>=0.5.3 Using cached editdistance-0.6.0-cp38-cp38-manylinux2010_x86_64.whl (286 kB) Collecting pandas~=1.1.5 Using cached pandas-1.1.5-cp38-cp38-manylinux1_x86_64.whl (9.3 MB) Requirement already satisfied: pydicom>=2.1.2 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (2.2.2) Collecting rawpy>=0.16.0 Using cached rawpy-0.17.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB) Requirement already satisfied: jstyleson~=0.0.2 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (0.0.2) Collecting defusedxml>=0.7.1 Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB) Requirement already satisfied: requests>=2.25.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (2.27.1) Requirement already satisfied: PyYAML>=5.4.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (6.0) Requirement already satisfied: py-cpuinfo>=7.0.0 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (8.0.0) Collecting opencv-python==4.5.* Using cached opencv_python-4.5.5.62-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.4 MB) Requirement already satisfied: tqdm>=4.54.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (4.62.3) Collecting scipy~=1.5.4 Using cached scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl (25.8 MB) Collecting scikit-image>=0.17.2 Using cached scikit_image-0.19.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.0 MB) Requirement already satisfied: sentencepiece>=0.1.95 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (0.1.96) Requirement already satisfied: shapely>=1.7.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (1.8.1.post1) Collecting networkx~=2.5 Using cached networkx-2.6.3-py3-none-any.whl (1.9 MB) Collecting nltk>=3.5 Using cached nltk-3.7-py3-none-any.whl (1.5 MB) Requirement already satisfied: pillow>=8.1.2 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (9.0.0) Requirement already satisfied: yamlloader>=0.5 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (1.1.0) Requirement already satisfied: addict>=2.4.0 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (2.4.0) Collecting nibabel>=3.2.1 Using cached nibabel-3.2.2-py3-none-any.whl (3.3 MB) Collecting hyperopt~=0.1.2 Using cached hyperopt-0.1.2-py3-none-any.whl (115 kB) Requirement already satisfied: texttable~=1.6.3 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (1.6.4) Requirement already satisfied: progress>=1.5 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (1.6) Collecting parasail>=1.2.4 Using cached parasail-1.2.4-py2.py3-none-manylinux2010_x86_64.whl (14.1 MB) Collecting openvino==2021.4.2 Using cached openvino-2021.4.2-3976-cp38-cp38-manylinux2014_x86_64.whl (28.9 MB) Collecting scikit-learn>=0.24.1 Using cached scikit_learn-1.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB) Requirement already satisfied: fast-ctc-decode>=0.2.5 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (0.3.0) Requirement already satisfied: tokenizers>=0.10.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from openvino-dev) (0.11.5) Requirement already satisfied: six in <PATH_TO_ENV>/lib/python3.8/site-packages (from hyperopt~=0.1.2->openvino-dev) (1.16.0) Collecting future Using cached future-0.18.2-py3-none-any.whl Requirement already satisfied: pymongo in <PATH_TO_ENV>/lib/python3.8/site-packages (from hyperopt~=0.1.2->openvino-dev) (4.0.1) Requirement already satisfied: setuptools in <PATH_TO_ENV>/lib/python3.8/site-packages (from nibabel>=3.2.1->openvino-dev) (56.0.0) Requirement already satisfied: packaging>=14.3 in <PATH_TO_ENV>/lib/python3.8/site-packages (from nibabel>=3.2.1->openvino-dev) (21.3) Requirement already satisfied: regex>=2021.8.3 in <PATH_TO_ENV>/lib/python3.8/site-packages (from nltk>=3.5->openvino-dev) (2022.1.18) Collecting click Using cached click-8.0.4-py3-none-any.whl (97 kB) Requirement already satisfied: joblib in <PATH_TO_ENV>/lib/python3.8/site-packages (from nltk>=3.5->openvino-dev) (1.1.0) Requirement already satisfied: python-dateutil>=2.7.3 in <PATH_TO_ENV>/lib/python3.8/site-packages (from pandas~=1.1.5->openvino-dev) (2.8.2) Requirement already satisfied: pytz>=2017.2 in <PATH_TO_ENV>/lib/python3.8/site-packages (from pandas~=1.1.5->openvino-dev) (2021.3) Requirement already satisfied: urllib3<1.27,>=1.21.1 in <PATH_TO_ENV>/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev) (1.26.8) Requirement already satisfied: idna<4,>=2.5 in <PATH_TO_ENV>/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev) (3.3) Requirement already satisfied: certifi>=2017.4.17 in <PATH_TO_ENV>/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev) (2021.10.8) Requirement already satisfied: charset-normalizer~=2.0.0 in <PATH_TO_ENV>/lib/python3.8/site-packages (from requests>=2.25.1->openvino-dev) (2.0.12) Collecting tifffile>=2019.7.26 Using cached tifffile-2022.2.9-py3-none-any.whl (180 kB) Collecting PyWavelets>=1.1.1 Using cached PyWavelets-1.2.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (6.3 MB) Collecting imageio>=2.4.1 Using cached imageio-2.16.0-py3-none-any.whl (3.3 MB) Requirement already satisfied: threadpoolctl>=2.0.0 in <PATH_TO_ENV>/lib/python3.8/site-packages (from scikit-learn>=0.24.1->openvino-dev) (3.1.0) Using cached imageio-2.15.0-py3-none-any.whl (3.3 MB) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in <PATH_TO_ENV>/lib/python3.8/site-packages (from packaging>=14.3->nibabel>=3.2.1->openvino-dev) (3.0.7) Installing collected packages: numpy, networkx, future, editdistance, defusedxml, click, tifffile, scipy, rawpy, PyWavelets, parasail, pandas, openvino, opencv-python, nltk, nibabel, imageio, scikit-learn, scikit-image, hyperopt, openvino-dev Attempting uninstall: numpy Found existing installation: numpy 1.22.2 Uninstalling numpy-1.22.2: Successfully uninstalled numpy-1.22.2 ERROR: Could not install packages due to an OSError: [Errno 39] Directory not empty: '<PATH_TO_ENV>/lib/python3.8/site-packages/~~mpy.libs' Traceback (most recent call last): File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/inference_learners/openvino.py", line 23, in from openvino.inference_engine import IECore ModuleNotFoundError: No module named 'openvino' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "test_pytorch.py", line 3, in from nebullvm import optimize_torch_model File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/init.py", line 1, in from nebullvm.api.frontend.torch import optimize_torch_model # noqa F401 File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/api/frontend/torch.py", line 10, in from nebullvm.optimizers.multi_compiler import MultiCompilerOptimizer File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/optimizers/init.py", line 5, in from nebullvm.optimizers.openvino import OpenVinoOptimizer File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/optimizers/openvino.py", line 5, in from nebullvm.inference_learners.openvino import ( File "<PATH_TO_ENV>/lib/python3.8/site-packages/nebullvm/inference_learners/openvino.py", line 33, in from openvino.inference_engine import IECore ModuleNotFoundError: No module named 'openvino'

    The error about not having a module named openvino was present even when I ranpython -c "import nebullvm" but since the README.md asks to ignore import errors, I did. I tried pip install openvino but openvino needs numpy < 1.20 while tensorflow needs numpy >= 1.20 so there were incompatibilities. I had used pip to install nebullvm

    My specifications: python 3.8 torch 1.10.2+cu13 CPU: intel i5-9300H PC: Acer Predator Helios PH315-52 V1.12 OS: Arch Linux x86_64

    bug 
    opened by IamMarcIvanov 3
  • Add support for HuggingFace's TensorFlow models

    Add support for HuggingFace's TensorFlow models

    New Feature:

    This PR implements support for HuggingFace's TensorFlow models, solving #125. It also adds a notebook that shows a use case with BERT.

    Example of usage:

    from speedster import optimize_model
from transformers import TFAutoModel, AutoTokenizer

# Make sure to always load a matching tokenizer and model!
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
model = TFAutoModel.from_pretrained("bert-base-cased")

# Let's load some data and tokenize it
test_string = "This is a sentence!"
tokenized_inputs = tokenizer(test_string, return_tensors="np")

dynamic_info = {
    "inputs": [
        {0: "batch", 1: "num_tokens"},
        {0: "batch", 1: "num_tokens"},
        {0: "batch", 1: "num_tokens"},
    ],
    "outputs": [{0: "batch", 1: "num_tokens"}, {0: "batch"}],
}

# Run Speedster optimization
optimized_model = optimize_model(
  model, input_data=[tokenized_inputs], dynamic_info=dynamic_info
)

    Minor changes:

    • Added a warning when dynamic_info is not provided with HuggingFace models
    • The final report on the optimized model now contains also the name of the compiler used
    • Fix a tvm issue when using the PyTorch implementation with gpu
    • Added dynamic shape support for optimized models converted from TensorFlow to ONNX
    opened by valeriosofi 0
  • Support XLA for both torch and tensorflow

    Support XLA for both torch and tensorflow

    Description

    Currently nebullvm does not support TF-built in compiler XLA, which also allows the model to be compiled on Google TPUs. XLA is available for JAX, TF and PyTorch. Details about the TensorFlow and PyTorch implementations can be found in the following links:

    Integration

    We should implement two new classes in nebullvm both inheriting from Compiler. They should implement the interfaces for PyTorch and TensorFlow of the XLA compiler. Note that XLA also supports reduced precision, so the quantization types must be managed too. For inspiration while coding the class interface you can analyse the Compilers built on top of TensorRT.

    Note that XLA performance should be tested on a TPU for accessing its real performance.

    TODO list

    • [ ] Implement a PoC of the feature on Colab
    • [ ] If not available a TPU, ping @valeriosofi for running the final version of the Colab notebook on Nebuly’s GCP account (Note that the debugging can be performed by yourself using other HW devices).
    • [ ] Analyse the impact of the feature respect the actual AutoBoost
    • [ ] If a positive impact of the feature is assessed we should implement it as a Compiler object else send your results to [email protected] detailing your findings and explaining why we should not proceed with the implementation of the feature
    • [ ] Fork the nebullvm repo https://github.com/nebuly-ai/nebullvm
    • [ ] Read the Contribution Guidelines
    • [ ] Create a PR to main explaining your changes and showing the improvements obtained with the XLA compilers respect the previous versions
    opened by diegofiori 1
  • Argument Parsing for model and input_data param's.

    Argument Parsing for model and input_data param's.

    Instead of taking the inputs from predefined variables, what if we could import the parameters from another file where those parameters are already defined to be accessed as parsed arguments?

    opened by mbilalai 0
  • Input Data

    Input Data

    What if we could just pass PyTorch dataloader instead of list of tuple of tuple of things? In this way one can simply put entire optimization logic under an if condition.

    opened by isgursoy 0
Releases(v0.4.4)

  • v0.4.4(Oct 20, 2022)

    nebullvm 0.4.4 Release Notes

    This release of Nebullvm provides new optimizers and various improvements in code stability.

    New Features

    • Update notebooks with new api.
    • Improve test coverage.
    • Add Intel Neural compressor pruning and quantization.
    • The computation of the latency of the models now uses all the data and not only the first sample.
    • Dynamic shape of openvino has been updated with the new method available from version 2
    • Now the optimized model is discarted if the result is different from the original model (metric_drop_ths=0)

    Bug fixed

    • Fix an issue during onnx quantization, now it's much faster than before.
    • Fix a tensor RT bug in static quantization with onnx interface.
    • Fixes and improvements on the torchscript compiler: now it supports also trace and torch.fx for tracing the model.
    • Fix a bug on macos related to ONNX and int8 quantization.
    • Fix a bug on sparseml that prevented it from working on colab.
    • Bug-fixes on the deepsparse compiler.
    • Fixes and improvements on the onnx internal model handling.
    • Fix an issue on tensorflow backend.
    • Fixes on torch and onnx tensorrt with transformers.
    • Fix a bug on tensor rt static quantization when using a new version of polygraphy
    • Fix a bug on huggingface when passing the tokenizer to the optimize_model function
    • Fix a bug when using quantization with a few data

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    Source code(tar.gz)
    Source code(zip)

  • v0.4.3(Sep 12, 2022)

    nebullvm 0.4.3 Release Notes

    Minor release that fixes some bugs added in v0.4.2.

    Bug fixed

    • Fix bug preventing the installation without TensorFlow.
    • Fix a bug while using the HuggingFace Interface

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    Source code(tar.gz)
    Source code(zip)

  • v0.4.2(Sep 8, 2022)

    nebullvm 0.4.2 Release Notes

    Minor release that fixes some bugs and reduces the number of strict requirements needed to run Nebullvm.

    New Features

    • Support ignore_compilers also for torchscript and tflite
    • Tensorflow is not a strict nebullvm requirement anymore.

    Bug fixed

    • Solve bug on half-precision with onnx-runtime
    • Fix a bug on tensor rt quantization: numpy arrays were passed to inference learner instead of tensors.

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    Source code(tar.gz)
    Source code(zip)

  • v0.4.1(Sep 2, 2022)

    nebullvm 0.4.1 Release Notes

    Minor release fixing some bugs and extending support for TensorRT directly with the PyTorch interface.

    New Features

    • Support for TensorRT directly with PyTorch models.

    Bug fixed

    • Bug in conversion to onnx that could lead to wrong inference results

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    Source code(tar.gz)
    Source code(zip)

  • v0.4.0(Jul 26, 2022)

    nebullvm 0.4.0 Release Notes

    "One API to rule them all". This major release of Nebullvm provides a brand new API unique to all Deep Learning frameworks.

    New Features

    • New unique API for all the Deep Learning frameworks.
    • Support for SparseML pruning.
    • Beta-feature Support for Intel-Neural-Compressor's Pruning.
    • Add support for BladeDISC compiler.
    • Modify the latency calculation for each model by using the median instead of the mean across different model runs.
    • Implement an early stop mechanism for latency computation.

    Bug fixed

    • Fix bug with HuggingFace models causing a failure during optimizations.

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    • Reiase (@reiase)
    Source code(tar.gz)
    Source code(zip)

  • v0.3.2(Jul 19, 2022)

    nebullvm 0.3.2 Release Notes

    Minor release for maintenance purposes. It fixes bugs and generally improves the code stability.

    New Features

    • In the Pytorch framework, whenever input data is provided for optimization, the model converter also uses it during the conversion of the model to onnx, instead of using the data only at the stage of applying the "precision reduction techniques."

    Bug fixed

    • Fix bug with OpenVino 2.0 not working with 1-dimensional arrays.
    • Fix bug while using TensorRT engine which was returning cpu-tensors also when input tensors where on GPU.
    • Fix requirements conflicts on Intel CPUs due to an old numpy version required by OpenVino.

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    • SolomidHero (@SolomidHero)
    • Emile Courthoud (@emilecourthoud)
    Source code(tar.gz)
    Source code(zip)

  • v0.3.1(Jun 28, 2022)

    nebullvm 0.3.1 Release Notes

    We are pleased to announce that we have added the option to run nebullvm from a Docker container. We provide both a Docker image on Docker Hub and the Dockerfile code to produce the Docker container directly from the latest version of the source code.

    New Features

    • Add Dockerfile and upload docker images on Docker Hub.
    • Implement new backend for the Tensorflow API running on top of TensorFlow and TFLite.
    • Implement new backend for the PyTorch API running on top of TorchScript.

    Bug fixed

    • Fix bug with TensorRT in the Tensorflow API.
    • Fix bug with OpenVino 2.0 not using the quantization on intel devices.

    Contributors

    • Diego Fiori (@morgoth95)
    • Valerio Sofi (@valeriosofi)
    • Emile Courthoud (@emilecourthoud)
    Source code(tar.gz)
    Source code(zip)

  • v0.3.0(May 10, 2022)

    nebullvm 0.3.0 Release Notes

    We are super excited to announce the new major release nebullvm 0.3.0, where nebullvm's AI inference accelerator becomes more powerful, stable and covers more use cases.

    nebullvm is an open-source library that generates an optimized version of your deep learning model that runs 2-10 times faster in inference without performance loss by leveraging multiple deep learning compilers (OpenVINO, TensorRT, etc.). With the new release 0.3.0, nebullvm can now accelerate inference up to 30x if you specify that you are willing to trade off a self-defined amount of accuracy/precision to get an even lower response time and a lighter model. This additional acceleration is achieved by exploiting optimization techniques that slightly modify the model graph to make it lighter, such as quantization, half precision, distillation, sparsity, etc.

    Find tutorials and examples on how to use nebullvm, as well as installation instructions in the main readme of nebullvm library. And check below if you want to learn more about

    • Overview of Nebullvm 0.3.0
    • Benchmarks
    • How the new Nebullvm 0.3.0 API Works
    • New Features & Bug Fixes

    Overview of Nebullvm

    With this new version, nebullvm continues in its mission to be:

    ☘️ Easy-to-use. It takes a few lines of code to install the library and optimize your models.

    🔥 Framework agnostic. nebullvm supports the most widely used frameworks (PyTorch, TensorFlow, 🆕ONNX🆕 and Hugging Face, etc.) and provides as output an optimized version of your model with the same interface (PyTorch, TensorFlow, etc.).

    💻 Deep learning model agnostic. nebullvm supports all the most popular deep learning architectures such as transformers, LSTM, CNN and FCN.

    🤖 Hardware agnostic. The library now works on most CPU and GPU and will soon support TPU and other deep learning-specific ASIC.

    🔑 Secure. Everything runs locally on your hardware.

    ✨ Leveraging the best optimization techniques. There are many inference techniques such as deep learning compilers, 🆕quantization or half precision🆕, and soon sparsity and distillation, which are all meant to optimize the way your AI models run on your hardware.

    Benchmarks

    We have tested nebullvm on popular AI models and hardware from leading vendors.

    The table below shows the inference speedup provided by nebullvm. The speedup is calculated as the response time of the unoptimized model divided by the response time of the accelerated model, as an average over 100 experiments. As an example, if the response time of an unoptimized model was on average 600 milliseconds and after nebullvm optimization only 240 milliseconds, the resulting speedup is 2.5x times, meaning 150% faster inference.

    A complete overview of the experiment and findings can be found on this page.

    | | M1 Pro | Intel Xeon | AMD EPYC | Nvidia T4 | |-------------------------|:------------:|:---------------:|:-------------:|:-------------:| | EfficientNetB0 | 23.3x | 3.5x | 2.7x | 1.3x | | EfficientNetB2 | 19.6x | 2.8x | 1.5x | 2.7x | | EfficientNetB6 | 19.8x | 2.4x | 2.5x | 1.7x | | Resnet18 | 1.2x | 1.9x | 1.7x | 7.3x | | Resnet152 | 1.3x | 2.1x | 1.5x | 2.5x | | SqueezeNet | 1.9x | 2.7x | 2.0x | 1.3x | | Convnext tiny | 3.2x | 1.3x | 1.8x | 5.0x | | Convnext large | 3.2x | 1.1x | 1.6x | 4.6x | | GPT2 - 10 tokens | 2.8x | 3.2x | 2.8x | 3.8x | | GPT2 - 1024 tokens | - | 1.7x | 1.9x | 1.4x | | Bert - 8 tokens | 6.4x | 2.9x | 4.8x | 4.1x | | Bert - 512 tokens | 1.8x | 1.3x | 1.6x | 3.1x | | ____________________ | ____________ | ____________ | ____________ | ____________ |

    Overall, the library provides great results, with more than 2x acceleration in most cases and around 20x in a few applications. We can also observe that acceleration varies greatly across different hardware-model couplings, so we suggest you test nebullvm on your model and hardware to assess its full potential. You can find the instructions below.

    Besides, across all scenarios, nebullvm is very helpful for its ease of use, allowing you to take advantage of inference optimization techniques without having to spend hours studying, testing and debugging these technologies.

    How the New Nebullvm API Works

    With the latest release, nebullvm has a new API and can be deployed in two ways.

    Option A: 2-10x acceleration, NO performance loss

    If you choose this option, nebullvm will test multiple deep learning compilers (TensorRT, OpenVINO, ONNX Runtime, etc.) and identify the optimal way to compile your model on your hardware, increasing inference speed by 2-10 times without affecting the performance of your model.

    Option B: 2-30x acceleration, supervised performance loss

    Nebullvm is capable of speeding up inference by much more than 10 times in case you are willing to sacrifice a fraction of your model's performance. If you specify how much performance loss you are willing to sustain, nebullvm will push your model's response time to its limits by identifying the best possible blend of state-of-the-art inference optimization techniques, such as deep learning compilers, distillation, quantization, half precision, sparsity, etc.

    Performance monitoring is accomplished using the perf_loss_ths (performance loss threshold), and the perf_metric for performance estimation.

    When a predefined metric (e.g. "accuracy") or a custom metric is passed as the perf_metric argument, the value of perf_loss_ths will be used as the maximum acceptable loss for the given metric evaluated on your datasets (Option B.1).

    When no perf_metric is provided as input, nebullvm calculates the performance loss using the default precision function. If the dataset is provided, the precision will be calculated on 100 sampled data (option B.2). Otherwise, the data will be randomly generated from the metadata provided as input, i.e. input_sizes and batch_size (option B.3).

    Check out the main GitHub readme if you want to take a look at nebullvm's performance and benchmarks, tutorials and notebooks on how to implement nebullvm with ease. And please leave a ⭐ if you enjoy the project and join the Discord community where we chat about nebullvm and AI optimization.

    New Features and Bug Fixes

    New features

    • Implemented quantization or half precision optimization techniques
    • Added support for models in the ONNX framework
    • Improved performance of Microsoft ONNX Runtime with transformers
    • Implemented nebullvm into Jina's amazing Clip-as-a-Service library for performance boost ( coming soon)
    • Accelerated library installation
    • Refactored the code to include support for datasets as an API
    • Released new benchmarks, notebooks and tutorials that can be found on the github readme

    Bug fixing

    • Fixed bug related to Intel OpenVINO applied to dynamic shapes. Thanks @kartikeyporwal for the support!
    • Fixed bug with model storage.
    • Fixed bug causing issues with NVIDIA TensorRT output. Thanks @UnibsMatt for identifying the problem.

    Contributors

    • @morgoth95 🥳
    • @emilecourthoud 🚀
    • @kartikeyporwal 🥇
    • @aurimgg 🚗
    Source code(tar.gz)
    Source code(zip)

  • v0.2.2(Apr 30, 2022)

    nebullvm 0.2.2 Release Notes

    The nebullvm 0.2.2 is minor release fixing some bugs.

    New Features

    • Allow the user to select the maximum number of CPU-threads per model to use during optimization and inference.

    Bug fixed

    • Fix bug in ONNXRuntime InferenceLearner

    Contributors

    • Diego Fiori (@morgoth95)
    Source code(tar.gz)
    Source code(zip)

  • v0.2.1(Apr 26, 2022)

    nebullvm 0.2.1 Release Notes

    The nebullvm 0.2.1 is minor release fixing some bugs and supporting optimization directly on ONNX models.

    New Features

    • ONNX interface for model optimization

    Bug fixed

    • Fix bug in tensorRT

    Contributors

    • Diego Fiori (@morgoth95)
    Source code(tar.gz)
    Source code(zip)

  • v0.2.0(Apr 3, 2022)

    nebullvm 0.2.0 Release Notes

    The nebullvm 0.2.0 is major release implementing new important features and fixing some bugs.

    New Features

    • Support for dynamic shapes for both the PyTorch and TensorFlow interfaces
    • Support for Transformer models built using the HuggingFace framework
    • Add ONNXRuntime to the supported backends for optimized models
    • New README, updated with benchmarks on SOTA models for both NLP and Computer Vision

    Bug fixed

    • Fix error in the tensorflow API preventing the usage of the optimize_tf_model function

    Contributors

    • Diego Fiori (@morgoth95)
    • Emile Courthoud (@emilecourthoud)
    Source code(tar.gz)
    Source code(zip)

  • v0.1.2(Mar 1, 2022)

    nebullvm 0.1.2 Release Notes

    The nebullvm 0.1.2 is maintenance release fixing few bugs and implementing new features.

    New Features

    • Support for the TorchScript API when optimising with ApacheTVM compiler.

    Bug fixed

    • The learners optimised with OpenVino now do not raise KeyErrors at prediction time anymore.
    • The learners optimised with ApacheTVM can be saved and loaded multiple times. Previously, trying to save a loaded model ended up in raising an error.
    • Fix bug in the auto-installer feature due to incompatibilities between Tensorflow 2.8 and OpenVino
    • Modify the behaviour of MultiCompilerOptimizeravoiding errors due to the pickling of C-related files.

    Contributors

    • Diego Fiori (@morgoth95)
    Source code(tar.gz)
    Source code(zip)

  • v0.1.1(Feb 28, 2022)

    nebullvm 0.1.1 Release Notes

    Official Alpha release of the nebullvmlibrary. The all-in-one library for deep learning compilers.

    Main features

    The main release contains:

    • wheels for installing with pip
    • auto-installation feature for supported compilers
    • support for OpenVINO, TensorRT and ApacheTVM
    • support for model built in Tensorflow and PyTorch
    • Optimised model API identical to the one of the input model

    Contributors

    A total of 3 people contributed to this release.

    • Diego Fiori (@morgoth95)
    • Emile Courthoud (@emilecourthoud)
    • Francesco Signorato (@FrancescoSignorato)
    Source code(tar.gz)
    Source code(zip)
Owner
Nebuly
Optimize everywhere, all the time.
Nebuly
GitHub
🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools

Hugging Face Optimum ?? Optimum is an extension of ?? Transformers, providing a set of performance optimization tools enabling maximum efficiency to t

Hugging Face 842 Dec 30, 2022
This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes Introduction This is the unofficial code of Deep Dual-re

chenjun 113 Dec 23, 2022
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

null 152 Jan 2, 2023
PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

PyTorch-LIT PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices. With

Amin Rezaei 157 Dec 11, 2022
🐦 Opytimizer is a Python library consisting of meta-heuristic optimization techniques.

Opytimizer: A Nature-Inspired Python Optimizer Welcome to Opytimizer. Did you ever reach a bottleneck in your computational experiments? Are you tired

Gustavo Rosa 546 Dec 31, 2022
Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification.

Easy Few-Shot Learning Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification. This repository is made for you

Sicara 399 Jan 8, 2023
Deepparse is a state-of-the-art library for parsing multinational street addresses using deep learning

Here is deepparse. Deepparse is a state-of-the-art library for parsing multinational street addresses using deep learning. Use deepparse to Use the pr

GRAAL/GRAIL 192 Dec 20, 2022
LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

LWCC: A LightWeight Crowd Counting library for Python LWCC is a lightweight crowd counting framework for Python. It wraps four state-of-the-art models

Matija Teršek 39 Dec 28, 2022
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

TorchMultimodal (Alpha Release) Introduction TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

Meta Research 663 Jan 6, 2023
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Adversarial Robustness Toolbox (ART) is a Python library for Machine Learning Security. ART provides tools that enable developers and researchers to defend and evaluate Machine Learning models and applications against the adversarial threats of Evasion, Poisoning, Extraction, and Inference. ART supports all popular machine learning frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn, XGBoost, LightGBM, CatBoost, GPy, etc.), all data types (images, tables, audio, video, etc.) and machine learning tasks (classification, object detection, speech recognition, generation, certification, etc.).

null 3.4k Jan 4, 2023
library for nonlinear optimization, wrapping many algorithms for global and local, constrained or unconstrained, optimization

NLopt is a library for nonlinear local and global optimization, for functions with and without gradient information. It is designed as a simple, unifi

Steven G. Johnson 1.4k Dec 25, 2022
Boost learning for GNNs from the graph structure under challenging heterophily settings. (NeurIPS'20)

Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu,

GEMS Lab: Graph Exploration & Mining at Scale, University of Michigan 70 Dec 18, 2022
An algorithm study of the 6th iOS 10 set of Boost Camp Web Mobile

알고리즘 스터디 ?? 부스트캠프 웹모바일 6기 iOS 10조의 알고리즘 스터디 입니다. 개인적인 사정 등으로 S034, S055만 참가하였습니다. 스터디 목적 상진: 코테 합격 + 부캠끝나고 아침에 일어나기 위해 필요한 사이클 기완: 꾸준하게 자리에 앉아 공부하기 +

null 2 Jan 11, 2022
This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

null 4 Aug 2, 2022
State of the Art Neural Networks for Deep Learning

pyradox This python library helps you with implementing various state of the art neural networks in a totally customizable fashion using Tensorflow 2

Ritvik Rastogi 60 May 29, 2022
Code for paper "A Critical Assessment of State-of-the-Art in Entity Alignment" (https://arxiv.org/abs/2010.16314)

A Critical Assessment of State-of-the-Art in Entity Alignment This repository contains the source code for the paper A Critical Assessment of State-of

Max Berrendorf 16 Oct 14, 2022
Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, ...)

Image Classification Project Killer in PyTorch This repo is designed for those who want to start their experiments two days before the deadline and ki

null 349 Dec 8, 2022
State of the art Semantic Sentence Embeddings

Contrastive Tension State of the art Semantic Sentence Embeddings Published Paper · Huggingface Models · Report Bug Overview This is the official code

Fredrik Carlsson 88 Dec 30, 2022
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models

LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models. Developers can reproduce these SOTA methods and build their own methods.

TuZheng 405 Jan 4, 2023