A Comparative Framework for Multimodal Recommender Systems

Preferred.AI

Last update: Jan 3, 2023

Related tags

Deep Learning collaborative-filtering matrix-factorization recommendation-system recommendation-engine recommender-system recommendation-algorithms multimodal-learning multimodal-representation

Overview

Cornac

Cornac is a comparative framework for multimodal recommender systems. It focuses on making it convenient to work with models leveraging auxiliary data (e.g., item descriptive text and image, social network, etc). Cornac enables fast experiments and straightforward implementations of new models. It is highly compatible with existing machine learning libraries (e.g., TensorFlow, PyTorch).

Quick Links

Installation

Currently, we are supporting Python 3. There are several ways to install Cornac:

From PyPI (you may need a C++ compiler):
```
pip3 install cornac
```
From Anaconda:
```
conda install cornac -c conda-forge
```

From the GitHub source (for latest updates):

pip3 install Cython
git clone https://github.com/PreferredAI/cornac.git
cd cornac
python3 setup.py install

Note:

Additional dependencies required by models are listed here.

Some algorithm implementations use OpenMP to support multi-threading. For Mac OS users, in order to run those algorithms efficiently, you might need to install gcc from Homebrew to have an OpenMP compiler:

brew install gcc | brew link gcc

Getting started: your first Cornac experiment

Flow of an Experiment in Cornac

import cornac
from cornac.eval_methods import RatioSplit
from cornac.models import MF, PMF, BPR
from cornac.metrics import MAE, RMSE, Precision, Recall, NDCG, AUC, MAP

# load the built-in MovieLens 100K and split the data based on ratio
ml_100k = cornac.datasets.movielens.load_feedback()
rs = RatioSplit(data=ml_100k, test_size=0.2, rating_threshold=4.0, seed=123)

# initialize models, here we are comparing: Biased MF, PMF, and BPR
models = [
    MF(k=10, max_iter=25, learning_rate=0.01, lambda_reg=0.02, use_bias=True, seed=123),
    PMF(k=10, max_iter=100, learning_rate=0.001, lambda_reg=0.001, seed=123),
    BPR(k=10, max_iter=200, learning_rate=0.001, lambda_reg=0.01, seed=123),
]

# define metrics to evaluate the models
metrics = [MAE(), RMSE(), Precision(k=10), Recall(k=10), NDCG(k=10), AUC(), MAP()]

# put it together in an experiment, voilà!
cornac.Experiment(eval_method=rs, models=models, metrics=metrics, user_based=True).run()

Output:

	MAE	RMSE	AUC	MAP	NDCG@10	Precision@10	Recall@10	Train (s)	Test (s)
MF	0.7430	0.8998	0.7445	0.0407	0.0479	0.0437	0.0352	0.13	1.57
PMF	0.7534	0.9138	0.7744	0.0491	0.0617	0.0533	0.0479	2.18	1.64
BPR	N/A	N/A	0.8695	0.0753	0.0975	0.0727	0.0891	3.74	1.49

For more details, please take a look at our examples as well as tutorials.

Models

The recommender models supported by Cornac are listed below. Why don't you join us to lengthen the list?

Year	Model and paper	Additional dependencies	Examples
2021	Bilateral Variational Autoencoder for Collaborative Filtering (BiVAECF), paper	requirements.txt	PreferredAI/bi-vae
2018	Collaborative Context Poisson Factorization (C2PF), paper	N/A	c2pf_exp.py
	Multi-Task Explainable Recommendation (MTER), paper	N/A	mter_exp.py
	Neural Attention Rating Regression with Review-level Explanations (NARRE), paper	requirements.txt	narre_example.py
	Probabilistic Collaborative Representation Learning (PCRL), paper	requirements.txt	pcrl_exp.py
	Variational Autoencoder for Collaborative Filtering (VAECF), paper	requirements.txt	vaecf_citeulike.py
2017	Collaborative Variational Autoencoder (CVAE), paper	requirements.txt	cvae_exp.py
	Conditional Variational Autoencoder for Collaborative Filtering (CVAECF), paper	requirements.txt	cvaecf_filmtrust.py
	Generalized Matrix Factorization (GMF), paper	requirements.txt	ncf_exp.py
	Indexable Bayesian Personalized Ranking (IBPR), paper	requirements.txt	ibpr_exp.py
	Matrix Co-Factorization (MCF), paper	N/A	mcf_office.py
	Multi-Layer Perceptron (MLP), paper	requirements.txt	ncf_exp.py
	Neural Matrix Factorization (NeuMF) / Neural Collaborative Filtering (NCF), paper	requirements.txt	ncf_exp.py
	Online Indexable Bayesian Personalized Ranking (Online IBPR), paper	requirements.txt
	Visual Matrix Factorization (VMF), paper	requirements.txt	vmf_clothing.py
2016	Collaborative Deep Ranking (CDR), paper	requirements.txt	cdr_exp.py
	Collaborative Ordinal Embedding (COE), paper	requirements.txt
	Convolutional Matrix Factorization (ConvMF), paper	requirements.txt	convmf_exp.py
	Spherical K-means (SKM), paper	N/A	skm_movielens.py
	Visual Bayesian Personalized Ranking (VBPR), paper	requirements.txt	vbpr_tradesy.py
2015	Collaborative Deep Learning (CDL), paper	requirements.txt	cdl_exp.py
	Hierarchical Poisson Factorization (HPF), paper	N/A	hpf_movielens.py
2014	Explicit Factor Model (EFM), paper	N/A	efm_exp.py
	Social Bayesian Personalized Ranking (SBPR), paper	N/A	sbpr_epinions.py
2013	Hidden Factors and Hidden Topics (HFT), paper	N/A	hft_exp.py
2012	Weighted Bayesian Personalized Ranking (WBPR), paper	N/A	bpr_netflix.py
2011	Collaborative Topic Modeling (CTR), paper	N/A	ctr_citeulike.py
Earlier	Baseline Only, paper	N/A	svd_exp.py
	Bayesian Personalized Ranking (BPR), paper	N/A	bpr_netflix.py
	Factorization Machines (FM), paper	Linux only	fm_example.py
	Global Average (GlobalAvg), paper	N/A	biased_mf.py
	Item K-Nearest-Neighbors (ItemKNN), paper	N/A	knn_movielens.py
	Matrix Factorization (MF), paper	N/A	biased_mf.py, given_data.py
	Maximum Margin Matrix Factorization (MMMF), paper	N/A	mmmf_exp.py
	Most Popular (MostPop), paper	N/A	bpr_netflix.py
	Non-negative Matrix Factorization (NMF), paper	N/A	nmf_exp.py
	Probabilistic Matrix Factorization (PMF), paper	N/A	pmf_ratio.py
	Singular Value Decomposition (SVD), paper	N/A	svd_exp.py
	Social Recommendation using PMF (SoRec), paper	N/A	sorec_filmtrust.py
	User K-Nearest-Neighbors (UserKNN), paper	N/A	knn_movielens.py
	Weighted Matrix Factorization (WMF), paper	requirements.txt	wmf_exp.py

Support

Your contributions at any level of the library are welcome. If you intend to contribute, please:

Fork the Cornac repository to your own account.
Make changes and create pull requests.

You can also post bug reports and feature requests in GitHub issues.

Citation

If you use Cornac in a scientific publication, we would appreciate citations to the following paper:

Cornac: A Comparative Framework for Multimodal Recommender Systems, Salah et al., JMLR 21, pp. 1-5, 2020.

Bibtex entry:

@article{cornac,
  author  = {Aghiles Salah and Quoc-Tuan Truong and Hady W. Lauw},
  title   = {Cornac: A Comparative Framework for Multimodal Recommender Systems},
  journal = {Journal of Machine Learning Research},
  year    = {2020},
  volume  = {21},
  number  = {95},
  pages   = {1-5},
  url     = {http://jmlr.org/papers/v21/19-805.html}
}

License

Apache License 2.0

Comments

[Expand my own dataset?]

Description

hi, thanks for developing this outstanding tools, and could use my own dataset instead of using the default dataset? If it is ok, how I replace the dataset?

Other Comments
help wanted question

opened by ZuoxiYang 17
Feature/stratified evaluation
Description

Add a new evaluation method (Stratified Evaluation) proposed here and update the documents accordingly (add new tutorial,...).

Related Issues

Checklist:

[ ] I have added tests.

[*] I have updated the documentation accordingly.

[*] I have updated README.md (if you are adding a new model).

[ ] I have updated examples/README.md (if you are adding a new example).

[ ] I have updated datasets/README.md (if you are adding a new dataset).
opened by amirj 13
[ASK]

Description

Great library! Is there a mechanism to remove already seen/liked/interacted-with items upon prediction for some model? MS Recommenders uses this.

Other Comments

opened by deychak 12
[ASK]The NDCG value seems a bit strange
Description

Hi，

Why are the NDCG values calculated by cornac.metrics.NDCG mostly at [0.01,0.1], while the results calculated by many recommendation system papers are mostly at [0.1,1]?

Are the two methods of calculating NDCG different?

What should I do if I want to reproduce the results of the paper?

BTW, cornac is awesome!

Other Comments
opened by zwhe99 10
[ASK] question about cornac with implicit dataset

Description

Hi,

When I used cornac with implicit datset, some evaluations make me puzzled(the "MAE" and "RMSE" were always 0, is it right or a bug?). Note that the implicit dataset in my experiments only have the positive feedback(the ratings is 1) while the negative feedbacks have beed removed.

when the epochs=20, the results were shown as follow:

when the epochs=40, the results were shown as follow:

when the epochs=60, the results were shown as follow:

the codes have been used were shown as follow:

Thanks for your reading and your excellent jobs in this tool!

In which platform does it happen?

How do we replicate the issue?

Expected behavior (i.e. solution)

Other Comments
question

opened by joy-more 10
Feature/propensity stratified evaluation tutorial
Description

I just added a new tutorial to represent propensity-based stratified evaluation.

Related Issues

Checklist:

[ ] I have added tests.

[ ] I have updated the documentation accordingly.

[ ] I have updated README.md (if you are adding a new model).

[ ] I have updated examples/README.md (if you are adding a new example).

[ ] I have updated datasets/README.md (if you are adding a new dataset).
opened by amirj 9

No module named 'cornac.utils.fast_sparse_funcs' when trying to run first.example under pycharm

Description

Hello, I use Ubunto version 20.04, and installed cornac using the pip3 option (under venv) and conda (under conda venv), and in both case, I am receiving this error when I want to run cornac, first_example.py:

Traceback (most recent call last):
  File "/home/yas/PycharmProjects/cornac/examples/first_example.py", line 17, in <module>
    import cornac
  File "/home/yas/PycharmProjects/cornac/cornac/__init__.py", line 16, in <module>
    from . import data
  File "/home/yas/PycharmProjects/cornac/cornac/data/__init__.py", line 18, in <module>
    from .text import TextModality, ReviewModality
  File "/home/yas/PycharmProjects/cornac/cornac/data/text.py", line 27, in <module>
    from ..utils import normalize
  File "/home/yas/PycharmProjects/cornac/cornac/utils/__init__.py", line 16, in <module>
    from .common import validate_format
  File "/home/yas/PycharmProjects/cornac/cornac/utils/common.py", line 21, in <module>
    from .fast_sparse_funcs import (
ModuleNotFoundError: No module named 'cornac.utils.fast_sparse_funcs'

Let me add that I use PyCharm. Any idea why the issue emerged and how it could be handled?

opened by yasdel 9

Fit a model using modalities

Fit using modalities

Hello,

First of all, I can't thank you enough for your efforts with Cornac, it is a really powerful tool. I'm trying to implement PCRL using Cornac. The experiments work as shown in examples. But I can't fit the model. There seems to be a lack of documentation on this area, I can't find how to do it in any page. Here's the issue:

I hope you can help me.

my very best,

Miguel Ângelo Rebelo

opened by miguelangelorebelo 8
[H] How to see per user evaluation result

Description

Thank you for putting up cornac, it is a very useful resource for pushing research.

I just wanted to ask if there is a way to see the per-user evaluation score for RS models? Let's assume, we do not want to average over user-obtained NDCGs but we want to compute a PDF on the results. How can we get user-related evaluation scores in order to aggregate the results in a specific manner (beyond simple average)? Thank you for your input.
question

opened by yasdel 8
NARRE: Neural Attentional Rating Regression with Review-level Explanations
Description

Related Issues

Checklist:

[ ] I have added tests.

[x] I have updated the documentation accordingly.

[x] I have updated README.md (if you are adding a new model).

[x] I have updated examples/README.md (if you are adding a new example).

[ ] I have updated datasets/README.md (if you are adding a new dataset).
opened by lthoang 8
[BUG] `exclude_unknowns=False` makes evaluation fail

Hello guys, great work, way better than Microsoft! I have a small issue, please read below:

Description

If I set exclude_unknowns=False then all the ratings/rankings fail with a similar error: e.g. IndexError('index 1840 is out of bounds for axis 0 with size 1840')

In which platform does it happen?

Linux 5.11.0-36-generic

How do we replicate the issue?

This is easy, just make sure to have non-zeros for: Number of unknown users = 5 Number of unknown items = 4 after when doing RatioSplit

Expected behavior (i.e. solution)

I would still expect to get the scores even when there are non-matching indices between train_set and test_set

opened by vlainic 7
Use of deprecated tensorflow method

Description

The WMF model cannot be used in Experiment, due to a depricated tensorflow method. The new method must be 'tf.random.set_seed()'. The error output is like:

/usr/local/lib/python3.8/dist-packages/cornac/models/wmf/recom_wmf.py in _fit_cf(self) 165 graph = tf.Graph() 166 with graph.as_default(): --> 167 tf.set_random_seed(self.seed) 168 model = Model( 169 n_users=n_users,

AttributeError: module 'tensorflow' has no attribute 'set_random_seed'

In which platform does it happen?

Found while using in google colab.

How do we replicate the issue?

Try to use the WMF model in an Experiment

solution

In line 167 of recom_wmf.py change 'tf.set_random_seed()' to 'tf.random.set_seed()'.

opened by yvonnerahnfeld 1
Why are the examples not running? Problem with import of cornac

I just installed cornac from github, liked described in the readme file. But now if I try to execute one of the examples, I always get the error: ModuleNotFoundError: No module named 'cornac.utils.fast_sparse_funcs' why does the import of cornac fail if everything is installed? On Colab within a Jupyter Notebook everything works fine, except the WMF model, because of a deprecated method. Because of this bug I`d like to use cornac on my laptop instead of running it from Colab, to fix this bug.

opened by yvonnerahnfeld 1
GNU library compatibility issues

Description

Dear cornac developers,

I am trying to use cornac within the server of my university. However, I get a compatibility issue when I import; cornac requires GNU libc version of 2.27, and ours is 2.25.

I wonder if you can suggest any work around. E.g., is there an older cornac version that requires an earlier version of libc?

Thank you in advance!

opened by SavvinaDaniil 0
z-scoREC Implementation
Description

Hi,

We introduced two novel collaborative filtering techniques for recommendation systems in cases of various cold-start situations and incomplete datasets.

Hakan Yilmazer, Selma Ayşe Özel, ImposeSVD: Incrementing PureSVD For Top-N Recommendations for Cold-Start Problems and Sparse Datasets, The Computer Journal, 2022;, bxac106, https://doi.org/10.1093/comjnl/bxac106.

The first model z-scoREC establishes an asymmetric weight matrix between items without using item meta-data and eradicates the disadvantages of neighborhood approaches by automatic determination of threshold values.

The implementation of z-scoREC model is implemented in this PR.

Related Issues

Checklist:

[ ] I have added tests.

[x] I have updated the documentation accordingly.

[x] I have updated README.md (if you are adding a new model).

[x] I have updated examples/README.md (if you are adding a new example).
opened by yilmazerhakan 0
[ASK] For BiVAECF, the interactive items in trainset are not removed during the evaluation phase.
Description

I check the recommender.rank() and bivaecf.score() and find that the interactive items in trainset are not removed during the evaluation phase.

When recommending top-n items with highest score from whole item space, whether it's necessary to remove the interactive item in trainset or give these imteractive item a smallest value (or zero)?

I give these interactive items zero value by adding follow codes in ***bivaecf.score()***,

if item_idx is None: ... train_mat = self.train_set.csr_matrix csr_row = train_mat.getrow(user_idx) pos_items = [item_idx for (item_idx, rating) in zip(csr_row.indices, csr_row.data) if rating >= 4.0] known_item_scores[pos_items] = 0. return known_item_scores

From the results on ml-100k, there was a significant improvement for each metrics,

| Model | NDCG@50 | Precision@50 |Recall@50 | | ------------- | ------------- | ------------- | ------------- | | BiVAECF | 0.2378 | 0.0739 | 0.4288 | | BiVAECF with setting zero value | 0.3392 | 0.1048 | 0.5233 |

Other Comments
opened by georgeguo-cn 6

Releases(v1.14.2)

v1.14.2(Feb 19, 2022)

Add mean and std values per metric in CVResult (#467)
Source code(tar.gz)
Source code(zip)
v1.14.1(Sep 26, 2021)

Change name of lambda args of SoRec model (#438)
Source code(tar.gz)
Source code(zip)
v1.14.0(Sep 17, 2021)
Improvements

Fix sign in AMR model (#429)

Add model selection strategy (best or last) for NARRE model (#435)

Speed up building kNN graph in GraphModality (#436)

add_modalities method for separately built modalities (#437)

Source code(tar.gz)
Source code(zip)
v1.13.5(Jul 13, 2021)

Fix pre-built wheels for MacOS
Source code(tar.gz)
Source code(zip)
v1.13.4(Jul 12, 2021)

Use ubuntu-18.04 CI image for better compatibility builds
Source code(tar.gz)
Source code(zip)
v1.13.3(Jul 12, 2021)

Add missing numpy~=1.19.0 in publish action
Source code(tar.gz)
Source code(zip)
v1.13.2(Jul 12, 2021)

Fix numpy.ndarray size changed from NumPy v1.20
Source code(tar.gz)
Source code(zip)
v1.13.1(Jul 11, 2021)

Support Python 3.9 with pre-built wheels
Source code(tar.gz)
Source code(zip)
v1.13.0f(Jul 9, 2021)
New models

CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation (#415)

Adversarial Training Towards Robust Multimedia Recommender System (#420)

New feature

Propensity Stratified Evaluation Method (#403) and tutorial (#408)

Improvements

Add GitHub actions for CI testing and pypi upload

Update model names

Source code(tar.gz)
Source code(zip)
v1.12.0(Mar 30, 2021)
New models

Explainable Recommendation with Comparative Constraints on Product Aspects (ComparER) (#392)

Improvements

Fix some broken links

Fix linux platform check (#397)

Source code(tar.gz)
Source code(zip)
v1.11.0(Jan 29, 2021)
New model

Factorization Machines (FM) (#343)

Improvements

Update HPF initialization to use seed value (#380)

Add example Poisson Factorization vs BPR on MovieLens data (#381)

Add clarification when test_size and val_size > 1 (#382)

Shorten README (#383)

Update NARRE model (#384)

Import filmtrust in datasets module (#388)

Source code(tar.gz)
Source code(zip)
v1.10.0(Dec 28, 2020)
New models

NARRE: Neural Attentional Rating Regression with Review-level Explanations (#356)

BiVAECF: Bilateral Variational Autoencoder for Collaborative Filtering (#375)

Improvements

Update SKMeans to account for max_iter and seed (#377)

Add SKMeans example (#378)

Add function to generate gamma variates (#379)

Source code(tar.gz)
Source code(zip)
v1.9.0(Nov 4, 2020)
New features

StratifiedSplit evaluation method (#367)

Source code(tar.gz)
Source code(zip)
v1.8.0(Oct 15, 2020)
New changes

Update link to download the Film Trust dataset (#355)

Add review-level for review modality (#353)

Add Amazon Digital Music dataset including ratings and reviews (#357)

Fix rank() function to work with arbitrary item_indices (#363)

Source code(tar.gz)
Source code(zip)
v1.7.1(Jul 15, 2020)
New changes

Fix ConvMF loss info (#349)

Source code(tar.gz)
Source code(zip)
v1.7.0(Jul 15, 2020)
New changes

Add Review Modality (#344)

Conditional VAE for Collaborative Filtering (CVAECF) model (#345)

Update init args and fix dimension mismatch in ConvMF model (#348)

Source code(tar.gz)
Source code(zip)
v1.6.1(Jun 6, 2020)
New improvements

Fix bug of Z dims in SoRec (#340)

Add a reference to topic model in CTR (#341)

Source code(tar.gz)
Source code(zip)
v1.6.0(May 29, 2020)
New features

Add Mean Average Precision (MAP) metric (#338)

Source code(tar.gz)
Source code(zip)
v1.5.2(May 11, 2020)
New improvements

Fix bug in graph modality (#333)

Update models to support clone function (#334)

Add lambda_reg argument for NMF model (#335)

Use tqdm.auto for multiple environment compatibility (#336)

Source code(tar.gz)
Source code(zip)
v1.5.1(May 6, 2020)
New improvements

Improve the efficiency of KNN methods (#331)

Use tqdm.auto for compatibility on Jupyter notebook env (#332)

Source code(tar.gz)
Source code(zip)
v1.5.0(May 1, 2020)
New models

MTER model is faster with Cython (#320)

Neighborhood-based methods (UserKNN and ItemKNN) (#329)

Source code(tar.gz)
Source code(zip)
v1.4.1(Feb 12, 2020)
New features and improvements

Refactor code in examples (#317)

CI tools support Python 3.7

Update and test all tutorials and examples

Source code(tar.gz)
Source code(zip)
v1.4.0(Feb 6, 2020)
New models and datasets

Weighted Bayesian Personalized Ranking (WBPR) model (#309)

Maximum Margin Matrix Factorization (MMMF) model (#310)

New features and improvements

Reset random number generator for reproducibility (#301)

Fix issue in NCRR metric (#313)

Use C++ Boost Random library for reproducibility across platforms (#315)

Support model saving and loading (#316)

Source code(tar.gz)
Source code(zip)
v1.3.1(Jan 28, 2020)
New features and improvements

Add default attributes total_users and total_items to Dataset (#300)

Source code(tar.gz)
Source code(zip)
v1.3.0(Jan 23, 2020)
New models and datasets

MovieLens 10M and 20M datasets (#291)

New features and improvements

Standardize datasets load_feedback() API (#278)

Add hyperopt for hyper-parameter tuning (#286)

Show validation results (optional) if exists (#289)

Update Recommender.rank() to support unknown item scores (#283)

Support multiple values of K for ranking metrics (#297)

Tutorial on how to work with auxiliary data (#264)

Tutorial on hyperparameter search for VAECF (#290)

Examples for VAECF, VMF, and SoRec (#272, #276, #287)

Source code(tar.gz)
Source code(zip)
v1.2.2(Nov 29, 2019)
New features and improvements

Add FilmTrust dataset (#266)

Update MCF, C2PF, and SoRec models for the compatibility (#261, #262)

Support retrieving node degree in GraphModality (#267)

Source code(tar.gz)
Source code(zip)
v1.2.1(Nov 11, 2019)
New features and improvements

Support UIRT data format (#252)

Fix bug in AUC metric (#256)

Tutorial on contributing an evaluation metric (#258)

Source code(tar.gz)
Source code(zip)
v1.2.0(Sep 26, 2019)
New models

Multi-Task Explainable Recommendation (MTER) (#238)

Most Popular (MostPop) ranking baseline (#244)

Global Average (GlobalAvg) rating baseline (#245)

New features and improvements

Update VAECF model (#237)

normalize() function in utils (#248)

TfidfVectorizer in TextModality (#249)

Source code(tar.gz)
Source code(zip)
v1.1.2(Sep 18, 2019)
New features and improvements

Improve documentation of models (#239)

Update BPR model with removing scaling in score() function (#240)

Source code(tar.gz)
Source code(zip)
v1.1.1(Sep 16, 2019)
New features and improvements

Fix bug of list multiplication in ranking_eval() function (#236)

Source code(tar.gz)
Source code(zip)

A Comparative Framework for Multimodal Recommender Systems

Related tags

Overview

Cornac

Quick Links

Installation

Getting started: your first Cornac experiment

Models

Support

Citation

License

Comments

Description

Other Comments

Description

Related Issues

Checklist:

Description

Other Comments

Description

Other Comments

Description

In which platform does it happen?

How do we replicate the issue?

Expected behavior (i.e. solution)

Other Comments

Description

Related Issues

Checklist:

Description

Fit using modalities

Description

Description

Related Issues

Checklist:

Description

In which platform does it happen?

How do we replicate the issue?

Expected behavior (i.e. solution)

Description

In which platform does it happen?

How do we replicate the issue?

solution

Description

Description

Related Issues

Checklist:

Description

Other Comments

Releases(v1.14.2)

v1.14.2(Feb 19, 2022)

v1.14.1(Sep 26, 2021)

v1.14.0(Sep 17, 2021)

Improvements

v1.13.5(Jul 13, 2021)

v1.13.4(Jul 12, 2021)

v1.13.3(Jul 12, 2021)

v1.13.2(Jul 12, 2021)

v1.13.1(Jul 11, 2021)

v1.13.0f(Jul 9, 2021)

New models

New feature

Improvements

v1.12.0(Mar 30, 2021)

New models

Improvements

v1.11.0(Jan 29, 2021)

New model

Improvements

v1.10.0(Dec 28, 2020)

New models

Improvements

v1.9.0(Nov 4, 2020)

New features

v1.8.0(Oct 15, 2020)

New changes

v1.7.1(Jul 15, 2020)

New changes

v1.7.0(Jul 15, 2020)

New changes