Qlib is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment. With Qlib, you can easily try your ideas to create better Quant investment strategies.

Microsoft

Last update: Dec 31, 2022

Related tags

Finance python platform finance machine-learning fintech quant quantitative-finance investment stock-data algorithmic-trading quantitative-trading quant-dataset quant-models auto-quant

Overview

Qlib is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment.

It contains the full ML pipeline of data processing, model training, back-testing; and covers the entire chain of quantitative investment: alpha seeking, risk modeling, portfolio optimization, and order execution.

With Qlib, users can easily try ideas to create better Quant investment strategies.

For more details, please refer to our paper "Qlib: An AI-oriented Quantitative Investment Platform".

News and Plans
Framework of Qlib
Quick Start
Quant Model Zoo
- Run a single model
- Run multiple models
Quant Dataset Zoo
More About Qlib
Offline Mode and Online Mode
- Performance of Qlib Data Server
Related Reports
Contact Us
Contributing

News And Plans

New features under development(order by estimated release time). Your feedbacks about the features are very important.

Feature	Status
Online serving and automatic model rolling	Under review: https://github.com/microsoft/qlib/pull/290
Planning-based portfolio optimization	Under review: https://github.com/microsoft/qlib/pull/280
Fund data supporting and analysis	Under review: https://github.com/microsoft/qlib/pull/292
Point-in-Time database	Under review: https://github.com/microsoft/qlib/pull/343
High-frequency trading	Initial opensource version under development
Meta-Learning-based data selection	Initial opensource version under development

Recent released features

Feature	Status
DoubleEnsemble Model	Released https://github.com/microsoft/qlib/pull/286
High-frequency data processing example	Released https://github.com/microsoft/qlib/pull/257
High-frequency trading example	Part of code released https://github.com/microsoft/qlib/pull/227
High-frequency data(1min)	Released https://github.com/microsoft/qlib/pull/221
Tabnet Model	Released https://github.com/microsoft/qlib/pull/205

Features released before 2021 are not listed here.

Framework of Qlib

At the module level, Qlib is a platform that consists of the above components. The components are designed as loose-coupled modules, and each component could be used stand-alone.

Name	Description
`Infrastructure` layer	`Infrastructure` layer provides underlying support for Quant research. `DataServer` provides a high-performance infrastructure for users to manage and retrieve raw data. `Trainer` provides a flexible interface to control the training process of models, which enable algorithms to control the training process.
`Workflow` layer	`Workflow` layer covers the whole workflow of quantitative investment. `Information Extractor` extracts data for models. `Forecast Model` focuses on producing all kinds of forecast signals (e.g. alpha, risk) for other modules. With these signals `Portfolio Generator` will generate the target portfolio and produce orders to be executed by `Order Executor`.
`Interface` layer	`Interface` layer tries to present a user-friendly interface for the underlying system. `Analyser` module will provide users detailed analysis reports of forecasting signals, portfolios and execution results

The modules with hand-drawn style are under development and will be released in the future.
The modules with dashed borders are highly user-customizable and extendible.

Quick Start

This quick start guide tries to demonstrate

It's very easy to build a complete Quant research workflow and try your ideas with Qlib.
Though with public data and simple models, machine learning technologies work very well in practical Quant investment.

Here is a quick demo shows how to install Qlib, and run LightGBM with qrun. But, please make sure you have already prepared the data following the instruction.

Installation

This table demonstrates the supported Python version of Qlib:

	install with pip	install from source	plot
Python 3.6	✔️	✔️ (only with `Anaconda`)	✔️
Python 3.7	✔️	✔️	✔️
Python 3.8	✔️	✔️	✔️
Python 3.9	❌	✔️	❌

Note:

Please pay attention that installing cython in Python 3.6 will raise some error when installing Qlib from source. If users use Python 3.6 on their machines, it is recommended to upgrade Python to version 3.7 or use conda's Python to install Qlib from source.
For Python 3.9, Qlib supports running workflows such as training models, doing backtest and plot most of the related figures (those included in notebook). However, plotting for the model performance is not supported for now and we will fix this when the dependent packages are upgraded in the future.

Install with pip

Users can easily install Qlib by pip according to the following command.

  pip install pyqlib

Note: pip will install the latest stable qlib. However, the main branch of qlib is in active development. If you want to test the latest scripts or functions in the main branch. Please install qlib with the methods below.

Install from source

Also, users can install the latest dev version Qlib by the source code according to the following steps:

Before installing Qlib from source, users need to install some dependencies:
```
pip install numpy
pip install --upgrade  cython
```
Clone the repository and install Qlib as follows.
- If you haven't installed qlib by the command pip install pyqlib before:
```
git clone https://github.com/microsoft/qlib.git && cd qlib
python setup.py install
```
- If you have already installed the stable version by the command pip install pyqlib:
```
git clone https://github.com/microsoft/qlib.git && cd qlib
pip install .
```
Note: Only the command pip install . can overwrite the stable version installed by pip install pyqlib, while the command python setup.py install can't.

Tips: If you fail to install Qlib or run the examples in your environment, comparing your steps and the CI workflow may help you find the problem.

Data Preparation

Load and prepare data by running the following code:

# get 1d data
python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn

# get 1min data
python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data_1min --region cn --interval 1min

This dataset is created by public data collected by crawler scripts, which have been released in the same repository. Users could create the same dataset with it.

Please pay ATTENTION that the data is collected from Yahoo Finance, and the data might not be perfect. We recommend users to prepare their own data if they have a high-quality dataset. For more information, users can refer to the related document.

Auto Quant Research Workflow

Qlib provides a tool named qrun to run the whole workflow automatically (including building dataset, training models, backtest and evaluation). You can start an auto quant research workflow and have a graphical reports analysis according to the following steps:

Quant Research Workflow: Run qrun with lightgbm workflow config (workflow_config_lightgbm_Alpha158.yaml as following.

  cd examples  # Avoid running program under the directory contains `qlib`
  qrun benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml

If users want to use qrun under debug mode, please use the following command:

python -m pdb qlib/workflow/cli.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml

The result of qrun is as follows, please refer to Intraday Trading for more details about the result.

'The following are analysis results of the excess return without cost.'
                       risk
mean               0.000708
std                0.005626
annualized_return  0.178316
information_ratio  1.996555
max_drawdown      -0.081806
'The following are analysis results of the excess return with cost.'
                       risk
mean               0.000512
std                0.005626
annualized_return  0.128982
information_ratio  1.444287
max_drawdown      -0.091078

Here are detailed documents for qrun and workflow.

Graphical Reports Analysis: Run examples/workflow_by_code.ipynb with jupyter notebook to get graphical reports
- Forecasting signal (model prediction) analysis
  - Cumulative Return of groups
  - Return distribution
  - Information Coefficient (IC)
  - Auto Correlation of forecasting signal (model prediction)
- Portfolio analysis
  - Backtest return
- Explanation of above results

Building Customized Quant Research Workflow by Code

The automatic workflow may not suit the research workflow of all Quant researchers. To support a flexible Quant research workflow, Qlib also provides a modularized interface to allow researchers to build their own workflow by code. Here is a demo for customized Quant research workflow by code.

Quant Model Zoo

Here is a list of models built on Qlib.

Your PR of new Quant models is highly welcomed.

The performance of each model on the Alpha158 and Alpha360 dataset can be found here.

Run a single model

All the models listed above are runnable with Qlib. Users can find the config files we provide and some details about the model through the benchmarks folder. More information can be retrieved at the model files listed above.

Qlib provides three different ways to run a single model, users can pick the one that fits their cases best:

Users can use the tool qrun mentioned above to run a model's workflow based from a config file.
Users can create a workflow_by_code python script based on the one listed in the examples folder.
Users can use the script run_all_model.py listed in the examples folder to run a model. Here is an example of the specific shell command to be used: python run_all_model.py --models=lightgbm, where the --models arguments can take any number of models listed above(the available models can be found in benchmarks). For more use cases, please refer to the file's docstrings.

Run multiple models

Qlib also provides a script run_all_model.py which can run multiple models for several iterations. (Note: the script only support Linux for now. Other OS will be supported in the future. Besides, it doesn't support parrallel running the same model for multiple times as well, and this will be fixed in the future development too.)

The script will create a unique virtual environment for each model, and delete the environments after training. Thus, only experiment results such as IC and backtest results will be generated and stored.

Here is an example of running all the models for 10 iterations:

python run_all_model.py 10

It also provides the API to run specific models at once. For more use cases, please refer to the file's docstrings.

Quant Dataset Zoo

Dataset plays a very important role in Quant. Here is a list of the datasets built on Qlib:

Dataset	US Market	China Market
Alpha360	√	√
Alpha158	√	√

Here is a tutorial to build dataset with Qlib. Your PR to build new Quant dataset is highly welcomed.

More About Qlib

The detailed documents are organized in docs. Sphinx and the readthedocs theme is required to build the documentation in html formats.

cd docs/
conda install sphinx sphinx_rtd_theme -y
# Otherwise, you can install them with pip
# pip install sphinx sphinx_rtd_theme
make html

You can also view the latest document online directly.

Qlib is in active and continuing development. Our plan is in the roadmap, which is managed as a github project.

Offline Mode and Online Mode

The data server of Qlib can either deployed as Offline mode or Online mode. The default mode is offline mode.

Under Offline mode, the data will be deployed locally.

Under Online mode, the data will be deployed as a shared data service. The data and their cache will be shared by all the clients. The data retrieval performance is expected to be improved due to a higher rate of cache hits. It will consume less disk space, too. The documents of the online mode can be found in Qlib-Server. The online mode can be deployed automatically with Azure CLI based scripts. The source code of online data server can be found in Qlib-Server repository.

Performance of Qlib Data Server

The performance of data processing is important to data-driven methods like AI technologies. As an AI-oriented platform, Qlib provides a solution for data storage and data processing. To demonstrate the performance of Qlib data server, we compare it with several other data storage solutions.

We evaluate the performance of several storage solutions by finishing the same task, which creates a dataset (14 features/factors) from the basic OHLCV daily data of a stock market (800 stocks each day from 2007 to 2020). The task involves data queries and processing.

	HDF5	MySQL	MongoDB	InfluxDB	Qlib -E -D	Qlib +E -D	Qlib +E +D
Total (1CPU) (seconds)	184.4±3.7	365.3±7.5	253.6±6.7	368.2±3.6	147.0±8.8	47.6±1.0	7.4±0.3
Total (64CPU) (seconds)					8.8±0.6	4.2±0.2

+(-)E indicates with (out) ExpressionCache
+(-)D indicates with (out) DatasetCache

Most general-purpose databases take too much time to load data. After looking into the underlying implementation, we find that data go through too many layers of interfaces and unnecessary format transformations in general-purpose database solutions. Such overheads greatly slow down the data loading process. Qlib data are stored in a compact format, which is efficient to be combined into arrays for scientific computation.

Related Reports

Contact Us

If you have any issues, please create issue here or send messages in gitter.
If you want to make contributions to Qlib, please create pull requests.
For other reasons, you are welcome to contact us by email([email protected]).
- We are recruiting new members(both FTEs and interns), your resumes are welcome!

Join IM discussion groups:

Gitter

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the right to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Comments

请问在使用自己的数据集时如果生成例如csi300.txt的instrument

❓ Questions and Help

We sincerely suggest you to carefully read the documentation of our library as well as the official paper. After that, if you still feel puzzled, please describe the question clearly under this issue.

您好！我在尝试引入自己的数据，使用dump_bin.py生成features，calendars和instruments，但是instruments中没有csi300和csi100.在跑的时候会报错csi300.txt不存在，请问这个instruments中的csi300如何生成，谢谢！
question stale

opened by TompaBay 20
Error when running LightGBM_Alpha158 model with minimal custom data
🐛 Bug Description

I try to import my own mini csv data (2 symbol and sh000300 index), and run the same model as demo:

qrun benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml

However it gives this error:

TypeError: Can only swap levels on a hierarchical axis.

To Reproduce

Steps to reproduce the behavior:

import 3 data series into own data folder cn_1d through csv import.

000615 2008-01-02 2021-04-30 603919 2016-03-10 2021-04-30 SH000300 2008-01-02 2021-04-30

Copy csi300.txt to own data folder subfolder cn_1d\instruments

Update workflow_config_lightgbm_Alpha158.yaml to point to own data folder cn_1d

run benchmark:

qrun benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml

Expected Behavior

Similar analysis result as running on demo cn 1d data, with reduced data set.

Screenshot

Environment

Note: User could run cd scripts && python collect_info.py all under project directory to get system information and paste them here directly.

Windows AMD64 Windows-10-10.0.19041-SP0 10.0.19041

Python version: 3.8.8 (default, Feb 24 2021, 15:54:32) [MSC v.1928 64 bit (AMD64)]

Qlib version: 0.6.3.99 numpy==1.20.3 pandas==1.2.4 scipy==1.6.0 requests==2.25.1 sacred==0.8.2 python-socketio==3.1.2 redis==3.5.3 python-redis-lock==3.7.0 schedule==1.0.0 cvxpy==1.0.21 hyperopt==0.1.1 fire==0.4.0 statsmodels==0.12.1 xlrd==2.0.1 plotly==4.12.0 matplotlib==3.1.3 tables==3.6.1 pyyaml==5.4.1 mlflow==1.14.0 tqdm==4.61.0 loguru==0.5.3 lightgbm==3.1.1 tornado==6.1 joblib==1.0.1 fire==0.4.0 ruamel.yaml==0.16.12

Commit number (optional, please provide it if you are using the dev version): cbbf6cd82260e533662a7433ba72cf3391347619

Additional Notes
bug
opened by arisliang 20
如何使用低频数据

❓ Questions and Help

我将日频数据只取偶数交易日的值，去掉奇数交易日的数据，然后dump_all成qlib类型的数据，然后导入到实例程序workflow_by_code.ipynb里进行训练，在train model过程中出现报错。但我看您之前回答某个问题中说“Qlib didn't hard code the data frequency.”，请问一下如果数据不是日频的，应该如何使用（包括导入，训练，回测）？谢谢
question

opened by panshuaiyin 17
Would you introduce about library of mlflow?

❓ Questions and Help

What is the function of this library？#MLFLOW this is just a machinelearning frame，when i run one of benchmrks ,but it cound't it.
question stale

opened by vinsvison 17
confusions about dump_bin.py

In https://github.com/microsoft/qlib/blob/main/scripts/dump_bin.py#L352 and https://github.com/microsoft/qlib/blob/main/scripts/dump_bin.py#L315 There are two classes that I don't know how to use, class DumpDataFix(DumpDataAll), and class DumpDataUpdate(DumpDataBase), can anyone explain it for me? I am looking for some ways to update dump data, but did not succeed till now.
question stale

opened by zymmatt 16
Add two transformer models via upload
Add a naive transformer model and an improved transformer model.

Description

The tested successful requirement is Python 3.6/3.7/3.8 and Pytorch 1.12/1.2.

The naive transformer implemented here for financial time series prediction follows the paper "Attention is all you need": Given the input (N, T, F),

An embedding layer that maps the input (N, T, F) to representation (N, T, F’);

A positional encoding layer that adds the positional sigmoid;

An encoder that consists of several encoding layers, each of which uses a self-attention layer as the computing module (function of query, key, and value).

A decoder that consists of an MLP (or a Linear layer) that maps the representation of the last time (N, 1, F') into output (N, 1).

The improved transformer is a simple self-designed transformer (based on the paper 'SLGT: Self-adaptive Local-global aware Transformer for Sequential Recommendation', which is submitted to a conference and will be available on ArXiv soon). Localformer imports 1-dimensional convolutional layers besides the encoder layer as a locality inductive bias to supplement the long-term dependent self-attention module, which updates the representation of sequence at each time locally. Specifically, the input representation that passes through each encoder layer (self-attention layer) will be the original input adds (+) the output of the input passing through an extra 1-d convolutional layer. For example, if the encoder originally contains three self-attention layers attn-attn-attn, it will now be conv-attn-conv-attn-conv-attn. After the transformer module, a GRU is added to further aggregate the representation with sequential inductive bias (provided by the RNN layers).

Motivation and Context

It adds two famous transformer models for customers to select, besides other base models that Qlib already contains. The model performance reaches a 1.47 information ratio, which is fairly high. The improved version transformer adds convolution and RNN to supplement inductive bias, which is simple but effective.

How Has This Been Tested?

[ ] Pass the test by running: qrun benchmarks/Transformer/workflow_config_localformer_Alpha158.yaml under upper directory of qlib, where 'workflow_config_localformer_Alpha158.yaml' only needs to change this line of code 'task: model: class: LocalformerModel' or 'task: model: class: TransformerModel'.

[ ] The performances of the two models are described above.

Screenshots of Test Results (if appropriate):

Transformer Results on Alpha158: ''' 'IC': 0.03186587768611013, 'ICIR': 0.2556910881045764, 'Rank IC': 0.04735251936658551, 'Rank ICIR': 0.388378955424602

'The following are analysis results of the excess return without cost.' risk mean 0.000309 std 0.004209 annualized_return 0.077839 information_ratio 1.164993 max_drawdown -0.106215

'The following are analysis results of the excess return with cost.' risk mean 0.000126 std 0.004209 annualized_return 0.031707 information_ratio 0.474567 max_drawdown -0.131948

Transformer Results on Alpha360: {'IC': 0.011659216755690713, 'ICIR': 0.07383408561758713, 'Rank IC': 0.03505118059955821, 'Rank ICIR': 0.2453042675836217} 'The following are analysis results of the excess return without cost.' risk mean 0.000026 std 0.005318 annualized_return 0.006658 information_ratio 0.078865 max_drawdown -0.104203

Localformer Results on Alpha158: {'IC': 0.037426503365732174, 'ICIR': 0.28977883455541603, 'Rank IC': 0.04659889541774283, 'Rank ICIR': 0.373569340092482}

'The following are analysis results of the excess return without cost.' risk mean 0.000381 std 0.004109 annualized_return 0.096066 information_ratio 1.472729 max_drawdown -0.094917

'The following are analysis results of the excess return with cost.' risk mean 0.000213 std 0.004111 annualized_return 0.053630 information_ratio 0.821711 max_drawdown -0.113694

Localformer Results on Alpha360: {'IC': 0.03766845905185995, 'ICIR': 0.26793394150788935, 'Rank IC': 0.0530091645633088, 'Rank ICIR': 0.40090294387953357} 'The following are analysis results of the excess return without cost.' risk mean 0.000131 std 0.004943 annualized_return 0.033129 information_ratio 0.422228 max_drawdown -0.127502

Types of changes

[ ] Fix bugs

[x] Add new feature

[ ] Update documentation
opened by yingtaoluo 15
about qlib data

If you use the pre-reinstatement price，it will imply future information.Howerer,If you use the real price,there will be problems in the calculation of return.Which kind of raw data is used in qilb?
question stale

opened by ZFgan 15
Bug fix for Rank and WMA operators
Description

change the scaling in the inner function of Rank operator from len(x1) to 100

add 1 on the non-normalized weights of WMA operators

Motivation and Context

For 1): I would guess the goal is to scale the ranking result between 0 and 1. According to the scipy document, percentileofscore

Compute the percentile rank of a score relative to a list of scores. A percentileofscore of, for example, 80% means that 80% of the scores in a are below the given score.

it doesn't make sense to divide by length to array size, instead, should divide by 100.

For 2): The common convention of linear weighted MA with window size d should be weighted by d, d-1, ..., 1, instead of d-1, d-2, ..., 0, see e.g. https://www.fidelity.com/learning-center/trading-investing/technical-analysis/technical-indicator-guide/wma and the decay_linear function in famous 101 Formulaic Alphas by Zura Kakushadze from WorldQuant. Though convention problem is arguable I think it's better to be more intuitive.

How Has This Been Tested?

[x] Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.

[ ] If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

Pipeline test:

Your own tests:

Types of changes

[x] Fix bugs

[ ] Add new feature

[ ] Update documentation

bug
opened by qianyun210603 13
Label does not match calculation from sorce data

❓ Questions and Help

Tried to understand how Qlib works and hit this issue: I calculated a label by Excel and compared it with the same label dumped from Qlib code, by same stock instrument in same year. They did NOT match, not evel close. I know even the 'source data' from Yahoo Finance was pre-processed by some notmalization but since my lebel involves only the close feature, it is supposed to remain unchanged through the process. See my screenshot below.

In the left marked in yello is the source data calculation in Excel, data was fetched by D.feature and stored in csv, dumped to .bin then finally feed into 'provider_uri'; in the right marked in blue is the dump from DatasetH.prepare() by Qlib code. Please see both labels have same calculating expression but totally different values.

So why am I seeing this?
question stale

opened by hensejiang 13
Qlib run into error by joblib or multiprocessing

❓ Questions and Help

I was trying to develop some alpha following the doc at https://qlib.readthedocs.io/en/latest/advanced/alpha.html The code was pasted from the example as:

if name == 'main': from qlib.data.dataset.loader import QlibDataLoader#MACD_EXP = '(EMA($close, 12) - EMA($close, 26))/$close - EMA((EMA($close, 12) - EMA($close, 26))/$close, 9)/$close' close_EXP = '($close)'` fields = [close_EXP] # close itself names = ['close_itself'] print('label test is trying to set label') labels = ['((Ref($close, -3) + Ref($close, -2) + Ref($close, -1)) / 3) / Ref($close) - 1'] # set label to IDC label_names = ['LABEL_IDC'] data_loader_config = { "feature": (fields, names), "label": (labels, label_names), } print('label test is trying to load data') data_loader = QlibDataLoader(config=data_loader_config) df = data_loader.load(instruments='csi300', start_time='2010-01-01', end_time='2010-01-31') df.to_csv(os.path.join(model_saving_uri,'label_mean-3_try.csv')) print(df) print('label test data is done')

print('done...')

Result from running this simple test is a long error message:

TypeError Traceback (most recent call last) ~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset_init_.py in init(self, handler, segments, **kwargs) 113 } 114 """ --> 115 self.handler: DataHandler = init_instance_by_config(handler, accept_types=DataHandler) 116 self.segments = segments.copy() 117 self.fetch_kwargs = {}

~\AppData\Roaming\Python\Python38\site-packages\qlib\utils_init_.py in init_instance_by_config(config, default_module, accept_types, try_kwargs, **kwargs) 334 # 1: XXX() got multiple values for keyword argument 'YYY' 335 # 2: `XXX() got an unexpected keyword argument 'YYY' --> 336 return klass(**cls_kwargs, **kwargs) 337 338

~\AppData\Roaming\Python\Python38\site-packages\qlib\contrib\data\handler.py in init(self, instruments, start_time, end_time, freq, infer_processors, learn_processors, fit_start_time, fit_end_time, filter_pipe, inst_processor, **kwargs) 78 } 79 ---> 80 super().init( 81 instruments=instruments, 82 start_time=start_time,

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\handler.py in init(self, instruments, start_time, end_time, data_loader, infer_processors, learn_processors, shared_processors, process_type, drop_raw, **kwargs) 387 self.process_type = process_type 388 self.drop_raw = drop_raw --> 389 super().init(instruments, start_time, end_time, data_loader, **kwargs) 390 391 def get_all_processors(self):

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\handler.py in init(self, instruments, start_time, end_time, data_loader, init_data, fetch_orig) 103 if init_data: 104 with TimeInspector.logt("Init data"): --> 105 self.setup_data() 106 super().init() 107

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\handler.py in setup_data(self, init_type, **kwargs) 523 """ 524 # init raw data --> 525 super().setup_data(**kwargs) 526 527 with TimeInspector.logt("fit & process data"):

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\handler.py in setup_data(self, enable_cache) 147 with TimeInspector.logt("Loading data"): 148 # make sure the fetch method is based on a index-sorted pd.DataFrame --> 149 self._data = lazy_sort_index(self.data_loader.load(self.instruments, self.start_time, self.end_time)) 150 # TODO: cache 151

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\loader.py in load(self, instruments, start_time, end_time) 135 if self.is_group: 136 df = pd.concat( --> 137 { 138 grp: self.load_group_df(instruments, exprs, names, start_time, end_time, grp) 139 for grp, (exprs, names) in self.fields.items()

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\loader.py in (.0) 136 df = pd.concat( 137 { --> 138 grp: self.load_group_df(instruments, exprs, names, start_time, end_time, grp) 139 for grp, (exprs, names) in self.fields.items() 140 },

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\dataset\loader.py in load_group_df(self, instruments, exprs, names, start_time, end_time, gp_name) 211 212 freq = self.freq[gp_name] if isinstance(self.freq, dict) else self.freq --> 213 df = D.features( 214 instruments, exprs, start_time, end_time, freq=freq, inst_processors=self.inst_processor.get(gp_name, []) 215 )

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\data.py in features(self, instruments, fields, start_time, end_time, freq, disk_cache, inst_processors) 1014 ) 1015 except TypeError: -> 1016 return DatasetD.dataset(instruments, fields, start_time, end_time, freq, inst_processors=inst_processors) 1017 1018

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\data.py in dataset(self, instruments, fields, start_time, end_time, freq, inst_processors) 749 end_time = cal[-1] 750 --> 751 data = self.dataset_processor( 752 instruments_d, column_names, start_time, end_time, freq, inst_processors=inst_processors 753 )

~\AppData\Roaming\Python\Python38\site-packages\qlib\data\data.py in dataset_processor(instruments_d, column_names, start_time, end_time, freq, inst_processors) 522 zip( 523 inst_l, --> 524 ParallelExt(n_jobs=workers, backend=C.joblib_backend, maxtasksperchild=C.maxtasksperchild)(task_l), 525 ) 526 )

~\AppData\Roaming\Python\Python38\site-packages\joblib\parallel.py in call(self, iterable) 1054 1055 with self._backend.retrieval_context(): -> 1056 self.retrieve() 1057 # Make sure that we get a last message telling us we are done 1058 elapsed_time = time.time() - self._start_time

~\AppData\Roaming\Python\Python38\site-packages\joblib\parallel.py in retrieve(self) 933 try: 934 if getattr(self._backend, 'supports_timeout', False): --> 935 self._output.extend(job.get(timeout=self.timeout)) 936 else: 937 self._output.extend(job.get())

~.conda\envs\pt\lib\multiprocessing\pool.py in get(self, timeout) 769 return self._value 770 else: --> 771 raise self._value 772 773 def _set(self, i, obj):

TypeError: init() missing 1 required positional argument: 'N'

It seems the error lies in involking multiprocessing from joblib. I can't handle an error so deep down there, could someone look into this?

From searching on web I got a possible reason that somewhere in a package is missing a 'self' in the init() function. While I can't tell it's from the code of Qlib or some other place. My own code was too simple to directly call any initiation function except the dataloaderlp and it appears with enough parameters sent in 'config' argument.

Thanks.
question stale

opened by hensejiang 13
Workflow questions
First of all, thanks to you developers and Qlib contributors. This platform is contributing to my learning of the Quant world and especially Python programming, topics that I'm still a beginner.

I'm from Brazil and running Qlib strategies for brazilian stocks. Luckily I managed to convert my dataset originally from CSV to qlib format and it seems to be working.

That's why I have some doubts about the workflow, which if solved, will help me to flow in the work here.

In “market” do I define a benchmark for my country? In Brazil we use the IBOVESPA benchmark, but as I have not yet selected the stocks that make up the index, I left this option as “all”. Whereas the “market” option is where I define the benchmark. What is the “benchmark” option in the workflow? lol is confusing to me. Was it my portfolio?

I am using LGBModel for testing. I see that in the prediction results the “sr.generate()” generates a score of the actions. What would this result be? Are the stocks that improve performed in the backtest?

If I understand the “market” and “benchmark” functions in workflow I should understand these results below. But I would like to know if I can change the frequency to “week” instead of “day”? Would I have to change my dataset?

One suggestion: write some comments in the workflow_by_code.ipynb file I think it will help beginners like me.
question stale
opened by ander-son-almeida 13
TCN model code error
🐛 Bug Description

qlib/contrib/model/pytorch_tcn_ts.py line 171-line174 feature = data[:, :, 0:-1].to(self.device) label = data[:, -1, -1].to(self.device)

pred = self.TCN_model(feature.float())

To Reproduce

Steps to reproduce the behavior:

change seq len as 10, and run \examples\benchmarks\TCN\workflow_config_tcn_Alpha158.yaml

Expected Behavior

Screenshot

x as input with shape(batch,seq_len,feat), but right input should be as shape(batch,feat,seq_len,) so nn.Conv1d can be calculate

Environment

Note: User could run cd scripts && python collect_info.py all under project directory to get system information and paste them here directly.

Qlib version:

Python version:

OS (Windows, Linux, MacOS):

Commit number (optional, please provide it if you are using the dev version):

Additional Notes
bug
opened by yycyyyc321 0
There is a vulnerability in lxml:4.8.0,upgrade recommended

https://github.com/microsoft/qlib/blob/667fb0e4d99012dc24aa468230eabcd240fcde8a/scripts/data_collector/br_index/requirements.txt#L11

CVE-2022-2309

Recommended upgrade version：4.9.1

opened by QiAnXinCodeSafe 0
There is a vulnerability in certifi:2021.10.8,upgrade recommended

https://github.com/microsoft/qlib/blob/667fb0e4d99012dc24aa468230eabcd240fcde8a/scripts/data_collector/br_index/requirements.txt#L3

CVE-2022-23491

Recommended upgrade version：2022.12.7

opened by QiAnXinCodeSafe 0
how to save rolling trained models ?

❓ Questions and Help

hi, I know that we can save trained models with R.save_object(trained_model=model). but in the rolling example where the model is trained like : "trainer = TrainerR(); trainer(tasks)". I want to save all the trained models , but I am not sure how to add such a save model option? thanks
question

opened by specialuse 1
Bump certifi from 2021.10.8 to 2022.12.7 in /scripts/data_collector/br_index
Bumps certifi from 2021.10.8 to 2022.12.7.

Commits

9e9e840 2022.12.07

b81bdb2 2022.09.24

939a28f 2022.09.14

aca828a 2022.06.15.2

de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...

b8eb5e9 2022.06.15.1

47fb7ab Fix deprecation warning on Python 3.11 (#199)

b0b48e0 fixes #198 -- update link in license

9d514b4 2022.06.15

4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
滚动步长step和再平衡间隔。

在这个例子https://github.com/microsoft/qlib/blob/main/examples/benchmarks_dynamic/baseline/rolling_benchmark.py中，滚动步长step是20天，预测长度horizon是20天。每次滚动中，测试集长度也是20天（估计是由horizon决定的），也就这20天中每天都在再平衡，不知我的理解对否。

而我想要的是每次滚动中，滚动步长20天，预测长度是20天，但测试集长度为1天，也就是我想每20天再平衡一次。这相当于实盘中，每20天做一次训练和预测和调仓。也就是两次再平衡调仓间隔20天，间隔中间不调仓。不知怎样实现这个要求。
enhancement

opened by quant2008 0

Releases(v0.9.0)

v0.9.0(Dec 9, 2022)
Changes

Qlib data doc @chenditc (#1207)

🌟 Features

Fix logging_level: make logging level specified in qlib.init applies to all loggers @qianyun210603 (#1368)

Add early stopping to double ensemble model, add example @chenditc (#1375)

Set _artifact_uri when mlflow_run is not None. @ChiahungTai (#1367)

Optimize the implementation of uri & Fix async log bug @you-n-g (#1364)

update TSDataSampler refineing the memory layout of data array to speed up NN training @peteryang1 (#1342)

Refine RL todos @lihuoran (#1332)

Don't disable existing logger when initializing qlib. @jingedawang (#1339)

Use mock data for element operator tests. @ChiahungTai (#1330)

fixed_flake8_error_in_CI @SunsetWolf (#1325)

optimize_CI @SunsetWolf (#1314)

Migrate amc4th training @lihuoran (#1316)

Add REG_US and REG_TW into test case: test_utils.py. @ChiahungTai (#1310)

RL backtest with simulator @lihuoran (#1299)

General handler for open source data preprocessing @Arthur-Null (#1302)

Migrate backtest logic from NT @lihuoran (#1263)

Qlib simulator refinement (redo of PR 1244) @lihuoran (#1262)

Add simplified download command @HyeongminMoon (#1234)

Add csi500 benchmark for MLP model. @jingedawang (#1215)

Add Linear model results on dataset=csi500 @aprilpear (#1210)

Add result of DoubleEnsemble model on csi500 @huajunzhao (#1201)

Update LightGBM alpha158 csi500 result @tecton (#1199)

Add csi500 experiment result to CatBoost @lcrun (#1197)

More time for slow test @you-n-g (#1247)

Migrate NeuTrader to Qlib RL @lihuoran (#1169)

Update handler.py to fix CI @you-n-g (#1227)

Update test_qlib_from_source_slow.yml @you-n-g (#1222)

Add data handler for order book data @Arthur-Null (#1212)

Use average weights in DoubleEnsemble. @lwwang1995 (#1205)

Be compatible with Google Colab @you-n-g (#1188)

Add a make.bat file in docs folder for Windows @bingyao (#1131)

Adding ChangeInstrument op @wan9c9 (#1005)

Add retry for git actions & Fix MacOS Segment Error @you-n-g (#1173)

Backtest Mypy @lihuoran (#1130)

Auto log uncommmitted code @you-n-g (#1167)

Qlib RL framework (stage 2) - trainer @ultmaster (#1125)

change_pitdata_source @SunsetWolf (#1171)

split_CI @SunsetWolf (#1141)

Add time limit for CI @you-n-g (#1127)

🐛 Bug Fixes

Fix warning in processor.py. @ChiahungTai (#1386)

Fix RL example bug @lihuoran (#1384)

Resolve issues while running Automatic update of daily frequency data (from yahoo finance) for US region @HyeongminMoon (#1358)

Fix RL command Error @you-n-g (#1382)

Fix the Errors/Warnings when building Qlib's documentation @MaximSmolskiy (#1381)

fix csi500 end date issue @qianyun210603 (#1373)

Fixed log_param error @SunsetWolf (#1362)

Bug fix for Rank and WMA operators @qianyun210603 (#1228)

fix bug in clip_outlier in class RobustZScoreNorm(Processor) @NotF404 (#1294)

Fix Import Path Error @you-n-g (#1347)

fix position access error @lerit (#1267)

fix dump_bin @lerit (#1273)

Update cache.py @wony-zheng (#1329)

fixed_flake8_error_in_CI @SunsetWolf (#1325)

optimize_CI @SunsetWolf (#1314)

Fix _update_dealt_order_amount bug. @lihuoran (#1291)

Fix CI pylint bug @you-n-g (#1270)

fix_yahoo_collector_bug @SunsetWolf (#1257)

fixes #1187 error "Please install necessary libs for CatBoostModel." @OussCHE (#1246)

BUGFIX: remove_fields_space() function will drop Feature object field @wony-zheng (#1213)

chore: bugfix, judging system platform error on Mac @plpycoin (#1177)

fix bug on TRA dataset @HyeongminMoon (#1135)

Fix mount path bug @you-n-g (#1129)

📚 Documentation

Fix the Errors/Warnings when building Qlib's documentation @MaximSmolskiy (#1381)

Fix typos and grammar errors in docstrings and comments @qianyun210603 (#1366)

Fix typo. @ChiahungTai (#1365)

Fix the Errors with unexpected indentation when building Qlib's documentation @MaximSmolskiy (#1352)

Fix the Warnings with duplicate object description when building Qlib's documentation @MaximSmolskiy (#1353)

Fix the Warnings in rst files when building Qlib's documentation @MaximSmolskiy (#1349)

Add docs for qlib.rl @lwwang1995 (#1322)

Correct errors and typos in doc strings @qianyun210603 (#1338)

Update README.md @wan9c9 (#1279)

fix:doc for search_records @lerit (#1287)

Fix typo @ChiahungTai (#1308)

Add docs for the reampling example. @you-n-g (#1285)

Expm typo fix add log @wan9c9 (#1271)

update recorder.rst fixed typo @animic (#1264)

Update signal_strategy.py @ChengzhenDu (#1251)

Refine type hint and recorder @you-n-g (#1248)

add liability @wangershi (#1230)

Update README.md fixed typo @EricChangMSR (#1221)

Update the math of Metrics @you-n-g (#1211)

Update docs of strategy @you-n-g (#1209)

Update Data Updating Docs @you-n-g (#1203)

Add introduction for workflow_by_code.py @you-n-g (#1186)

Update FAQ for benchmarks @you-n-g (#1185)

Improve the style of documentation @bingyao (#1132)

Update README.md @lwwang1995 (#1179)

Update test_qlib_from_source_slow.yml's timeout setting. @you-n-g (#1178)

Qlib dev doc @you-n-g (#1142)

Source code(tar.gz)
Source code(zip)
v0.8.6(Jun 15, 2022)
Changes

🌟 Features

Csi500 example @you-n-g (#1126)

Add log info for ensemble @you-n-g (#1113)

opt local trainer (better mem releasing) @you-n-g (#1116)

Refine backtest codes @lihuoran (#1120)

fix_pylint_for_CI @SunsetWolf (#1119)

change_datasource @SunsetWolf (#1109)

Add .idea/ into gitignore @lihuoran (#1110)

Add .idea/ into gitignore @lihuoran (#1108)

init_instance_by_config enhancement @you-n-g (#1103)

Qlib RL framework (stage 1) - single-asset order execution @ultmaster (#1076)

fix_issue_1060 @SunsetWolf (#1092)

add_test_pit @SunsetWolf (#1089)

fix_macos_CI @SunsetWolf (#1081)

fix SepDataFrame when we del it to empty @you-n-g (#1082)

Make sepdf more like DataFrame @you-n-g (#1080)

🐛 Bug Fixes

Fix hist_ref in update.py @you-n-g (#1096)

add gym @you-n-g (#1104)

Update detailed_workflow.ipynb @ChiahungTai (#1084)

fix_issue_715 @SunsetWolf (#1070)

Fixed pandas FutureWarning @Hubedge (#1073)

Missing f prefix on f-strings fix @code-review-doctor (#1072)

📚 Documentation

Fixed a few mixed Chinese punctuation typos @bingyao (#1123)

Refine backtest codes @lihuoran (#1120)

Add explanation for the evalution metrics of Qlib @you-n-g (#1090)

Qlib RL framework (stage 1) - single-asset order execution @ultmaster (#1076)

Update README.md @you-n-g (#1091)

fix_issue_1060 @SunsetWolf (#1092)

Add instructions to add models @you-n-g (#1088)

Yahoo data Docs @you-n-g (#1077)

fix_macos_CI @SunsetWolf (#1081)

Fixed typos in workflow.rst @jingedawang (#1068)

Source code(tar.gz)
Source code(zip)
v0.8.5(Apr 24, 2022)
Changes

update doc for TopK-Drop @wan9c9 (#1015)

Known Limitations In Recorder @you-n-g (#999)

Fix pit docs format @Chaoyingz (#957)

Add REG_TW. @ChiahungTai (#955)

🌟 Features

fast fillna @you-n-g (#1074)

fix_issue_1019 @SunsetWolf (#1046)

Change Power to a NpPairOperator @wuzhe1234 (#1052)

Use the region in qlib.config for FileCalendarStorage. @ChiahungTai (#1049)

Update test_macos.yml with test doc @Wangwuyi123 (#1055)

update ci with test doc @Wangwuyi123 (#1054)

Add the HIST and IGMTF model on Alpha360 @Wentao-Xu (#1040)

Add high-frequency feature engineering code @Arthur-Null (#1022)

Add Qlib notebook tutorial @you-n-g (#1037)

Ibovespa index support @igor17400 (#990)

Fix Chinese punctuation regex comment @Chaoyingz (#1012)

Add pre-commit @you-n-g (#992)

Support feature names contain Chinese punctuation @Chaoyingz (#1003)

Add PRef operator (#988) @Chaoyingz (#1000)

Add lightgbm min version. @ChiahungTai (#995)

Add backtest example to online simulation @you-n-g (#984)

Optimize the pit collector script @Chaoyingz (#982)

Fix pit download_data script TypeError (#978) @Chaoyingz (#979)

Skip idx.is_lexsorted() when pandas version is larger than 1.3.0. @ChiahungTai (#973)

Use callback in LGBM.train. @ChiahungTai (#974)

Remove redundant import [fix pylint] @you-n-g (#962)

feat: add instrument context to inst_processor @PalanQu (#959)

Fix csi500 @SunsetWolf (#938)

Support Point-in-time Data Operation @bxdd (#343)

🐛 Bug Fixes

Fix shared DF Error @you-n-g (#1044)

fix tra dataset bug @you-n-g (#1050)

Fixed issue #943 about TCTS init_fore_model @TuozhenLiu (#1047)

chore: bug-fix for crypto data collector @plpycoin (#1038)

Update setup.py to fix doc @Wangwuyi123 (#1043)

update cli.py @wan9c9 (#1008)

Fix pit download_data script TypeError (#978) @Chaoyingz (#979)

Fix load_object bug @you-n-g (#977)

safe remove file and more friendly log @you-n-g (#967)

📚 Documentation

Update setup.py @you-n-g (#1048)

Update README.md @qinmoelei (#1039)

Add Qlib notebook tutorial @you-n-g (#1037)

Add docs for CSRankNorm @you-n-g (#1032)

fix ddgda run all bug & pylint @you-n-g (#1031)

Fix format for PULL_REQUEST_TEMPLATE.md @Chaoyingz (#1001)

Add lightgbm min version. @ChiahungTai (#995)

Add backtest example to online simulation @you-n-g (#984)

Fix comment typo @Chaoyingz (#987)

fix-issue948 @SunsetWolf (#986)

Update report.rst Metric @you-n-g (#980)

Update Tool Info @you-n-g (#981)

Update yahooquery marked words @Wangwuyi123 (#966)

fix annotation in PIT script @bxdd (#958)

Source code(tar.gz)
Source code(zip)
v0.8.4(Mar 8, 2022)
Changes

Update initialization.rst @SunsetWolf (#941)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/TabNet @dependabot (#898)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/Transformer @dependabot (#897)

Bump numpy from 1.17.4 to 1.21.0 in /examples/hyperparameter/LightGBM @dependabot (#896)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/XGBoost @dependabot (#895)

🌟 Features

[949] - Remove argument internal in BaseRun::download_data. @ChiahungTai (#953)

Fix error message in position.py @cuicorey (#922)

Fix wrong error messages. @ChiahungTai (#946)

[931] Remove mutable default argument. @ChiahungTai (#932)

performance mprovement @you-n-g (#921)

Add data analysis feature for report @you-n-g (#918)

Enhance pytorch nn @you-n-g (#917)

Fix pylint @SunsetWolf (#888)

Add more docs about initialization @you-n-g (#880)

Make the logic of handler Clear @you-n-g (#877)

🐛 Bug Fixes

fix bug @wan9c9 (#950)

[930] Fix typo HasingStockStorage to HashingStockStorage. @ChiahungTai (#947)

Support Reweighter for HighFreq Model @you-n-g (#908)

add weight param @aurora5161 (#907)

fix workflow bug @you-n-g (#882)

📚 Documentation

Update Metric Explanation @you-n-g (#920)

Add data analysis feature for report @you-n-g (#918)

Update README.md @wendili-cs (#915)

Some links about high-frequency trading @you-n-g (#884)

Update Portfolio README.md @you-n-g (#900)

Update Benchmark Docs @you-n-g (#899)

Docs improvement of backtest @you-n-g (#885)

Update handler processors docs @you-n-g (#879)

expression example @you-n-g (#887)

Docs to get original price @you-n-g (#878)

Source code(tar.gz)
Source code(zip)
v0.8.3(Jan 19, 2022)
Changes

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/ADARNN @dependabot (#870)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/ADD @dependabot (#869)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/ALSTM @dependabot (#868)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/CatBoost @dependabot (#867)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/DoubleEnsemble @dependabot (#866)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/GRU @dependabot (#833)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/LightGBM @dependabot (#830)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/SFM @dependabot (#829)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/TCTS @dependabot (#834)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/GATs @dependabot (#831)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/TRA @dependabot (#832)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/Localformer @dependabot (#835)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/MLP @dependabot (#836)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/TCN @dependabot (#837)

Bump numpy from 1.17.4 to 1.21.0 in /examples/benchmarks/LSTM @dependabot (#838)

🌟 Features

fall back error @you-n-g (#875)

Disable Cache By Default @you-n-g (#871)

Supporting Arctic Backend Provider & Orderbook, Tick Data Example @luocy16 (#744)

Add future calendar collector @zhupr (#795)

🐛 Bug Fixes

Remove arctic from Qlib core to Contrib @you-n-g (#865)

Fix pytorch_nn.py step bug @you-n-g (#864)

📚 Documentation

Add Desc For the limitation of 3.9 @Wangwuyi123 (#863)

Fix code and docs for issues @you-n-g (#853)

Source code(tar.gz)
Source code(zip)
v0.8.1(Jan 15, 2022)
Changes

🌟 Features

pylint code refine & Fix nested example @you-n-g (#848)

chore: remove hard code input dimension of model pytorch_tcts @PalanQu (#843)

[840] - Test case for operators. @ChiahungTai (#841)

DDG-DA paper code @you-n-g (#743)

[807] Move the REG_CONSTANT/EPS to constant.py. @ChiahungTai (#811)

Some Optimization of online code @you-n-g (#784)

Add Crypto dataset from coingecko @b4thesunrise (#733)

Reformat example data names: use {region}_data for 1-day data, and {region}_data_1min for 1-min data @cning112 (#781)

support optimization based strategy @evanzd (#754)

Add hook for supporting RL strategy @you-n-g (#768)

Enhance Task Dict Var @you-n-g (#778)

Add Cache to avoid frequently loading Calendar @you-n-g (#766)

Update BCELoss in MLP model @cuicorey (#756)

solve VERSION.txt bug @b4thesunrise (#732)

Hyperopt upgrade @upgradvisor-bot (#741)

Add method parameter for volume @you-n-g (#734)

🐛 Bug Fixes

Fix the read the docs error @you-n-g (#852)

pylint code refine & Fix nested example @you-n-g (#848)

fix: highfreq_gdbt_model of prepare data @PalanQu (#846)

Fix some warnings in log.py. @ChiahungTai (#805)

Fix $volume normalization issue @2young-2simple-sometimes-naive (#792)

Fix account shared bug @you-n-g (#791)

fix dump_bin:DumpDataUpdate @zhupr (#783)

Fix torch models on cpu @cning112 (#782)

fix cn_index collector @zhupr (#780)

Use encoding="utf-8" in open. @ChiahungTai (#773)

fix IndexError of the last trading day in backtest calendar @zhupr (#751)

remove unneeded code from workflow_by_code.ipynb && fix analysis_model_performance @zhupr (#740)

📚 Documentation

Add description of the pr template. @ChiahungTai (#812)

Fix typos and comments. @ChiahungTai (#815)

Update README.md @you-n-g (#802)

Fix BaseStrategy path. @ChiahungTai (#801)

Add a more understandable example of data workflow @you-n-g (#797)

fix_typo @SunsetWolf (#790)

Update Contributor list @you-n-g (#779)

Update Docs of Alpha360 @you-n-g (#777)

Replace scripts/get_data.py to get_data.py. @ChiahungTai (#775)

Fix typo leanable to learnable. @ChiahungTai (#774)

Add docs about the patameters @you-n-g (#771)

add description of dataset document @zhupr (#742)

Source code(tar.gz)
Source code(zip)
v0.8.0(Dec 7, 2021)
Changes

docs improvement @you-n-g (#730)

Fix backtest @zhupr (#719)

add logger @demon143 (#709)

Add SigAnaRecord in to workflow_by_code.py @you-n-g (#707)

support adding from date when updating pred @you-n-g (#703)

Add A New Baseline: ADD @f-cg (#704)

Fix high-freq data @zhupr (#702)

Update task_management.rst @demon143 (#700)

Update workflow_by_code.ipynb @wan9c9 (#697)

Update task_management.rst @demon143 (#654)

Merging Backtest improve @you-n-g (#694)

Add AdaRNN baseline. @lwwang1995 (#689)

add ops_warning_log to config @zhupr (#685)

Callable Exp @you-n-g (#683)

bugfix: Fix the problem that caused highfreq's yaml to be unusable @Ckend (#678)

add default protocol_version @zhupr (#677)

An example to get index from TSDataSampler @you-n-g (#679)

handler demo cache @you-n-g (#606)

Improve the backtest design and APIs @you-n-g (#650)

More citations @markzhao98 (#673)

Add A New Baseline: TCN @f-cg (#668)

GPU identification bug fix for GATs @markzhao98 (#669)

fix 'duplicate axis' error when updating cache @zhupr (#661)

add logs in case of ops.py errors @zhupr (#659)

Adjusting gbdt.py's parameter @you-n-g (#660)

🌟 Features

Auto injecting model and dataset for Recorder @you-n-g (#645)

Checking dataset empty @you-n-g (#647)

Update TCTS. @lwwang1995 (#643)

Add support for MacOS-11 @zhupr (#630)

🐛 Bug Fixes

Fix update record bug @you-n-g (#723)

improve the doc of auto init @you-n-g (#541)

fix workflow_config_lightgbm_multi_freq.yaml @zhupr (#635)

remove 3.6 @you-n-g (#629)

📚 Documentation

Update the docs of ops.py @you-n-g (#718)

improve the doc of auto init @you-n-g (#541)

more detailed docs for workflow @you-n-g (#639)

Update manage.py @demon143 (#628)

Update benchmark based on new backtest @you-n-g (#634)

Source code(tar.gz)
Source code(zip)
v0.8.0a1(Sep 30, 2021)
Changes

Merge nested main @wangwenxi-handsome (#597)

🌟 Features

Support Highfreq Backtest with the Model/Rule/RL Strategy @bxdd (#438)

Source code(tar.gz)
Source code(zip)
v0.7.2(Sep 30, 2021)
Changes

make the prediction update more friendly @you-n-g (#609)

Fix Models @you-n-g (#483)

update change doc @demon143 (#623)

Update initialization.rst @demon143 (#622)

Share version number @you-n-g (#620)

Add file lock for MLflowExpManager @you-n-g (#619)

Update code_standard.rst @demon143 (#587)

update cvxpy version @you-n-g (#616)

🐛 Bug Fixes

Fix the type of filter_pipe @zhupr (#611)

Source code(tar.gz)
Source code(zip)
v0.7.1(Sep 16, 2021)
Changes

Update setup.py @anukaal (#600)

Fix SimpleDatasetCache @zhupr (#604)

Fix undefined names in Python code @cclauss (#599)

Supporting shared processor @you-n-g (#596)

test.yml: Remove redundant code @cclauss (#595)

fix typos @saintmalik (#592)

Fix TRA @evanzd (#591)

Modify the Feature to be case sensitive @zhupr (#589)

Update manage.py @demon143 (#586)

Update FAQ.rst @zhupr (#588)

Multiple freq support @zhupr (#580)

without_conda @Wangwuyi123 (#575)

Update ensemble.py @demon143 (#560)

Update README.md @you-n-g (#546)

Update strategy.py @ZhangTP1996 (#545)

fix TRA when logdir is None @evanzd (#542)

refactor TRA @evanzd (#531)

refactor online serving rolling api @you-n-g (#539)

sort index after loader @you-n-g (#538)

Update data.rst @panshuaiyin (#532)

Add two transformer models via upload @yingtaoluo (#508)

Fix macos test ci @zhupr (#530)

Remove the date fetch from the expression and calculate it when used instead @zhupr (#527)

Update qrun to automaticly save the config to the artifacts uri @wuzhe1234 (#525)

Add a check if change is mutated to YahooNormalize1d @zhupr (#524)

Issue fix @slowy07 (#523)

Add filter string to mlflow experiment list_recorders function. @chaosddp (#512)

Add a check if change is mutated to YahooNormalize1d @zhupr (#514)

Bug fix: ClientProvider not set connection to calendar and instrument provider when initializing. @chaosddp (#500)

updated readme of yahoo collector where region parameter was incorrect @2796gaurav (#504)

🌟 Features

MVP for Indian Stocks in qlib using yahooquery @2796gaurav (#513)

🐛 Bug Fixes

check lexsort in the 'lazy_sort_index' function @markzhao98 (#566)

Fix multi-process loop calls @zhupr (#574)

Fix shared task dict BUG @you-n-g (#576)

Source code(tar.gz)
Source code(zip)
v0.7.0(Jul 11, 2021)
Changes

Update workflow.rst @xiaowuhu (#497)

Update workflow.rst @xiaowuhu (#496)

Update TCTS. @lwwang1995 (#495)

support check_transform_proc module_path @bxdd (#494)

Make OnlineToolR more user-friendly and fix some bugs @lzh222333 (#475)

examples/workflow_by_code.ipynd: fix an error in R.get_recorder() par… @docularxu (#481)

Add TRA Model @linhx25 (#479)

Support using config to register custom operators @bxdd (#476)

Set self.fitted = True instead of self._fitted. @topskychen (#478)

Support extend data @zhupr (#463)

simplify the code and prevent float when shifting @you-n-g (#473)

Update TCTS README. @lwwang1995 (#472)

Add TCTS baseline. @lwwang1995 (#464)

Remove non-existing parameter description @arisliang (#462)

fix DelayTrainerRM @lzh222333 (#459)

Multiprocessing support for Online Serving @lzh222333 (#435)

fix XGBoost predict error @zhupr (#457)

Add import stock pool in doc @arisliang (#452)

add get_feature_importance to model interpret @zhupr (#444)

Update 1min demo data in CSV format @arisliang (#442)

Remove repeated package from requirements @arisliang (#447)

Update integration.rst @arisliang (#448)

Update report.rst @arisliang (#450)

Fix YahooCollector can't download 1min data @zhupr (#441)

Update collector.py @arisliang (#440)

Update README.md @arisliang (#439)

add data storage @zhupr (#372)

Update README.md @arisliang (#433)

Fix get_module_by_module_path to support pickle module loaded from arbitrage source file @evanzd (#431)

LightGBM hyperparameter @ChengYen-Tang (#423)

Add configurable dataset to examples @zhupr (#416)

Add set_global_logger_level @zhupr (#412)

Add future trading date collector @zhupr (#405)

Fix online mode bugs @zhupr (#401)

update high freq demo @javaThonc (#358)

Fix print issue @D-X-Y (#381)

Modify get_exp & get_recorder api @Derek-Wds (#380)

Add DataLoader Based on DataHandler & Add Rolling Process Example & Restructure the Config & Setup_data @bxdd (#374)

Fix us_index collector @zhupr (#379)

Add MultiSegRecord and add segment kwargs in model.pred @D-X-Y (#378)

debug @lewwang1995 (#373)

Update notebook @Derek-Wds (#371)

fix docs @Flouse (#365)

Add load_object function for R @D-X-Y (#362)

Fix data doc @bxdd (#359)

Fix yahoo_collector @zhupr (#357)

Fix dump_bin.py && check_dump_bin.py @zhupr (#350)

Update init.py @wendili-cs (#351)

Update Contact Us Information @you-n-g (#348)

fix bug of consider TURE as boolean instead of stock code @2young-2simple-sometimes-naive (#347)

Fix errors when SignalRecord is not called before SigAna/PortAna @D-X-Y (#345)

Fix bugs in Ghost BN in TabNet and typos in README @D-X-Y (#337)

another typo of docs @Rekind1e (#333)

Fix typo of docs @Rekind1e (#332)

Fixed code quality issues @withshubh (#311)

Fix Various Bugs for contrib.pytorch_ models @D-X-Y (#329)

Add BaseCollector @zhupr (#324)

Move get_path to get_or_create_path, use the best model of SFM / TabNet @D-X-Y (#328)

Fix code in ops @bxdd (#318)

Fix collector doc @ChengYen-Tang (#272)

Update FAQ doc & Update ops docstring @bxdd (#300)

Update the Wrapper class to have an informative str representation @D-X-Y (#298)

Add a new method to benchmarks: DoubleEnsemble @meng-ustc (#286)

Update README.md @wendili-cs (#291)

Update filter.py @you-n-g (#277)

safe yaml loader @you-n-g (#273)

Update workflow.rst @lbaiao (#268)

Fix typo in Data Layer @miaekim (#265)

Add Qlib highfreq doc & Update DatatSet Init Method @bxdd (#257)

Fix typo in workflow_by_code.py @pingsutw (#259)

Fix Highfreq Freq @bxdd (#256)

🌟 Features

Online bug fix, enhancement & docs for dataset, workflow, trainer ... @you-n-g (#466)

init version of online serving and rolling @you-n-g (#290)

Modify set_global_logger_level use of contextmanager @zhupr (#418)

Stale bot update @Derek-Wds (#413)

Support start exp with given exp & recorder id @Derek-Wds (#404)

Add fund data as an example @wangershi (#292)

Support resuming recorder @Derek-Wds (#340)

Implement Enhanced Indexing as a Portfolio Optimizer @yongzhengqi (#280)

(1) Fix /0 bug in double_ensemble, (2) remove _default_uri for R/expm, (3) support model size in pytorch models @D-X-Y (#314)

Update repr for dataset/workflow classes and add uri kwarg for QlibRecorder @D-X-Y (#302)

Add model saving for qrun and workflow example @Derek-Wds (#264)

Update models to enable save/load @Derek-Wds (#261)

🐛 Bug Fixes

Online bug fix, enhancement & docs for dataset, workflow, trainer ... @you-n-g (#466)

Update Recorder Wrapper to prevent reinitialization @Derek-Wds (#471)

Fix exception hook bug @Derek-Wds (#461)

Fix bug and update doc @Derek-Wds (#419)

Fix logger pickling error @Derek-Wds (#407)

Update qlib logger @Derek-Wds (#393)

Support resuming recorder @Derek-Wds (#340)

(1) Fix /0 bug in double_ensemble, (2) remove _default_uri for R/expm, (3) support model size in pytorch models @D-X-Y (#314)

Fix pytorch ts model loader bug @Derek-Wds (#322)

Update repr for dataset/workflow classes and add uri kwarg for QlibRecorder @D-X-Y (#302)

Update Highfreq Handler & Ops / Fix a Highfreq Bug @bxdd (#254)

Update models to enable save/load @Derek-Wds (#261)

Update readme and setup.py @Derek-Wds (#258)

CI: the MacOS numpy bug. @Derek-Wds (#253)

Fix CI @Derek-Wds (#250)

📚 Documentation

Online bug fix, enhancement & docs for dataset, workflow, trainer ... @you-n-g (#466)

Fix bug and update doc @Derek-Wds (#419)

Update doc @Derek-Wds (#399)

Update Filter doc @Derek-Wds (#319)

Update docs about record-temp @Derek-Wds (#260)

update 1min docs @zhupr (#252)

Source code(tar.gz)
Source code(zip)
v0.6.2(Feb 2, 2021)
Changes

move freq params to dataloader @you-n-g (#234)

Update pytorch_nn.py @wendili-cs (#241)

Update pytorch_alstm_ts.py @wendili-cs (#243)

Update pytorch_gats.py @wendili-cs (#244)

Update pytorch_gru.py @wendili-cs (#245)

Update pytorch_gru_ts.py @wendili-cs (#246)

Update pytorch_lstm.py @wendili-cs (#247)

Update pytorch_lstm_ts.py @wendili-cs (#248)

Update pytorch_alstm.py @wendili-cs (#242)

Fix bug in alpha360 @bxdd (#236)

Add filter_pipe API @Derek-Wds (#220)

Update data handler @you-n-g (#228)

Qlib Highfreq Support & Highfreq DataHanlder/Operator/Processor Examples @bxdd (#222)

Add paper year @Derek-Wds (#224)

Support yahoo 1min data @zhupr (#221)

Fix the error when the stock code is a number @zhupr (#78)

Support Register of Custom Feature Operators Easily @bxdd (#92)

Update pytorch_nn.py @wendili-cs (#213)

tabnet @javaThonc (#205)

for IDE auto-complete with global Wrapper @GoooIce (#201)

Update .gitignore @GoooIce (#203)

Add dataset standalone usage example @you-n-g (#200)

Support Return Order List in Backtest & Assign Executor in Config @bxdd (#198)

fix setup error @GoooIce (#197)

Update pytorch_lstm_ts.py @wendili-cs (#194)

Update pytorch_gru_ts.py @wendili-cs (#193)

Update pytorch_gats_ts.py @wendili-cs (#192)

Update pytorch_alstm_ts.py @wendili-cs (#191)

Security Fix for Arbitrary Code Execution - huntr.dev @huntr-helper (#189)

Update docs and add Python 3.9 CI @Derek-Wds (#178)

Update initialization.rst @GoooIce (#188)

Fix create_recorder bug @Derek-Wds (#182)

Fix unrecognized config bug @fzc621 (#183)

Fix some typo in qlib/tests/data.py @fzc621 (#179)

Supporting dict to define strategy in evaluate.py @you-n-g (#177)

Update workflow_config_gru_Alpha158.yaml @wendili-cs (#175)

Update workflow_config_lstm_Alpha158.yaml @wendili-cs (#174)

Update docs @Derek-Wds (#164)

Fix the first trading day of the calendar extra in report_df @zhupr (#125)

Fix recorder temp dir bug @Derek-Wds (#148)

better MemCacheUnit implement @hadrianl (#145)

Update alpha.rst @bxdd (#143)

add more doc to PortAnaRecord @you-n-g (#138)

fixing typos #4 @maciejdomagala (#135)

fixing typos #3 @maciejdomagala (#134)

Update benchmark performance @Derek-Wds (#97)

fixing typos #2 @maciejdomagala (#130)

Config.getattr should raise AttributeError @hadrianl (#129)

Refactor to Python3 style @Gwill (#119)

Demo @Derek-Wds (#122)

Add stale bot @Derek-Wds (#124)

Typo fix @maciejdomagala (#123)

Update CI and script @Derek-Wds (#118)

Update the Sign operation in ops.py @YifanDengWHU (#113)

Source code(tar.gz)
Source code(zip)
v0.6.1(Dec 12, 2020)
Changes

Update the Install Method in README @bxdd (#106)

Add author names @Derek-Wds (#99)

fix typo @Gwill (#95)

Update CI @Derek-Wds (#94)

Fix readthedocs @bxdd (#89)

Fix readthedocs @bxdd (#84)

Add the Config of readthedocs @bxdd (#83)

FIX CI @you-n-g (#81)

Supporting TSDatasetH @you-n-g (#75)

Update scripts @Derek-Wds (#74)

Source code(tar.gz)
Source code(zip)
pyqlib-0.6.1-cp36-cp36m-macosx_10_14_x86_64.whl(271.83 KB)
pyqlib-0.6.1-cp36-cp36m-manylinux1_x86_64.whl(626.00 KB)
pyqlib-0.6.1-cp36-cp36m-manylinux2010_x86_64.whl(626.00 KB)
pyqlib-0.6.1-cp36-cp36m-win_amd64.whl(277.58 KB)
pyqlib-0.6.1-cp37-cp37m-macosx_10_14_x86_64.whl(270.59 KB)
pyqlib-0.6.1-cp37-cp37m-manylinux1_x86_64.whl(628.88 KB)
pyqlib-0.6.1-cp37-cp37m-manylinux2010_x86_64.whl(628.88 KB)
pyqlib-0.6.1-cp37-cp37m-win_amd64.whl(277.35 KB)
pyqlib-0.6.1-cp38-cp38-macosx_10_14_x86_64.whl(271.16 KB)
pyqlib-0.6.1-cp38-cp38-manylinux1_x86_64.whl(664.77 KB)
pyqlib-0.6.1-cp38-cp38-manylinux2010_x86_64.whl(664.77 KB)
pyqlib-0.6.1-cp38-cp38-win_amd64.whl(279.11 KB)
v0.6.0(Dec 1, 2020)
Changes

Add AI baselines

CatBoost @javaThonc

TFT @wendili-cs

Linear @evanzd

ALSTM @Pikapikachuu @Don-ustc

SFM @javaThonc

LSTM & GRU @evanzd @lwwang1995

Gats @meng-ustc @Turtlesyu-27

XGBoost @javaThonc @lwwang1995

Fix and move to version 0.6.0 @Derek-Wds (#73)

Help to PR @you-n-g (#72)

refine docs @you-n-g (#71)

Fix ops bug & add raise comments of Skew Kurt @bxdd (#70)

Add download qlib_data docs @zhupr (#69)

Refactor collector @zhupr (#68)

Fix re about provider_uri & Fix 'exit not defined' error in ipython @bxdd (#67)

Add RiskModel and Portfolio Module @evanzd (#60)

Support csi100 data collection && Fix data collector @zhupr (#57)

🌟 Features

Refactor Qlib interface @you-n-g @Derek-Wds @lwwang1995 @evanzd @zhupr @wendili-cs @javaThonc @meng-ustc @Don-ustc @Pikapikachuu @bxdd (#56)

support non-full-fill rate of executor @you-n-g (#66)

Make the DNN model compatible with CPU & Update the Estimator doc @bxdd (#49)

🐛 Bug Fixes

fix the float conversion bug @you-n-g (#62)

fix requirements.txt @morristai (#45)

📚 Documentation

Fix typo @ogabrielluiz (#55)

Make the DNN model compatible with CPU & Update the Estimator doc @bxdd (#49)

Update CI & add black formatter @Derek-Wds (#51)

Update badges in REAME.md @bxdd (#44)

Source code(tar.gz)
Source code(zip)
pyqlib-0.6.0-cp36-cp36m-macosx_10_14_x86_64.whl(254.74 KB)
pyqlib-0.6.0-cp36-cp36m-manylinux1_x86_64.whl(608.91 KB)
pyqlib-0.6.0-cp36-cp36m-manylinux2010_x86_64.whl(608.91 KB)
pyqlib-0.6.0-cp36-cp36m-win_amd64.whl(260.37 KB)
pyqlib-0.6.0-cp37-cp37m-macosx_10_14_x86_64.whl(253.50 KB)
pyqlib-0.6.0-cp37-cp37m-manylinux1_x86_64.whl(611.79 KB)
pyqlib-0.6.0-cp37-cp37m-manylinux2010_x86_64.whl(611.79 KB)
pyqlib-0.6.0-cp37-cp37m-win_amd64.whl(260.14 KB)
pyqlib-0.6.0-cp38-cp38-macosx_10_14_x86_64.whl(254.07 KB)
pyqlib-0.6.0-cp38-cp38-manylinux1_x86_64.whl(647.67 KB)
pyqlib-0.6.0-cp38-cp38-manylinux2010_x86_64.whl(647.67 KB)
pyqlib-0.6.0-cp38-cp38-win_amd64.whl(261.91 KB)
v0.5.1(Oct 1, 2020)
Changes

Add the CI/CD

add the test CI @Derek-Wds

add the publishing action to support pip installation @bxdd

Fix some bugs

Fix bugs in test scripts @you-n-g @zhupr

Add the error message when using Qlib in the repository directory @bxdd

Add a simpler dataset @zhupr

Update the documentation

Add the badges @bxdd

Add the FAQ document @zhupr

Add the description for the new installation method (pip) @bxdd

Fix some other errors @wendili-cs @vanshg

🌟 Features

Release v0.5.1 @you-n-g (#31)

🐛 Bug Fixes

Release v0.5.1 @you-n-g (#31)

📚 Documentation

Fix typo in examples README @vanshg (#32)

add test and mod doc @you-n-g (#17)

Update data.rst @wendili-cs (#24)

Update README.md @wendili-cs (#22)

Update estimator.rst @wendili-cs (#23)

Add a Gitter chat badge to README.md @gitter-badger (#21)

Update data related information @Derek-Wds (#16)

Source code(tar.gz)
Source code(zip)
pyqlib-0.5.1-cp36-cp36m-macosx_10_14_x86_64.whl(206.06 KB)
pyqlib-0.5.1-cp36-cp36m-manylinux1_x86_64.whl(559.86 KB)
pyqlib-0.5.1-cp36-cp36m-manylinux2010_x86_64.whl(559.86 KB)
pyqlib-0.5.1-cp36-cp36m-win_amd64.whl(210.99 KB)
pyqlib-0.5.1-cp37-cp37m-macosx_10_14_x86_64.whl(204.69 KB)
pyqlib-0.5.1-cp37-cp37m-manylinux1_x86_64.whl(562.31 KB)
pyqlib-0.5.1-cp37-cp37m-manylinux2010_x86_64.whl(562.32 KB)
pyqlib-0.5.1-cp37-cp37m-win_amd64.whl(210.77 KB)
pyqlib-0.5.1-cp38-cp38-macosx_10_14_x86_64.whl(205.47 KB)
pyqlib-0.5.1-cp38-cp38-manylinux1_x86_64.whl(598.61 KB)
pyqlib-0.5.1-cp38-cp38-manylinux2010_x86_64.whl(598.62 KB)
pyqlib-0.5.1-cp38-cp38-win_amd64.whl(212.47 KB)
v0.5.0(Sep 24, 2020)

Qlib is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment. With Qlib, you can easily try your ideas to create better Quant investment strategies.
Source code(tar.gz)
Source code(zip)