530w数据，从头训stage1, stage2.1 。仍然明显有safe response&重复的现象，请问是我训练的不够充分吗？ stage1 batch_size=16 训练了320000 step，stage2.1训练了batch_size=1024 18000step

改成选随机的候选感觉好一些。

配置里按照paddle分布式教程设置为：distributed_args="--ips 10.130.19.203,10.130.17.157 --selected_gpus 0,1"，两台机器可以建立通信但是不开始训练，GPU每张卡有2g内存占用， 下面这种配置可以正常训练：distributed_args="--ips 10.130.19.203 --selected_gpus 0,1"，

Hi thanks for your great work! I explore the plato-2 directory and just found there are .sh files, may I ask where is the .py files? so I could try the chatbot interaction, thanks for your help!

非常感谢您开源的XL模型。我尝试用8个A100（每块40G显存）训练自己的XL模型，但因参数过大，显存还是不够。看plato-XL论文里面提到：Given the limited memory of each device, vanilla data parallelism cannot support the training of such a model with up to 11 billion parameters.As such, we adopt the sharded data parallelism (Rajbhandari et al., 2020) to eliminate memory redundancies, by partitioning the optimizer states, gradients and parameters across multiple devices. 请问论文里提到的这种模型参数跨多个显卡的训练方法要如何实现？

中文对话数据，数据量400w，单卡可以跑完整个epoch，单机4卡运行到一定步数就退出

环境： paddlepaddle-gpu==2.0.1 cuda==11.0 cudnn==8.0

终端报错是：

INFO 2021-09-05 21:51:40,245 launch_utils.py:327] terminate all the procs ERROR 2021-09-05 21:51:40,245 launch_utils.py:584] ABORT!!! Out of all 4 trainers, the trainer process with rank=[3] was aborted. Please check its log. INFO 2021-09-05 21:51:43,248 launch_utils.py:327] terminate all the procs`

work_log.3里面报错如下：

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0   paddle::framework::ParallelExecutor::Run(std::vector<std::string, std::allocator<std::string > > const&, bool)
1   paddle::framework::details::ScopeBufferedSSAGraphExecutor::Run(std::vector<std::string, std::allocator<std::string > > const&, bool)
2   paddle::framework::details::FastThreadedSSAGraphExecutor::Run(std::vector<std::string, std::allocator<std::string > > const&, bool)
3   paddle::framework::BlockingQueue<unsigned long>::Pop()
4   paddle::framework::SignalHandle(char const*, int)
5   paddle::platform::GetCurrentTraceBackString[abi:cxx11]()

----------------------
Error Message Summary:
----------------------
FatalError: `Termination signal` is detected by the operating system.
  [TimeInfo: *** Aborted at 1630683022 (unix time) try "date -d @1630683022" if you are using GNU date ***]
  [SignalInfo: *** SIGTERM (@0x3e800000a5e) received by PID 2812 (TID 0x7f718f576b80) from PID 2654 ***]

这个训练一般会持续很久，很可能会断了之后继续训练，所以继续训练也是个刚需。建议把如何继续训练写到文档里面。

还有就是现在要继续训练要自己在参数里填check_point路径和当前的start_step，这样还是太麻烦了，建议在保存check_point的时候把这个信息保存一下，这样继续训练的时候先检测这个信息，然后自动从上次最后的step开始训练

Lic2022的baseline源码，在AIStudio可以正常跑，本地跑时train_query，infer_dial，infer_dial均无错误，只在infer_query时出现以下错误

paddlepaddle：2.2.2 cuda：11.2 cudnn：8.2

$ sh ./scripts/local/job.sh ./projects/lic2022/conf/query_infer.conf

2022-04-18 15:40:25,456-INFO: [topology.py:169:init] HybridParallelInfo: rank_id: 0, mp_degree: 1, sharding_degree: 1, pp_degree: 1, dp_degree: 1, mp_group: [0], sharding_group: [0], pp_group: [0], dp_gr oup: [0], check/clip group: [0] W0418 15:40:25.456908 14688 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.6, Runtime API Version: 11.2 W0418 15:40:25.472528 14688 device_context.cc:465] device: 0, cuDNN Version: 8.2. [WARN] Using constant learning rate because of warmup_steps is not positive while using NoamScheduler. Loading parameters from ./projects/lic2022/model_zoo/query_finetune.pdparams. Loading has done! Traceback (most recent call last): File "./knover/scripts/infer.py", line 140, in infer(args) File "./knover/scripts/infer.py", line 83, in infer predictions = task.infer_step(model, data) File "e:\jupyternotebookproject\lic2022\knover\knover\core\task.py", line 46, in infer_step predictions = model.infer_step(inputs) File "e:\jupyternotebookproject\lic2022\knover\knover\core\model.py", line 508, in infer_step predictions = self._model(*inputs, mode="infer") File "e:\jupyternotebookproject\lic2022\knover\knover\core\model.py", line 180, in call outputs = self.infer_step(inputs) File "e:\jupyternotebookproject\lic2022\knover\knover\core\model.py", line 170, in infer_step predictions = self.infer(inputs, outputs) File "e:\jupyternotebookproject\lic2022\knover\knover\models\unified_transformer.py", line 297, in infer outputs = self.generator(self, inputs, outputs) File "e:\jupyternotebookproject\lic2022\knover\knover\modules\generator.py", line 163, in call state = self._update_state(state, probs) File "e:\jupyternotebookproject\lic2022\knover\knover\modules\generator.py", line 390, in _update_state state["predictions"] = paddle.concat([state["predictions"], pred], axis=1) File "E:\software\Anaconda3\envs\Knover\lib\site-packages\paddle\tensor\manipulation.py", line 345, in concat return paddle.fluid.layers.concat(input=x, axis=axis, name=name) File "E:\software\Anaconda3\envs\Knover\lib\site-packages\paddle\fluid\layers\tensor.py", line 327, in concat return _C_ops.concat(input, 'axis', axis) ValueError: (InvalidArgument) Tensor holds the wrong type, it holds int, but desires to be int64_t. [Hint: Expected valid == true, but received valid:0 != true:1.] (at ../paddle/fluid/framework/tensor_impl.h:33) [operator < concat > error] INFO 2022-04-18 15:40:36,201 launch_utils.py:341] terminate all the procs ERROR 2022-04-18 15:40:36,201 launch_utils.py:604] ABORT!!! Out of all 1 trainers, the trainer process with rank=[0] was aborted. Please check its log. INFO 2022-04-18 15:40:39,210 launch_utils.py:341] terminate all the procs INFO 2022-04-18 15:40:39,210 launch.py:311] Local processes completed.

exit_code=0 [[ 0 != 0 ]] exit 0

First of all thank you very much for your work. When I run the code of PLATO-KAG as instructed, there is a problem. Would you mind answering this question？thank you very much.

看代码的规则，vocab里既要有[UNK]又要有<unk>，否则会报错，这两个token都代表未知词吧，有什么区别吗？ 另外我看例子中英语的vocab有些token的ids重复了，如下，不明白为什么，重复的id不会被覆盖吗？自己做vocab的时候也要改成重复的吗？ <unk> 0 <s> 1 </s> 2 [UNK] 0 [PAD] 0 [CLS] 1 [SEP] 2

When running DSTC10-Track2/task2 (Knowledge-grounded Dialogue Modeling), I got error message like this. Please help me to fix the following error.

the error message:

Load pretraining parameters from /home/Knover/projects/DSTC10-Track2/task2/models/SOP-32L-Detection
Traceback (most recent call last):
  File "/home/Knover/knover/data/dialog_reader.py", line 578, in __wrapper__
    for batch in batch_reader():
  File "/home/Knover/knover/data/dialog_reader.py", line 517, in __wrapper__
    for batch in batch_reader():
  File "/home/Knover/knover/data/dialog_reader.py", line 432, in __wrapper__
    for record in reader():
  File "/home/Knover/knover/data/dialog_reader.py", line 369, in __wrapper__
    yield from self._read_numerical_file(fp, phase, is_infer)
TypeError: _read_numerical_file() takes from 2 to 3 positional arguments but 4 were given
WARNING:root:Your reader has raised an exception!
Traceback (most recent call last):
Exception in thread   File "./knover/scripts/infer.py", line 145, in <module>
Thread-1:
Traceback (most recent call last):
      File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
infer(args)
    for step, data in enumerate(infer_generator(), 1):
  File "/home/Knover/myenv/lib/python3.8/site-packages/paddle/fluid/reader.py", line 1392, in __next__
        return self._reader.read_next()self.run()

SystemError:   File "/usr/lib/python3.8/threading.py", line 870, in run
(Fatal) Blocking queue is killed because the data reader raises an exception.
  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:166)

Thank you for taking the time to review this.

你好，我使用自己的数据对Knover Classfier进行finetune后保存checkpoint （我们发现保存的checkpoint有2612个参数文件比提供的SOP-32L-Context模型的522个多了近4倍），然后基于这个checkpoint使用infer.sh进行预测，但是同一个数据集的预测结果每次都不一致，请问这种情况正常么？该如何解决？

您好，我使用Knover训练了一个Plato2模型，但在使用hub serving start部署到我的后台后，使用jmeter测试，jmeter客户端报错。 报错内容如下： InvalidArgumentError: Broadcast dimension mismatch. Operands could not be broadcast together with the shape of X = [20, 16, 20, 27] and the shape of Y = [20, 16, 1, 8]. Received [27] in X is not equal to [8] in Y at i:3. [Hint: Expected x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1 == true, but received x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1:0 != true:1.] (at /paddle/paddle/phi/kernels/funcs/common_shape.h:84) [operator < elementwise_add > error]","results":"","status":"101"} 我的环境如下： 服务部署平台：paddlepaddle-gpu容器。容器版本：paddlepaddle/paddle 2.3.2-gpu-cuda11.2-cudnn8 Knover版本：0.0.6 GPU数量：1 paddlehub版本：2.3.0 我完成了以下方案的测试： 1、export CUDA_VISIBLE_DEVICES=0 2、因为本地运行interact.py对应脚本成功，因此我将AIstudio上一位开发者的开源项目中的module.py中关于数据加载的部分按照Knover/knover/core/model.py中对应的部分重写了一次，但仍然报错。对比两步骤发现：本地调用时，将数据转换为tensor的部分shape恒定为20，但hub部署的服务过程中的tensor的shape会随着文本分词后的长度而变化。我不太清楚应该修改哪个部分，请问是否有这方面的方案，或者在该版本下的plato2_en_base的部署教程啊？ 2中的开源作者的AIStudio的链接为：https://aistudio.baidu.com/aistudio/projectdetail/1197592 谢谢

感谢百度一直以来在中文对话上的工作~

我想咨询一下论文《Link theWorld: Improving Open-domain Conversation with Dynamic Spatiotemporal-aware Knowledge》中2.1节Service Information第一段中的service API具体是如何构建的。 我看了论文，觉得全文最重要的就是这个service API的构建，假若构造的足够好的话，确实可以大大提高人机交互体验。但是论文似乎并没有细说这部分的工作以及相关开源的代码/数据。确实非常好奇~

非常期待能得到您的回复！ 谢谢~

各位大佬，

请问如何基于现有的开源英文plato-2模型，搭建一个中文多轮对话机器人？本人看了下面的链接，但还是对如何使用英文的plato-2搭建适用于中文多轮对话任务的plato-2模型表示不太了解。能否请各位大佬提供一些更详细的细节？还能否请各位已经实现的大佬共享一些代码供小弟参考，谢谢。

链接： https://github.com/PaddlePaddle/Knover/issues/25