PaddleBoBo是基于PaddlePaddle和PaddleSpeech、PaddleGAN等开发套件的虚拟主播快速生成项目

Overview

PaddleBoBo - 元宇宙时代,你也可以动手做一个虚拟主播。

python version GitHub Repo stars 支持系统

PaddleBoBo是基于飞桨PaddlePaddle深度学习框架和PaddleSpeech、PaddleGAN等开发套件的虚拟主播快速生成项目。PaddleBoBo致力于简单高效、可复用性强,只需要一张带人像的图片和一段文字,就能快速生成一个虚拟主播的视频;并能通过简单的二次开发更改文字输入,实现视频实时生成和实时直播功能。

应用案例

运行环境

  • 飞桨AIStudio在线运行 (强烈推荐,Tesla V100冲!!!)
  • 自建本地环境
    • Windows 10
    • Python 3.7+
    • PaddlePaddle >= 2.2.1
    • Nvidia显卡 显存16G+(没测试过,希望有显卡的土豪大佬们反馈下)

快速开始

1.安装依赖包

pip install ppgan paddlespeech

2.配置文件(default.yaml)

GANDRIVING:
  FOM_INPUT_IMAGE: './file/input/test.png' #带人脸的静态图
  FOM_DRIVING_VIDEO: './file/input/zimeng.mp4' #用作表情迁移的参考视频
  FOM_OUTPUT_VIDEO: './file/input/test.mp4' #表情迁移后的视频输出路径

SAVEPATH:
  VIDEO_SAVE_PATH: './file/output/video/' #保存音频的路径
  AUDIO_SAVE_PATH: './file/output/audio/' #保存生成虚拟主播视频的路径

3.让静态人脸动起来

python create_virtual_human.py --config default.yaml

4.通用版本生成

python general_demo.py \
    --human ./file/input/test.mp4 \
    --output output.mp4 \
    --text 各位开发者大家好,欢迎使用飞桨。
参数 参数说明
human 第3步生成的人脸视频路径
output 生成虚拟主播视频的输出路径
text 虚拟主播语音文本

案例库

AI财经新闻主播

* 运行news_app.py 持续采集同花顺新闻数据并生成视频
* 运行play.py 实时和循环播放生成的视频

更多应用案例正在开发中,欢迎开发者投稿

TODO LIST

最近有点累,如果大佬们有什么想法的话可以提Issue,同时也欢迎PR。

参考资料

Comments
  •  Ubuntu 环境,运行快速开始时候,出现了一个错误

    Ubuntu 环境,运行快速开始时候,出现了一个错误

    ` 环境:Ubuntu 20.04 Python 3.8.10,前面安装等执行都正常 python3 general_demo.py --human ./file/input/test.mp4 --output output.mp4 --text 各位开发者大家好,欢迎使用飞桨。

    Building prefix dict from the default dictionary ... [2022-01-09 21:51:25] [DEBUG] [init.py:113] Building prefix dict from the default dictionary ... Loading model from cache /tmp/jieba.cache [2022-01-09 21:51:25] [DEBUG] [init.py:132] Loading model from cache /tmp/jieba.cache Loading model cost 0.539 seconds. [2022-01-09 21:51:25] [DEBUG] [init.py:164] Loading model cost 0.539 seconds. Prefix dict has been built successfully. [2022-01-09 21:51:25] [DEBUG] [init.py:166] Prefix dict has been built successfully. /usr/local/lib/python3.8/dist-packages/paddle/fluid/dygraph/math_op_patch.py:251: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.int64, but right dtype is paddle.int32, the right dtype will convert to paddle.int64 warnings.warn( Traceback (most recent call last): File "general_demo.py", line 18, in video = wav2lip(args.human,wavfile,args.output) #将音频合成到唇形视频 File "/root/PaddleBoBo/PaddleTools/GAN.py", line 11, in wav2lip wav2lip_predictor.run(input_video, input_audio, output) File "/usr/local/lib/python3.8/dist-packages/ppgan/apps/wav2lip_predictor.py", line 176, in run raise ValueError( ValueError: --face argument must be a valid path to video/image file `

    opened by laoxiong 9
  • 博主,报错了~请看下

    博主,报错了~请看下

    Traceback (most recent call last): File "news_app.py", line 35, in TTS = TTSExecutor() TypeError: init() missing 1 required positional argument: 'config'

    opened by zlszhonglongshen 7
  • 自定义头像生成demo video的时候失败了,报CUDNN_STATUS_NOT_SUPPORTED

    自定义头像生成demo video的时候失败了,报CUDNN_STATUS_NOT_SUPPORTED

    我租了一台8个GPU、单GPU 84G显存的tesla 100,使用缺省头像生成是没问题的。如果使用自动定义头像(我自己的照片,7MB),创建虚拟角色的步骤没有问题(就是生成这个me.mp4),但在生成demo video的时候失败了,报CUDNN_STATUS_NOT_SUPPORTED

    
    # python general_demo.py     --human ./file/input/me.mp4     --output me.mp4     --text 各位开发者大家好,我是您的专属虚拟主播,很高兴能为您服务。
    
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/librosa/core/constantq.py:1059: DeprecationWarning: `np.complex` is a deprecated alias for the builtin `complex`. To silence this warning, use `complex` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.complex128` here.
    Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
      dtype=np.complex,
    [01/05 02:11:00] ppgan INFO: Found /root/.cache/ppgan/GPEN-512.pdparams
    W0105 02:11:01.451380 57662 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 8.0, Driver API Version: 11.5, Runtime API Version: 11.2
    W0105 02:11:01.453052 57662 device_context.cc:465] device: 0, cuDNN Version: 8.1.
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
    Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
      if data.dtype == np.object:
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_0. generator.noises.noise_0 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_1. generator.noises.noise_1 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_2. generator.noises.noise_2 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_3. generator.noises.noise_3 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_4. generator.noises.noise_4 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_5. generator.noises.noise_5 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_6. generator.noises.noise_6 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_7. generator.noises.noise_7 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_8. generator.noises.noise_8 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_9. generator.noises.noise_9 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_10. generator.noises.noise_10 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_11. generator.noises.noise_11 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_12. generator.noises.noise_12 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_13. generator.noises.noise_13 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py:1436: UserWarning: Skip loading for generator.noises.noise_14. generator.noises.noise_14 is not found in the provided dict.
      warnings.warn(("Skip loading for {}. ".format(key) + str(err)))
    [2022-01-05 02:11:02,854] [    INFO] [log.py] [L57] - File /root/.paddlespeech/models/fastspeech2_csmsc-zh/fastspeech2_nosil_baker_ckpt_0.4.zip md5 checking...
    [2022-01-05 02:11:03,674] [    INFO] [log.py] [L57] - File /root/.paddlespeech/models/pwgan_csmsc-zh/pwg_baker_ckpt_0.4.zip md5 checking...
    encoder_type is transformer
    decoder_type is transformer
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/framework/io.py:415: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
      if isinstance(obj, collections.Iterable) and not isinstance(obj, (
    Building prefix dict from the default dictionary ...
    [2022-01-05 02:11:06] [DEBUG] [__init__.py:113] Building prefix dict from the default dictionary ...
    Loading model from cache /tmp/jieba.cache
    [2022-01-05 02:11:06] [DEBUG] [__init__.py:132] Loading model from cache /tmp/jieba.cache
    Loading model cost 0.470 seconds.
    [2022-01-05 02:11:07] [DEBUG] [__init__.py:164] Loading model cost 0.470 seconds.
    Prefix dict has been built successfully.
    [2022-01-05 02:11:07] [DEBUG] [__init__.py:166] Prefix dict has been built successfully.
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/math_op_patch.py:251: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.int64, but right dtype is paddle.int32, the right dtype will convert to paddle.int64
      warnings.warn(
    Reading video frames...
    Number of frames available for inference: 300
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/librosa/util/utils.py:2099: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
    Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
      np.dtype(np.float): np.complex,
    /root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/librosa/util/utils.py:2099: DeprecationWarning: `np.complex` is a deprecated alias for the builtin `complex`. To silence this warning, use `complex` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.complex128` here.
    Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
      np.dtype(np.float): np.complex,
    Length of mel chunks: 157
    [2022-01-05 02:11:13,369] [    INFO] - unique_endpoints {''}
    [2022-01-05 02:11:13,370] [    INFO] - Found /root/.cache/paddle/hapi/weights/wav2lip_hq.pdparams
    Model loaded
      0%|                                                                                                                                                                                 | 0/2 [00:00
        video = wav2lip(args.human,wavfile,args.output) #将音频合成到唇形视频
      File "/root/PaddleBoBo/PaddleTools/GAN.py", line 11, in wav2lip
        wav2lip_predictor.run(input_video, input_audio, output)
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/apps/wav2lip_predictor.py", line 257, in run
        for i, (img_batch, mel_batch, frames, coords) in enumerate(
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/tqdm/std.py", line 1180, in __iter__
        for obj in iterable:
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/apps/wav2lip_predictor.py", line 118, in datagen
        face_det_results = self.face_detect(
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/apps/wav2lip_predictor.py", line 74, in face_detect
        detector.get_detections_for_batch(
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/faceutils/face_detection/api.py", line 73, in get_detections_for_batch
        detected_faces = self.face_detector.detect_from_batch(images.copy())
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/faceutils/face_detection/detection/sfd/sfd_detector.py", line 56, in detect_from_batch
        bboxlists = batch_detect(self.face_detector, images)
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/faceutils/face_detection/detection/sfd/detect.py", line 76, in batch_detect
        olist = net(imgs)
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py", line 914, in __call__
        outputs = self.forward(*inputs, **kwargs)
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/ppgan/faceutils/face_detection/detection/sfd/net_s3fd.py", line 133, in forward
        h = F.relu(self.conv1_1(x))
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/fluid/dygraph/layers.py", line 914, in __call__
        outputs = self.forward(*inputs, **kwargs)
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/nn/layer/conv.py", line 665, in forward
        out = F.conv._conv_nd(
      File "/root/anaconda3/envs/paddle_env/lib/python3.9/site-packages/paddle/nn/functional/conv.py", line 123, in _conv_nd
        pre_bias = getattr(_C_ops, op_type)(x, weight, *attrs)
    OSError: (External) CUDNN error(9), CUDNN_STATUS_NOT_SUPPORTED.
      [Hint: 'CUDNN_STATUS_NOT_SUPPORTED'.  The functionality requested is not presently supported by cuDNN.  ] (at /paddle/paddle/fluid/platform/cudnn_desc.h:171)
      [operator < conv2d > error]
    
    opened by congling 6
  • cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils'

    cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils'

    运行general_demo.py报错如下: D:\InstallPath\Develop\Anaconda3\2020.07\envs\PaddleBoBo_Py3.7\python.exe G:/Project/AIBroadcast/PaddleBoBo/general_demo.py Traceback (most recent call last): File "G:/Project/AIBroadcast/PaddleBoBo/general_demo.py", line 6, in from PaddleTools.TTS import TTSExecutor File "G:\Project\AIBroadcast\PaddleBoBo\PaddleTools\TTS.py", line 11, in from paddlespeech.cli.utils import MODEL_HOME ImportError: cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils' (D:\InstallPath\Develop\Anaconda3\2020.07\envs\PaddleBoBo_Py3.7\lib\site-packages\paddlespeech\cli\utils.py)

    opened by WestbrookZero 5
  • gpen 模型是用哪个版本保存的,是不是有错误

    gpen 模型是用哪个版本保存的,是不是有错误

    [01/10 10:12:33] ppgan INFO: Found /root/.cache/ppgan/GPEN-512.pdparams Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/paddle/framework/io.py", line 936, in load load_result = pickle.load(f) _pickle.UnpicklingError: pickle data was truncated

    opened by ziyoujiyi 5
  • 关于GPU内存问题

    关于GPU内存问题

    #python general_demo.py --human ./file/input/test.mp4 --output ../output.mp4 --text 各位开发者大家好,我是您的专属虚拟主播,很高兴能为您服务。

    结果告诉我内存不足,Out of memory error on GPU 0. Cannot allocate 4.722656GB memory on GPU 0, 13.023560GB memory has been allocated and available memory is only 2.975952GB.

    我的机器是3080,16G显存,不管这句话多长都是分配这么多内存,配置建议多少啊?

    opened by congling 4
  • Out of memory error 请问如何解决啊

    Out of memory error 请问如何解决啊

    Traceback (most recent call last):
      File "E:/Graduate/PaddleBoBo-develop/create_virtual_human.py", line 23, in <module>
        cvh = FOM(GAN_Config['FOM_INPUT_IMAGE'],GAN_Config['FOM_DRIVING_VIDEO'],GAN_Config['FOM_OUTPUT_VIDEO'])
      File "E:\Graduate\PaddleBoBo-develop\PaddleTools\GAN.py", line 24, in FOM
        first_order_predictor.run(source_image, driving_video)
      File "E:\Python\environment\lib\site-packages\ppgan\apps\first_order_predictor.py", line 184, in run
        bboxes = self.extract_bbox(source_image.copy())
      File "E:\Python\environment\lib\site-packages\ppgan\apps\first_order_predictor.py", line 324, in extract_bbox
        predictions = detector.get_detections_for_image(np.array(frame))
      File "E:\Python\environment\lib\site-packages\ppgan\faceutils\face_detection\api.py", line 90, in get_detections_for_image
        detected_faces = self.face_detector.detect_from_batch(images.copy())
      File "E:\Python\environment\lib\site-packages\ppgan\faceutils\face_detection\detection\sfd\sfd_detector.py", line 56, in detect_from_batch
        bboxlists = batch_detect(self.face_detector, images)
      File "E:\Python\environment\lib\site-packages\ppgan\faceutils\face_detection\detection\sfd\detect.py", line 76, in batch_detect
        olist = net(imgs)
      File "E:\Python\environment\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in __call__
        return self._dygraph_call_func(*inputs, **kwargs)
      File "E:\Python\environment\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
        outputs = self.forward(*inputs, **kwargs)
      File "E:\Python\environment\lib\site-packages\ppgan\faceutils\face_detection\detection\sfd\net_s3fd.py", line 133, in forward
        h = F.relu(self.conv1_1(x))
      File "E:\Python\environment\lib\site-packages\paddle\fluid\dygraph\layers.py", line 917, in __call__
        return self._dygraph_call_func(*inputs, **kwargs)
      File "E:\Python\environment\lib\site-packages\paddle\fluid\dygraph\layers.py", line 907, in _dygraph_call_func
        outputs = self.forward(*inputs, **kwargs)
      File "E:\Python\environment\lib\site-packages\paddle\nn\layer\conv.py", line 665, in forward
        out = F.conv._conv_nd(
      File "E:\Python\environment\lib\site-packages\paddle\nn\functional\conv.py", line 123, in _conv_nd
        pre_bias = getattr(_C_ops, op_type)(x, weight, *attrs)
    SystemError: (Fatal) Operator conv2d raises an struct paddle::memory::allocation::BadAlloc exception.
    The exception content is
    :ResourceExhaustedError: 
    
    Out of memory error on GPU 0. Cannot allocate 297.650635MB memory on GPU 0, 1.821357GB memory has been allocated and available memory is only 182.805861MB.
    
    Please check whether there is any other process using GPU 0.
    1. If yes, please stop them, or start PaddlePaddle on another GPU.
    2. If no, please decrease the batch size of your model. 
    
     (at ..\paddle\fluid\memory\allocation\cuda_allocator.cc:79)
    . (at ..\paddle\fluid\imperative\tracer.cc:221)
    

    提示GPU内存不够 然后提供了两种解决方案:

    • 一是在另外一个GPU运行,我尝试过 import os os.environ['CUDA_VISIBLE_DEVICES'] = '1' 但还是相同的报错
    • 二是减少batch_size,我尽可能搜索网站信息没有找到如何改变batch_size的帖子,在这个项目里面我没有找到batch_size的关键词,后来我又在项目外(paddle中搜索)结果也不尽人意,尝试过很多比如直接添加batch_size=1,运行无果。甚至Out of memory error on GPU 0. Cannot allocate 297.650635MB memory on GPU 0, 1.821357GB memory has been allocated and available memory is only 182.805861MB.这些数值都没有改变。 小白在线求大佬们解惑~!
    opened by Merlin-xy 3
  • paddlespeech 报错:  cannot import name 'load' from 'paddleaudio.backends

    paddlespeech 报错: cannot import name 'load' from 'paddleaudio.backends

    我的环境: paddlepaddle-gpu :2.3.2 ppgan 2.1.0 paddlespeech 1.0.1 paddleaudio 1.0.2

    /content/drive/MyDrive/Virtual_broadcast /usr/local/lib/python3.7/dist-packages/librosa/core/constantq.py:1059: DeprecationWarning: np.complex is a deprecated alias for the builtin complex. To silence this warning, use complex by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.complex128 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations dtype=np.complex, Traceback (most recent call last): File "general_demo.py", line 6, in from PaddleTools.TTS import TTSExecutor File "/content/drive/MyDrive/Virtual_broadcast/PaddleTools/TTS.py", line 10, in from paddlespeech.cli.utils import download_and_decompress File "/usr/local/lib/python3.7/dist-packages/paddlespeech/cli/init.py", line 24, in from .vector import VectorExecutor File "/usr/local/lib/python3.7/dist-packages/paddlespeech/cli/vector/init.py", line 14, in from .infer import VectorExecutor File "/usr/local/lib/python3.7/dist-packages/paddlespeech/cli/vector/infer.py", line 25, in from paddleaudio.backends import load as load_audio ImportError: cannot import name 'load' from 'paddleaudio.backends' (/usr/local/lib/python3.7/dist-packages/paddleaudio/backends/init.py)

    opened by xiaoxifei1223 2
  • ValueError: --face argument must be a valid path to video/image file

    ValueError: --face argument must be a valid path to video/image file

    Traceback (most recent call last): File "general_demo.py", line 18, in video = wav2lip(args.human,wavfile,args.output) #将音频合成到唇形视频 File "/Users/apple/Downloads/PaddleBoBo-develop/PaddleTools/GAN.py", line 12, in wav2lip wav2lip_predictor.run(input_video, input_audio, output) File "/Users/apple/miniconda3/envs/py37/lib/python3.7/site-packages/ppgan/apps/wav2lip_predictor.py", line 177, in run '--face argument must be a valid path to video/image file') ValueError: --face argument must be a valid path to video/image file

    opened by yangboz 2
  • ImportError: cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils' (/root/anaconda3/envs/pdpd_py37_xnr/lib/python3.7/site-packages/paddlespeech/cli/utils.py)

    ImportError: cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils' (/root/anaconda3/envs/pdpd_py37_xnr/lib/python3.7/site-packages/paddlespeech/cli/utils.py)

    ImportError: cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils' (/root/anaconda3/envs/pdpd_py37_xnr/lib/python3.7/site-packages/paddlespeech/cli/utils.py)

    各位老板有报这个错的么???

    opened by wangw8451 1
  • 大神求救,Windows平台报找不到文件的错误呢

    大神求救,Windows平台报找不到文件的错误呢

    Traceback (most recent call last): File "D:\jellyfin\PaddleBoBo\general_demo.py", line 18, in video = wav2lip(args.human,wavfile,args.output) #将音频合成到唇形视频 File "D:\jellyfin\PaddleBoBo\PaddleTools\GAN.py", line 11, in wav2lip wav2lip_predictor.run(input_video, input_audio, output) File "C:\Users\ycy\AppData\Roaming\Python\Python39\site-packages\ppgan\apps\wav2lip_predictor.py", line 290, in run subprocess.call(command, shell=platform.system() != 'Windows') File "C:\Program Files\Python39\lib\subprocess.py", line 349, in call with Popen(*popenargs, **kwargs) as p: File "C:\Program Files\Python39\lib\subprocess.py", line 951, in init self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Program Files\Python39\lib\subprocess.py", line 1420, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] 系统找不到指定的文件。

    opened by ycy1164656 1
  • 运行PaddleBoBo报错MODEL_HOME的解决办法

    运行PaddleBoBo报错MODEL_HOME的解决办法

    如果大家在运行中发现了问题,那一般是库的版本问题,我自己看了看paddlespeech的库,我终于发现了他的改动,在之前的时候,<=1.0的时候,这个都是正确的。但是如果paddlespeech默认下载了1.1或者1.2的版本,在PaddleTools中的TTS.py的11行的MODEL_HOME中,MODEL_HOME就会出现问题,这是因为换位置了,所以我们需要将其换为,

    from paddlespeech.utils.env import MODEL_HOME

    亲测有效,或者安装paddlespeech为<=1.0的版本也可以,就直接不用改代码了

    opened by Dreaming-future 1
  • 运行demo报错,提示不能导入某个包

    运行demo报错,提示不能导入某个包

    您好, 执行python general_demo.py 报错。麻烦这个问题如何解决,谢谢 Traceback (most recent call last): File "general_demo.py", line 6, in from PaddleTools.TTS import TTSExecutor File "/data/hwpang/2020_dev/gan/PaddleBoBo/PaddleTools/TTS.py", line 11, in from paddlespeech.cli.utils import MODEL_HOME ImportError: cannot import name 'MODEL_HOME' from 'paddlespeech.cli.utils' (/root/anaconda3/envs/paddle/lib/python3.8/site-packages/paddlespeech/cli/utils.py)

    opened by panghongwei17 1
  • error: (-215:Assertion failed) !ssize.empty() in function 'cv::resize' 如何解决

    error: (-215:Assertion failed) !ssize.empty() in function 'cv::resize' 如何解决

    Traceback (most recent call last): File "create_virtual_human.py", line 13, in cvh = FOM(GAN_Config['FOM_INPUT_IMAGE'],GAN_Config['FOM_DRIVING_VIDEO'],GAN_Config['FOM_OUTPUT_VIDEO']) File "E:\python_project\PaddleBoBo\PaddleTools\GAN.py", line 23, in FOM first_order_predictor.run(source_image, driving_video) File "C:\Users\13277.conda\envs\virtual_human\lib\site-packages\ppgan\apps\first_order_predictor.py", line 190, in run face_image = cv2.resize(face_image, (self.image_size, self.image_size)) / 255.0 cv2.error: OpenCV(4.6.0) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\resize.cpp:4052: error: (-215:Assertion failed) !ssize.empty() in function 'cv::resize'

    opened by mambasmile 0
  • 请问贵团队有在其他数据集上训练wav2lip吗

    请问贵团队有在其他数据集上训练wav2lip吗

    各位老师好,目前wav2lip官方的模型(基于LRS2)训练的,应用到其他人脸(例如亚洲人脸),嘴型驱动部分存在很强的模糊,并且嘴型很多时候是错误的。基于此,我想在自己的专人数据集上训练此wav2lip模型。我的数据为两个小时的专人视频,在训练Lip-sync Expert嘴型和音频同步网络时候,loss一直不下降,卡在0.6左右。请问老师们能给我一点建议吗 谢谢 image

    opened by mrlihellohorld 0
Owner
null