Kaldi-compatible feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and autograd

Fangjun Kuang

Last update: Jan 3, 2023

Related tags

Overview

kaldifeat

Comments	Options	Feature Computer	Usage
FBANK	`kaldifeat.FbankOptions`	`kaldifeat.Fbank`	opts = kaldifeat.FbankOptions() opts.device = torch.device('cuda', 0) opts.frame_opts.window_type = 'povey' fbank = kaldifeat.Fbank(opts) features = fbank(wave)
MFCC	`kaldifeat.MfccOptions`	`kaldifeat.Mfcc`	opts = kaldifeat.MfccOptions(); opts.num_ceps = 13 mfcc = kaldifeat.Mfcc(opts) features = mfcc(wave)
PLP	`kaldifeat.PlpOptions`	`kaldifeat.Plp`	opts = kaldifeat.PlpOptions(); opts.mel_opts.num_bins = 23 plp = kaldifeat.Plp(opts) features = plp(wave)
Spectorgram	`kaldifeat.SpectrogramOptions`	`kaldifeat.Spectrogram`	opts = kaldifeat.SpectrogramOptions(); print(opts) spectrogram = kaldifeat.Spectrogram(opts) features = spectrogram(wave)

_{Comments
Options
Feature Computer
Usage

FBANK
kaldifeat.FbankOptions
kaldifeat.Fbank

opts = kaldifeat.FbankOptions()
opts.device = torch.device('cuda', 0)
opts.frame_opts.window_type = 'povey'
fbank = kaldifeat.Fbank(opts)
features = fbank(wave)

MFCC
kaldifeat.MfccOptions
kaldifeat.Mfcc

opts = kaldifeat.MfccOptions();
opts.num_ceps = 13
mfcc = kaldifeat.Mfcc(opts)
features = mfcc(wave)

PLP
kaldifeat.PlpOptions
kaldifeat.Plp

opts = kaldifeat.PlpOptions();
opts.mel_opts.num_bins = 23
plp = kaldifeat.Plp(opts)
features = plp(wave)

Spectorgram
kaldifeat.SpectrogramOptions
kaldifeat.Spectrogram

opts = kaldifeat.SpectrogramOptions();
print(opts)
spectrogram = kaldifeat.Spectrogram(opts)
features = spectrogram(wave)}

Feature extraction compatible with Kaldi using PyTorch, supporting CUDA, batch processing, chunk processing, and autograd.

The following kaldi-compatible commandline tools are implemented:

compute-fbank-feats
compute-mfcc-feats
compute-plp-feats
compute-spectrogram-feats

(NOTE: We will implement other types of features, e.g., Pitch, ivector, etc, soon.)

Usage

Let us first generate a test wave using sox:

# generate a wave of 1.2 seconds, containing a sine-wave
# swept from 300 Hz to 3300 Hz
sox -n -r 16000 -b 16 test.wav synth 1.2 sine 300-3300

HINT: Download test.wav.

Fbank

import torchaudio

import kaldifeat

filename = "./test.wav"
wave, samp_freq = torchaudio.load(filename)

wave = wave.squeeze()

opts = kaldifeat.FbankOptions()
opts.frame_opts.dither = 0
# Yes, it has same options like `Kaldi`

fbank = kaldifeat.Fbank(opts)
features = fbank(wave)

To compute features that are compatible with Kaldi, wave samples have to be scaled to the range [-32768, 32768]. WARNING: You don't have to do this if you don't care about the compatibility with Kaldi.

The following is an example:

wave *= 32768
fbank = kaldifeat.Fbank(opts)
features = fbank(wave)
print(features[:3])

The output is:

tensor([[15.0074, 21.1730, 25.5286, 24.4644, 16.6994, 13.8480, 11.2087, 11.7952,
         10.3911, 10.4491, 10.3012,  9.8743,  9.6997,  9.3751,  9.3476,  9.3559,
          9.1074,  9.0032,  9.0312,  8.8399,  9.0822,  8.7442,  8.4023],
        [13.8785, 20.5647, 25.4956, 24.6966, 16.9541, 13.9163, 11.3364, 11.8449,
         10.2565, 10.5871, 10.3484,  9.7474,  9.6123,  9.3964,  9.0695,  9.1177,
          8.9136,  8.8425,  8.5920,  8.8315,  8.6226,  8.8605,  8.9763],
        [13.9475, 19.9410, 25.4494, 24.9051, 17.0004, 13.9207, 11.6667, 11.8217,
         10.3411, 10.7258, 10.0983,  9.8109,  9.6762,  9.4218,  9.1246,  8.7744,
          9.0863,  8.7488,  8.4695,  8.6710,  8.7728,  8.7405,  8.9824]])

You can compute the fbank feature for the same wave with Kaldi using the following commands:

test.scp compute-fbank-feats --dither=0 scp:test.scp ark,t:test.txt head -n4 test.txt ">

echo "1 test.wav" > test.scp
compute-fbank-feats --dither=0 scp:test.scp ark,t:test.txt
head -n4 test.txt

The output is:

1  [
  15.00744 21.17303 25.52861 24.46438 16.69938 13.84804 11.2087 11.79517 10.3911 10.44909 10.30123 9.874329 9.699727 9.37509 9.347578 9.355928 9.107419 9.00323 9.031268 8.839916 9.082197 8.744139 8.40221
  13.87853 20.56466 25.49562 24.69662 16.9541 13.91633 11.33638 11.84495 10.25656 10.58718 10.34841 9.747416 9.612316 9.39642 9.06955 9.117751 8.913527 8.842571 8.59212 8.831518 8.622513 8.86048 8.976251
  13.94753 19.94101 25.4494 24.90511 17.00044 13.92074 11.66673 11.82172 10.34108 10.72575 10.09829 9.810879 9.676199 9.421767 9.124647 8.774353 9.086291 8.74897 8.469534 8.670973 8.772754 8.740549 8.982433

You can see that kaldifeat produces the same output as Kaldi (within some tolerance due to numerical precision).

HINT: Download test.scp and test.txt.

To use GPU, you can use:

import torch

opts = kaldifeat.FbankOptions()
opts.device = torch.device("cuda", 0)

fbank = kaldifeat.Fbank(opts)
features = fbank(wave.to(opts.device))

MFCC, PLP, Spectrogram

To compute MFCC features, please replace kaldifeat.FbankOptions and kaldifeat.Fbank with kaldifeat.MfccOptions and kaldifeat.Mfcc, respectively. The same goes for PLP and Spectrogram.

Please refer to

for more examples.

HINT: In the examples, you can find that

kaldifeat supports batch processing as well as chunk processing
kaldifeat uses the same options as Kaldi's compute-fbank-feats and compute-mfcc-feats

Installation

From PyPi with pip

If you install kaldifeat using pip, it will also install PyTorch 1.8.1. If this is not what you want (i.e, you have installed a different version of PyTorch and you don't want to replace it with PyTorch 1.8.1), please add an option --no-dependencies to pip install.

pip install kaldifeat                     # also installs torch 1.8.1
pip install --no-dependencies kaldifeat   # will NOT install torch 1.8.1

From source

The following are the commands to compile kaldifeat from source. We assume that you have installed cmake and PyTorch. cmake 3.11 is known to work. Other cmake versions may also work. PyTorch 1.8.1 is known to work. Other PyTorch versions may also work.

mkdir /some/path
git clone https://github.com/csukuangfj/kaldifeat.git
cd kaldifeat
python setup.py install

To test whether kaldifeat was installed successfully, you can run:

python3 -c "import kaldifeat; print(kaldifeat.__version__)"

Comments

Install error with both pip/conda (system MKL installed)
I can not install kaldifeat alongside with k2. Steps:

conda create --name k2 -c k2-fsa -c pytorch -c nvidia k2 pytorch pytorch-cuda=11.7 torchaudio => OK

conda install -c kaldifeat kaldifeat => Conflicting/ERROR: kaldifeat-cuda-install-error.log

Conda failed so I've tried to install via pip. Steps:

pip install --verbose kaldifeat => Failed with message saying that it can't find cuDNN: kaldifeat-pip-install-error.log

Try to install cuDNN from conda: conda install -c conda-forge cudnn => OK with cuDNN-conda-install-message:

Retry install kaldifeat via pip: pip install --verbose kaldifeat: .......annd- it failed with message saying that can not find mkl: kaldifeat-pip-install-error-cudnn-installed.log

Following the suggestion from #10, and it failed with problem of cuda: k2-pip-install-error-mkl-provided.log

Asking myself: "I have installed/trained/ran k2, icefall, kaldifeat, sherpa in this way in a server running ubuntu 20.04 why I can't reproduce it in my local machine, so I exported conda environment of the server and my local machine.... And I still can not make it work :/". Here is the two conda environments: conda environments

Trying everything for a day without hope and go here to call for help....

cuDNN-conda-install-message:

(k2) conda install -c conda-forge cudnn Collecting package metadata (current_repodata.json): done Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 22.9.0 latest version: 22.11.1 Please update conda by running $ conda update -n base -c conda-forge conda ## Package Plan ## environment location: /home/trungle/opt/anaconda3/envs/k2 added / updated specs: - cudnn The following NEW packages will be INSTALLED: cudatoolkit conda-forge/linux-64::cudatoolkit-10.2.89-h713d32c_10 None cudnn conda-forge/linux-64::cudnn-7.6.5.32-h01f27c4_1 None The following packages will be UPDATED: ca-certificates pkgs/main::ca-certificates-2022.10.11~ --> conda-forge::ca-certificates-2022.12.7-ha878542_0 None certifi pkgs/main/linux-64::certifi-2022.9.24~ --> conda-forge/noarch::certifi-2022.12.7-pyhd8ed1ab_0 None
opened by trunglebka 28

Cannot find `kaldifeat` after install (make wheel failed)

Hi, I am facing an installation issue. I first set the KALDIFEAT_CMAKE_ARGS to point to my CUDNN_LIBRARY_PATH and CUDNN_INCLUDE_PATH (as suggested in this issue), and the installation went through. However, it seems there was a problem creating the wheel for the package, so the installation used setup.py install. See the logs below:

Using pip 21.2.4 from /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/pip (python 3.8)
Collecting kaldifeat
  Using cached kaldifeat-1.12.tar.gz (471 kB)
    Running command python setup.py egg_info
    running egg_info
    creating /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info
    writing /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/dependency_links.txt
    writing top-level names to /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/top_level.txt
    writing manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
    reading manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no previously-included files found matching 'pyproject.toml'
    adding license file 'LICENSE'
    writing manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
Building wheels for collected packages: kaldifeat
  Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __fil
e__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
 setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6uer8h92
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-3.8/kaldifeat
  copying kaldifeat/python/kaldifeat/__init__.py -> build/lib.linux-x86_64-3.8/kaldifeat
  running build_ext
  For fast compilation, run:
  export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install
  Setting PYTHON_EXECUTABLE to /home/draj/anaconda3/envs/scale/bin/python
  build command is:

              cd build/temp.linux-x86_64-3.8

              cmake -DCUDNN_LIBRARY_PATH=/home/smielke/cuda-cudnn/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/home/smielke/cuda-cudnn/include -DPYTHON_EXECUTABLE=/home/draj/anaconda3/envs/scale/bin/python /tmp/pip-install-
vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36


              make  _kaldifeat

  -- The C compiler identification is GNU 6.3.0
  -- The CXX compiler identification is GNU 6.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- No CMAKE_BUILD_TYPE given, default to Release
  -- C++ Standard version: 14
  -- Downloading pybind11
  -- pybind11 is downloaded to /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/pybind11-src
  -- pybind11 v2.6.0
CMake Warning (dev) at /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/cmake/data/share/cmake-3.22/Modules/CMakeDependentOption.cmake:84 (message):
    Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
    Syntax.  Run "cmake --help-policy CMP0127" for policy details.  Use the
    cmake_policy command to set the policy and suppress this warning.
  Call Stack (most recent call first):
    build/temp.linux-x86_64-3.8/_deps/pybind11-src/CMakeLists.txt:91 (cmake_dependent_option)
  This warning is for project developers.  Use -Wno-dev to suppress it.

  -- Found PythonInterp: /home/draj/anaconda3/envs/scale/bin/python (found version "3.8.12")
  -- Found PythonLibs: /home/draj/anaconda3/envs/scale/lib/libpython3.8.so
  -- Performing Test HAS_FLTO
  -- Performing Test HAS_FLTO - Success
  -- Python executable: /home/draj/anaconda3/envs/scale/bin/python
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Found CUDA: /usr/local/cuda (found version "10.2")
  -- Caffe2: CUDA detected: 10.2
  -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
  -- Caffe2: CUDA toolkit directory: /usr/local/cuda
  -- Caffe2: Header version is: 10.2
  -- Found CUDNN: /home/smielke/cuda-cudnn/lib64/libcudnn.so
  -- Found cuDNN: v7.6.5  (include: /home/smielke/cuda-cudnn/include, library: /home/smielke/cuda-cudnn/lib64/libcudnn.so)
  -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 08c4863f
  -- Autodetected CUDA architecture(s):  6.1 6.1 6.1 6.1
  -- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
  -- Found Torch: /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/torch/lib/libtorch.so
  -- PyTorch version: 1.8.1
  -- Downloading googletest
  -- googletest is downloaded to /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/googletest-src
  -- googletest's binary dir is /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/googletest-build
  CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/CMakeLists.txt:4 (cmake_minimum_required):
    Compatibility with CMake < 2.8.12 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.


  CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
    Compatibility with CMake < 2.8.12 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.


  CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/googletest/CMakeLists.txt:56 (cmake_minimum_required):
    Compatibility with CMake < 2.8.12 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.


  -- Configuring done
  CMake Warning at build/temp.linux-x86_64-3.8/_deps/pybind11-src/tools/pybind11Tools.cmake:147 (add_library):
    Cannot generate a safe runtime search path for target _kaldifeat because
    files in some directories may conflict with libraries in implicit
    directories:

      runtime library [libcublas.so.10] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
        /home/draj/anaconda3/envs/scale/lib

    Some of these libraries may not be found correctly.
  Call Stack (most recent call first):
    kaldifeat/python/csrc/CMakeLists.txt:2 (pybind11_add_module)


  -- Generating done
  -- Build files have been written to: /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8
  [  5%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
  In file included from /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-fbank.h:13:0,
                   from /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-fbank.cc:7:
  /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-common.h:11:43: fatal error: kaldifeat/csrc/feature-window.h: No such file or directory
  compilation terminated.
  kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/build.make:75: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o' failed
  make[3]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o] Error 1
  CMakeFiles/Makefile2:340: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all' failed
  make[2]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all] Error 2
  CMakeFiles/Makefile2:427: recipe for target 'kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule' failed
  make[1]: *** [kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule] Error 2
  Makefile:225: recipe for target '_kaldifeat' failed
  make: *** [_kaldifeat] Error 2
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py", line 32, in <module>
      include_dirs=[np.get_include()],
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
      _build_ext.build_ext.run(self)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
      _build_ext.build_ext.build_extensions(self)
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/cmake/cmake_extension.py", line 86, in build_extension
  Exception:
  Build kaldifeat failed. Please check the error message.
  You can ask for help by creating an issue on GitHub.

  Click:
        https://github.com/csukuangfj/kaldifeat/issues/new

  Building wheel for kaldifeat (setup.py) ... error
  ERROR: Failed building wheel for kaldifeat
  Running setup.py clean for kaldifeat
  Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
  running clean
  removing 'build/temp.linux-x86_64-3.8' (and everything under it)
  'build/lib' does not exist -- can't clean it
  'build/bdist.linux-x86_64' does not exist -- can't clean it
  'build/scripts-3.8' does not exist -- can't clean it
Failed to build kaldifeat
Installing collected packages: kaldifeat
    Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wxsok0gm/install-record.txt --single-version-externally-managed --compile --install-headers /home/draj/anaconda3/envs/scale/include/python3.8/kaldifeat
    running install
    running build
    running install_egg_info
    running egg_info
    creating UNKNOWN.egg-info
    writing UNKNOWN.egg-info/PKG-INFO
    writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
    writing top-level names to UNKNOWN.egg-info/top_level.txt
    writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
    reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
    writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
    removing '/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info' (and everything under it)
    Copying UNKNOWN.egg-info to /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info
    running install_scripts
    writing list of installed files to '/tmp/pip-record-wxsok0gm/install-record.txt'
    Running setup.py install for kaldifeat ... done
  DEPRECATION: kaldifeat was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Successfully installed kaldifeat

However, the installation is not successful since import kaldifeat cannot find the module. Could you please suggest what may be going wrong?

opened by desh2608 20

Centos: ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file: No such file or directory

I installed kaldifeat by using python setup.py install. But it came out the follow errors:

ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file: No such file or directory

But i do found libmkl_intel_ilp64.so in the environment lib directory.

opened by PPGGG 16
Taking forever to install kaldifeat

Hello, I am trying to install kaldifeat following the instructions in the doc, with this command: pip install --verbose kaldifeat

It's been ~8 hours now but it seems the program got stuck at this place:

I was wondering if this is normal or if perhaps I should ask for some help now. Thank you!

opened by huangruizhe 11

Failed to install kaldifeat from source

I tried installing kaldifeat from source since both pip and conda installation failed. I followed the suggestions in FAQ

export KALDIFEAT_CMAKE_ARGS="-DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/include"

Then I run the python setup.py install

I could not capture the complete logs but I attached the CMakeOutput.log, and CMakeError.log

The installation started well but then failed for some reason, I managed to capture some of the logs manually because The command python setup.py install &>log gave file with unreadable data.

opying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
copying kaldifeat/python/kaldifeat/__init__.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
running build_ext
Setting PYTHON_EXECUTABLE to /home/local/QCRI/ahussein/anaconda3/envs/k2/bin/python
For fast compilation, run:
export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install
Setting make_args to '-j4'
build command is:

                cd build/temp.linux-x86_64-cpython-38

                cmake -DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/include -DPYTHON_EXECUTABLE=/home/local/QCRI/ahussein/anaconda3/envs/k2/bin/python -Dkaldifeat_BUILD_TESTS=OFF  -DCMAKE_INSTALL_PREFIX=/alt-arabic/speech/amir/k2/tmp/kaldifeat/build/lib.linux-x86_64-cpython-38/kaldifeat  /alt-arabic/speech/amir/k2/tmp/kaldifeat


                make  -j4  _kaldifeat install

-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /cm/shared/apps/gcc8/8.4.0/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /cm/shared/apps/gcc8/8.4.0/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- C++ Standard version: 14
-- Downloading pybind11

  Cannot generate a safe runtime search path for target _kaldifeat because
  files in some directories may conflict with libraries in implicit
  directories:

    runtime library [libcufft.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    runtime library [libcurand.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    runtime library [libcublas.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    runtime library [libnvrtc.so.10.2] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    runtime library [libnvToolsExt.so.1] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    runtime library [libcudart.so.10.2] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
      /home/local/QCRI/ahussein/anaconda3/envs/k2/lib

  Some of these libraries may not be found correctly.
Call Stack (most recent call first):
  kaldifeat/python/csrc/CMakeLists.txt:2 (pybind11_add_module)


-- Generating done
-- Build files have been written to: /alt-arabic/speech/amir/k2/tmp/kaldifeat/build/temp.linux-x86_64-cpython-38
Scanning dependencies of target kaldifeat_core
[ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
[ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
[ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
[ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
[ 25%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
[ 30%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
[ 35%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
[ 40%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
[ 45%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/online-feature.cc.o
[ 50%] Linking CXX shared library ../../lib/libkaldifeat_core.so
[ 50%] Built target kaldifeat_core
Scanning dependencies of target _kaldifeat
[ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-fbank.cc.o
[ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-mfcc.cc.o
[ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-plp.cc.o
[ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-spectrogram.cc.o
[ 80%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/kaldifeat.cc.o
[ 85%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/mel-computations.cc.o
[ 90%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/online-feature.cc.o
[ 95%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/utils.cc.o
[100%] Linking CXX shared module ../../../lib/_kaldifeat.cpython-38-x86_64-linux-gnu.so
[100%] Built target _kaldifeat
[ 45%] Built target kaldifeat_core
Scanning dependencies of target test_kaldifeat
[ 90%] Built target _kaldifeat
[ 95%] Building CXX object kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/test_kaldifeat.cc.o

 File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run

    _build_ext.run(self)
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
    self.build_extensions()
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
    self._build_extensions_serial()
  File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
    self.build_extension(ext)
  File "/alt-arabic/speech/amir/k2/tmp/kaldifeat/cmake/cmake_extension.py", line 124, in build_extension
    raise Exception(
Exception:
Build kaldifeat failed. Please check the error message.

CMakeOutput.log CMakeError.log

opened by AmirHussein96 9

Build error: /usr/bin/ld: cannot find /path/to/miniconda3/envs/k2s/lib: File format not recognized

Hi,

I am trying to install package from source. I set cmake arguments like this*. However I got "" error.

-- Generating done
-- Build files have been written to: /path/to/k2/kaldifeat/build/temp.linux-x86_64-3.8
Scanning dependencies of target kaldifeat_core
[  5%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
[ 11%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
[ 16%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
[ 22%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
[ 27%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
[ 33%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
[ 38%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
[ 44%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
[ 50%] Linking CXX shared library ../../lib/libkaldifeat_core.so
/usr/bin/ld: cannot find /path/to/miniconda3/envs/k2s/lib: File format not recognized
collect2: error: ld returned 1 exit status
kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/build.make:203: recipe for target 'lib/libkaldifeat_core.so' failed
make[3]: *** [lib/libkaldifeat_core.so] Error 1
CMakeFiles/Makefile2:393: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all' failed
make[2]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all] Error 2
CMakeFiles/Makefile2:454: recipe for target 'kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule' failed
make[1]: *** [kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule] Error 2
Makefile:220: recipe for target '_kaldifeat' failed
make: *** [_kaldifeat] Error 2

export KALDIFEAT_CMAKE_ARGS='-DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH="path/to/miniconda3/envs/k2s/lib" -DCUDNN_INCLUDE_PATH="path/to/miniconda3/envs/k2s/include"'

opened by EmreOzkose 8

changes for using C++ API by other projects
I use the C++ API in wenet runtime with FetchContent. But kaldifeat's CMakeLists.txt use CMAKE_SOURCE_DIR, which is path to wenet/runtime/libtorch, then kaldifeat couldn't find its own cmake module. so, I make two changes to make kaldifeat working fine as a dependenty project to other projects.:

change it to CMAKE_CURRENT_SOURCE_DIR.

add option kaldifeat_BUILD_PYMODULE, which can be set OFF by other projects. we only use C++ api, but kaldifeat default to build python module by add_subdirectory(python)

But, without these changes, kaldifeat works fine with k2, why? Because there're cmake modules in k2/cmake/ used by kaldifeat, in other words, kaldifeat use k2's cmake modules not its own.

~~With the 2nd change, k2 needs set kaldifeat_BUILD_PYMODULE to OFF in k2/cmake/kaldifeat.cmake.~~ k2 needs no change.
opened by veelion 6

fix tensor device type incompatibility

When I tried to extract fbank with this code:

opts = kaldifeat.FbankOptions()
opts.device = torch.device('cuda', 0)
opts.frame_opts.window_type = 'povey'
fbank = kaldifeat.Fbank(opts)
features = fbank(wave)

I got this error

Traceback (most recent call last):
  File "extract_feature.py", line 15, in <module>
    features = fbank(wave)
  File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/kaldifeat-1.10-py3.8-linux-x86_64.egg/kaldifeat/offline_feature.py", line 75, in forward
    features = self.compute(strided, vtln_warp)
  File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/kaldifeat-1.10-py3.8-linux-x86_64.egg/kaldifeat/offline_feature.py", line 111, in compute
    features = self.computer.compute_features(x, vtln_warp)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Hence I added .to(device) to related tensor

opened by EmreOzkose 5

_kaldifeat.get_strided got error

hi, when i run a pretrained model under icefall, i used this command : python ./pruned_transducer_stateless2/pretrained.py --decoding-method greedy_search --lang-dir ./luomingshuang_pruned_transducer_stateless2/data/lang_char --max-sym-per-frame 1 --sample-rate 16000 --checkpoint ./luomingshuang_pruned_transducer_stateless2/exp/pretrained_epoch_10_avg_2.pt ./luomingshuang_pruned_transducer_stateless2/test_wavs/VID_175648.wav

and i got this error: File "D:\ProgramData\Anaconda3\envs\python36\lib\site-packages\kaldifeat-1.18-py3.6-win-amd64.egg\kaldifeat\offline_feature.py", line 155, in convert_samples_to_frames return _kaldifeat.get_strided(wave, self.opts.frame_opts) RuntimeError: setStorage: sizes [1504, 400], strides [320, 2], storage offset 0, and itemsize 4 requiring a storage size of 1927036 are out of bounds for storage of size 963520

i searched everywhere and got no idea, please help, thank you!

opened by taotaoyuhust 4

Ubuntu/Debian `pip install kaldifeat`: cannot find CUDNN

either doing pip install kaldifeat, or cloning source and doing python setup.py install, yields the following message:

...
-- Found CUDA: /usr/local/cuda (found version "10.2")
-- Caffe2: CUDA detected: 10.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 10.2
-- Could NOT find CUDNN (missing: CUDNN_LIBRARY_PATH CUDNN_INCLUDE_PATH)
CMake Warning at /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:109 (message):
  Caffe2: Cannot find cuDNN library.  Turning the option off
Call Stack (most recent call first):
  /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  cmake/torch.cmake:13 (find_package)
  CMakeLists.txt:41 (include)

I am able to build it by doing this:

<inside sources root>
mkdir build
cd build
cmake .. -DCUDNN_LIBRARY_PATH="/home/pzelasko/miniconda3/envs/lhotse/lib" -DCUDNN_INCLUDE_PATH="/home/pzelasko/miniconda3/envs/lhotse/include"

But there seems to be no way to pass these values through env vars when doing pip install kaldifeat.

opened by pzelasko 4

FbankOpts (and others) conversion to/from dict
Would it be possible to add a conversion mechanism for the Opts classes to/from dict? It would make it much easier for me to integrate kaldifeat in Lhotse. I'm using dataclasses for configuration of all feature extractors, like this:

https://github.com/lhotse-speech/lhotse/blob/f5bd2063291b7d789d4b8a2a50396d2ffa76eb2c/lhotse/features/fbank.py#L9-L31

Example API could look like this:

# Read a dict config = {"frame_shift": 0.012, "use_energy": True} extractor = kaldifeat.Fbank(kaldifeat.FbankOptions.from_dict(config)) # Store to a dict config2 = extractor.opts.to_dict() # config2 = {... a list of all settings ...}
opened by pzelasko 4

cl: 命令行 error D8021 :无效的数值参数“/Wno-register”

c /Tpextensions/align.cpp /Fobuild\temp.win-amd64-3.9\Release\extensions/align.obj -std=c++11 -Wno-register -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char
      cl: 命令行 error D8021 :无效的数值参数“/Wno-register”
      error: command 'D:\\apps\\VisualStudio\\VC\\Tools\\MSVC\\14.33.31629\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

windows无法安装。

opened by jinfagang 15

/lib64/libm.so.6: version `GLIBC_2.27' not found (required by python3.7/site-packages/libkaldifeat_core.so

import kaldifeat Traceback (most recent call last): File "", line 1, in File "/home/cyn244124/espnet/espnet/tools/anaconda/envs/espnet/lib/python3.7/site-packages/kaldifeat/init.py", line 2, in from _kaldifeat import FbankOptions, FrameExtractionOptions, MelBanksOptions ImportError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /lib/python3.7/site-packages/libkaldifeat_core.so)

opened by maxandchen 3
python3 setup.py install error
running install /root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( /root/anaconda3/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( running bdist_egg running egg_info writing kaldifeat.egg-info/PKG-INFO writing dependency_links to kaldifeat.egg-info/dependency_links.txt writing top-level names to kaldifeat.egg-info/top_level.txt reading manifest file 'kaldifeat.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'kaldifeat.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py copying kaldifeat/python/kaldifeat/torch_version.py -> build/lib.linux-x86_64-3.9/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-3.9/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /root/anaconda3/bin/python3 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

cd build/temp.linux-x86_64-3.9 cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/root/anaconda3/bin/python3 -Dkaldifeat_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/root/kaldifeat/build/lib.linux-x86_64-3.9/kaldifeat /root/kaldifeat make _kaldifeat install

-- C++ Standard version: 14 -- Downloading pybind11 -- pybind11 is downloaded to /root/kaldifeat/build/temp.linux-x86_64-3.9/_deps/pybind11-src -- pybind11 v2.9.2 -- Python executable: /root/anaconda3/bin/python3 -- TORCH_DIR: /root/anaconda3/lib/python3.9/site-packages/torch -- Caffe2: CUDA detected: 11.3 -- Caffe2: CUDA nvcc is: /usr/local/cuda-11.3/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.3 -- Caffe2: Header version is: 11.3 -- Found cuDNN: v8.2.1 (include: /usr/local/cuda-11.3/include, library: /usr/local/cuda-11.3/lib64/libcudnn.so) -- /usr/local/cuda-11.3/lib64/libnvrtc.so shorthash is 8aa72235 -- Autodetected CUDA architecture(s): 6.0 6.0 6.0 -- Added CUDA NVCC flags for: -gencode;arch=compute_60,code=sm_60 CMake Warning at /root/anaconda3/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /root/anaconda3/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:14 (find_package) CMakeLists.txt:55 (include)

-- PyTorch version: 1.12.1 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -- CMAKE_INSTALL_PREFIX: /root/kaldifeat/build/lib.linux-x86_64-3.9/kaldifeat -- All headers: /root/kaldifeat/kaldifeat/csrc/feature-common-inl.h;/root/kaldifeat/kaldifeat/csrc/feature-common.h;/root/kaldifeat/kaldifeat/csrc/feature-fbank.h;/root/kaldifeat/kaldifeat/csrc/feature-functions.h;/root/kaldifeat/kaldifeat/csrc/feature-mfcc.h;/root/kaldifeat/kaldifeat/csrc/feature-plp.h;/root/kaldifeat/kaldifeat/csrc/feature-spectrogram.h;/root/kaldifeat/kaldifeat/csrc/feature-window.h;/root/kaldifeat/kaldifeat/csrc/log.h;/root/kaldifeat/kaldifeat/csrc/matrix-functions.h;/root/kaldifeat/kaldifeat/csrc/mel-computations.h;/root/kaldifeat/kaldifeat/csrc/online-feature-itf.h;/root/kaldifeat/kaldifeat/csrc/online-feature.h;/root/kaldifeat/kaldifeat/csrc/pitch-functions.h -- Configuring done -- Generating done -- Build files have been written to: /root/kaldifeat/build/temp.linux-x86_64-3.9 [ 50%] Built target kaldifeat_core [100%] Built target _kaldifeat [ 45%] Built target kaldifeat_core [ 50%] Linking CXX executable ../../bin/test_kaldifeat /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to [email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to[email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to [email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to[email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to `[email protected]' collect2: error: ld returned 1 exit status make[2]: *** [kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/build.make:98: bin/test_kaldifeat] Error 1 make[1]: *** [CMakeFiles/Makefile2:172: kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/all] Error 2 make: *** [Makefile:130: all] Error 2 Traceback (most recent call last): File "/root/kaldifeat/setup.py", line 32, in setuptools.setup( File "/root/anaconda3/lib/python3.9/site-packages/setuptools/init.py", line 87, in setup return distutils.core.setup(**attrs) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup return run_commands(dist) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands dist.run_commands() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands self.run_command(cmd) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 74, in run self.do_egg_install() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 123, in do_egg_install self.run_command('bdist_egg') File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 165, in run cmd = self.call_command('install_lib', warn_dir=0) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command self.run_command(cmdname) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install_lib.py", line 107, in build self.run_command('build_ext') File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run _build_ext.run(self) File "/root/anaconda3/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run self.build_extensions() File "/root/anaconda3/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "/root/kaldifeat/cmake/cmake_extension.py", line 122, in build_extension raise Exception( Exception: Build kaldifeat failed. Please check the error message. You can ask for help by creating an issue on GitHub.

Click: https://github.com/csukuangfj/kaldifeat/issues/new
opened by farmer21cn 13

fail to install kaldifeat in conda

pip install kaldifeat Looking in indexes: http://mirrors.cloud.tencent.com/pypi/simple Collecting kaldifeat Using cached http://mirrors.cloud.tencent.com/pypi/packages/2d/7b/0a5c9254de6a62cdca7bea829bd8d5d646e18508d34f9214f048ee003fed/kaldifeat-1.17.tar.gz (477 kB) Preparing metadata (setup.py) ... done Building wheels for collected packages: kaldifeat Building wheel for kaldifeat (setup.py) ... -

error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [184 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-38 creating build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-cpython-38/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

                  cd build/temp.linux-x86_64-cpython-38
  
                  cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783
  
  
                  make  _kaldifeat
  
  -- The C compiler identification is GNU 11.3.0
  -- The CXX compiler identification is GNU 11.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- C++ Standard version: 14
  -- Downloading pybind11
  -- pybind11 is downloaded to /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/_deps/pybind11-src
  -- pybind11 v2.9.2
  -- Found PythonInterp: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 (found version "3.8.13")
  -- Found PythonLibs: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libpython3.8.so
  -- Performing Test HAS_FLTO
  -- Performing Test HAS_FLTO - Success
  -- Python executable: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Found CUDA: /cfs/user/burkliu/dev/anaconda3/envs/k2_m (found version "11.7")
  -- The CUDA compiler identification is NVIDIA 11.7.64
  -- Detecting CUDA compiler ABI info
  -- Detecting CUDA compiler ABI info - done
  -- Check for working CUDA compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc - skipped
  -- Detecting CUDA compile features
  -- Detecting CUDA compile features - done
  -- Caffe2: CUDA detected: 11.7
  -- Caffe2: CUDA nvcc is: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc
  -- Caffe2: CUDA toolkit directory: /cfs/user/burkliu/dev/anaconda3/envs/k2_m
  CMake Error at /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:88 (message):
    Caffe2: Couldn't determine version from header: Change Dir:
    /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp
  
  
  
  
    Run Build Command(s):/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
    Makefile cmTC_e257c/fast &&
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
    CMakeFiles/cmTC_e257c.dir/build.make CMakeFiles/cmTC_e257c.dir/build
  
    make[1]: Entering directory
    '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
  
  
    Building CXX object CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
    -I/cfs/user/burkliu/dev/anaconda3/envs/k2_m/include -std=c++14 -o
    CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o -c
    /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/detect_cuda_version.cc
  
  
    Linking CXX executable cmTC_e257c
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cmake -E cmake_link_script
    CMakeFiles/cmTC_e257c.dir/link.txt --verbose=1
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
    CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o -o cmTC_e257c
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so
  
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
    reference to `memcpy@GLIBC_2.14'
  
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
    reference to `clock_gettime@GLIBC_2.17'
  
    collect2: error: ld returned 1 exit status
  
    make[1]: *** [CMakeFiles/cmTC_e257c.dir/build.make:100: cmTC_e257c] Error 1
  
    make[1]: Leaving directory
    '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
  
  
    make: *** [Makefile:127: cmTC_e257c/fast] Error 2
  
  
  
  Call Stack (most recent call first):
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    cmake/torch.cmake:13 (find_package)
    CMakeLists.txt:48 (include)
  
  
  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeError.log".
  make: *** No rule to make target '_kaldifeat'.  Stop.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/setup.py", line 32, in <module>
      setuptools.setup(
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
      self.run_command(cmd)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
      super().run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
      self.build_extensions()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
      self._build_extensions_serial()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/cmake/cmake_extension.py", line 110, in build_extension
      raise Exception(
  Exception:
  Build kaldifeat failed. Please check the error message.
  You can ask for help by creating an issue on GitHub.
  
  Click:
      https://github.com/csukuangfj/kaldifeat/issues/new
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for kaldifeat Running setup.py clean for kaldifeat Failed to build kaldifeat Installing collected packages: kaldifeat Running setup.py install for kaldifeat ... error error: subprocess-exited-with-error

× Running setup.py install for kaldifeat did not run successfully. │ exit code: 1 ╰─> [188 lines of output] running install /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.linux-x86_64-cpython-38 creating build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-cpython-38/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

                  cd build/temp.linux-x86_64-cpython-38
  
                  cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783
  
  
                  make  _kaldifeat
  
  -- The C compiler identification is GNU 11.3.0
  -- The CXX compiler identification is GNU 11.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- C++ Standard version: 14
  -- Downloading pybind11
  -- pybind11 is downloaded to /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/_deps/pybind11-src
  -- pybind11 v2.9.2
  -- Found PythonInterp: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 (found version "3.8.13")
  -- Found PythonLibs: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libpython3.8.so
  -- Performing Test HAS_FLTO
  -- Performing Test HAS_FLTO - Success
  -- Python executable: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Found CUDA: /cfs/user/burkliu/dev/anaconda3/envs/k2_m (found version "11.7")
  -- The CUDA compiler identification is NVIDIA 11.7.64
  -- Detecting CUDA compiler ABI info
  -- Detecting CUDA compiler ABI info - done
  -- Check for working CUDA compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc - skipped
  -- Detecting CUDA compile features
  -- Detecting CUDA compile features - done
  -- Caffe2: CUDA detected: 11.7
  -- Caffe2: CUDA nvcc is: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc
  -- Caffe2: CUDA toolkit directory: /cfs/user/burkliu/dev/anaconda3/envs/k2_m
  CMake Error at /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:88 (message):
    Caffe2: Couldn't determine version from header: Change Dir:
    /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp
  
  
  
  
    Run Build Command(s):/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
    Makefile cmTC_a8777/fast &&
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
    CMakeFiles/cmTC_a8777.dir/build.make CMakeFiles/cmTC_a8777.dir/build
  
    make[1]: Entering directory
    '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
  
  
    Building CXX object CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
    -I/cfs/user/burkliu/dev/anaconda3/envs/k2_m/include -std=c++14 -o
    CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o -c
    /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/detect_cuda_version.cc
  
  
    Linking CXX executable cmTC_a8777
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cmake -E cmake_link_script
    CMakeFiles/cmTC_a8777.dir/link.txt --verbose=1
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
    CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o -o cmTC_a8777
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so
  
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
    reference to `memcpy@GLIBC_2.14'
  
  
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
    reference to `clock_gettime@GLIBC_2.17'
  
    collect2: error: ld returned 1 exit status
  
    make[1]: *** [CMakeFiles/cmTC_a8777.dir/build.make:100: cmTC_a8777] Error 1
  
    make[1]: Leaving directory
    '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
  
  
    make: *** [Makefile:127: cmTC_a8777/fast] Error 2
  
  
  
  Call Stack (most recent call first):
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
    /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
    cmake/torch.cmake:13 (find_package)
    CMakeLists.txt:48 (include)
  
  
  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeError.log".
  make: *** No rule to make target '_kaldifeat'.  Stop.
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/setup.py", line 32, in <module>
      setuptools.setup(
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
      self.run_command(cmd)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/install.py", line 698, in run
      self.run_command('build')
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
      super().run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
      cmd_obj.run()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
      self.build_extensions()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
      self._build_extensions_serial()
    File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
      self.build_extension(ext)
    File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/cmake/cmake_extension.py", line 110, in build_extension
      raise Exception(
  Exception:
  Build kaldifeat failed. Please check the error message.
  You can ask for help by creating an issue on GitHub.
  
  Click:
      https://github.com/csukuangfj/kaldifeat/issues/new
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> kaldifeat

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.

opened by boji123 25

potential memory leak

Hi~

I use valgrind to test the below line:

import kaldifeat

The log is here: https://1drv.ms/t/s!AhtoTbISXXlSgSLtqDWjo8RP_ouA?e=Sn1cd0

Log summary:

==20510== LEAK SUMMARY:
==20510==    definitely lost: 0 bytes in 0 blocks
==20510==    indirectly lost: 0 bytes in 0 blocks
==20510==      possibly lost: 271,997 bytes in 129 blocks
==20510==    still reachable: 5,881,625 bytes in 22,065 blocks
==20510==                       of which reachable via heuristic:
==20510==                         stdstring          : 521,021 bytes in 6,954 blocks
==20510==         suppressed: 58,517 bytes in 5 blocks
==20510==
==20510== For lists of detected and suppressed errors, rerun with: -s
==20510== ERROR SUMMARY: 17745 errors from 434 contexts (suppressed: 14 from 2)

Reproduce

valgrind --leak-check=full --show-reachable=yes --trace-children=yes --track-origins=yes --trace-children=yes --log-file='log.txt' python3 test.py

test.py

import kaldifeat

Could you help check if these are potential issues? Thanks!

opened by Slyne 3

Releases(v1.22)

v1.22(Dec 3, 2022)

Source code(tar.gz)
Source code(zip)
v1.21(Nov 30, 2022)

Source code(tar.gz)
Source code(zip)
v1.20(Oct 28, 2022)

Source code(tar.gz)
Source code(zip)
v1.19(Aug 20, 2022)

Support building conda packages for macOS and Windows.
Source code(tar.gz)
Source code(zip)
v1.18(Aug 1, 2022)

Support make install.

See https://github.com/csukuangfj/kaldifeat-cmake-example about using kaldifeat in a cmake-based project with C++.
Source code(tar.gz)
Source code(zip)
v1.17(Jul 4, 2022)

Support android.
Source code(tar.gz)
Source code(zip)
v1.16(Jun 4, 2022)
Support windows.

Release the GIL for online feature extraction.

Source code(tar.gz)
Source code(zip)
v1.15(May 22, 2022)

Release the GIL in offline feature extration
Source code(tar.gz)
Source code(zip)
v1.14(Apr 8, 2022)

Support building on Windows.
Source code(tar.gz)
Source code(zip)
v1.13(Apr 2, 2022)

Support streaming feature extractors.
Source code(tar.gz)
Source code(zip)
v1.12(Dec 1, 2021)

Move audio data to GPU in a chunk-wise manner to avoid CUDA OOM for long utterances, i.e., those of more than 20 hours long.
Source code(tar.gz)
Source code(zip)
v1.11(Nov 28, 2021)
Fix chunkwise feature extraction. It was not used by a mistake and caused CUDA OOM for long utterances, i.e., those of 9 hours long.

Source code(tar.gz)
Source code(zip)
v1.10(Nov 4, 2021)

Add pickling support.
Source code(tar.gz)
Source code(zip)
v1.9(Oct 31, 2021)

Source code(tar.gz)
Source code(zip)
v1.8(Oct 15, 2021)

Add from_dict and as_dict to various Options in kaldifeat
Source code(tar.gz)
Source code(zip)
v1.7(Oct 11, 2021)

Support using kaldifeat in k2.
Source code(tar.gz)
Source code(zip)
v1.6(Aug 20, 2021)
Support installation with conda

Support torch 1.6.0 and above

Source code(tar.gz)
Source code(zip)
v1.5.4(Aug 18, 2021)

Let the user build kaldifeat from source so that the locally installed PyTorch can be reused.
Source code(tar.gz)
Source code(zip)
v1.4(Jul 19, 2021)

Support macOS.
Source code(tar.gz)
Source code(zip)
v1.3(Jul 18, 2021)

Support PLP features
Source code(tar.gz)
Source code(zip)
v1.2(Jul 17, 2021)
Support MFCC and Fbank

Support chunk processing

Support batch processing

More examples in the doc

Source code(tar.gz)
Source code(zip)