Kaldi-compatible feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and autograd

Overview

kaldifeat

Comments Options Feature Computer Usage
FBANK kaldifeat.FbankOptions kaldifeat.Fbank
opts = kaldifeat.FbankOptions()
opts.device = torch.device('cuda', 0)
opts.frame_opts.window_type = 'povey'
fbank = kaldifeat.Fbank(opts)
features = fbank(wave)
MFCC kaldifeat.MfccOptions kaldifeat.Mfcc
opts = kaldifeat.MfccOptions();
opts.num_ceps = 13
mfcc = kaldifeat.Mfcc(opts)
features = mfcc(wave)
PLP kaldifeat.PlpOptions kaldifeat.Plp
opts = kaldifeat.PlpOptions();
opts.mel_opts.num_bins = 23
plp = kaldifeat.Plp(opts)
features = plp(wave)
Spectorgram kaldifeat.SpectrogramOptions kaldifeat.Spectrogram
opts = kaldifeat.SpectrogramOptions();
print(opts)
spectrogram = kaldifeat.Spectrogram(opts)
features = spectrogram(wave)

Feature extraction compatible with Kaldi using PyTorch, supporting CUDA, batch processing, chunk processing, and autograd.

The following kaldi-compatible commandline tools are implemented:

  • compute-fbank-feats
  • compute-mfcc-feats
  • compute-plp-feats
  • compute-spectrogram-feats

(NOTE: We will implement other types of features, e.g., Pitch, ivector, etc, soon.)

Usage

Let us first generate a test wave using sox:

# generate a wave of 1.2 seconds, containing a sine-wave
# swept from 300 Hz to 3300 Hz
sox -n -r 16000 -b 16 test.wav synth 1.2 sine 300-3300

HINT: Download test.wav.

Fbank

import torchaudio

import kaldifeat

filename = "./test.wav"
wave, samp_freq = torchaudio.load(filename)

wave = wave.squeeze()

opts = kaldifeat.FbankOptions()
opts.frame_opts.dither = 0
# Yes, it has same options like `Kaldi`

fbank = kaldifeat.Fbank(opts)
features = fbank(wave)

To compute features that are compatible with Kaldi, wave samples have to be scaled to the range [-32768, 32768]. WARNING: You don't have to do this if you don't care about the compatibility with Kaldi.

The following is an example:

wave *= 32768
fbank = kaldifeat.Fbank(opts)
features = fbank(wave)
print(features[:3])

The output is:

tensor([[15.0074, 21.1730, 25.5286, 24.4644, 16.6994, 13.8480, 11.2087, 11.7952,
         10.3911, 10.4491, 10.3012,  9.8743,  9.6997,  9.3751,  9.3476,  9.3559,
          9.1074,  9.0032,  9.0312,  8.8399,  9.0822,  8.7442,  8.4023],
        [13.8785, 20.5647, 25.4956, 24.6966, 16.9541, 13.9163, 11.3364, 11.8449,
         10.2565, 10.5871, 10.3484,  9.7474,  9.6123,  9.3964,  9.0695,  9.1177,
          8.9136,  8.8425,  8.5920,  8.8315,  8.6226,  8.8605,  8.9763],
        [13.9475, 19.9410, 25.4494, 24.9051, 17.0004, 13.9207, 11.6667, 11.8217,
         10.3411, 10.7258, 10.0983,  9.8109,  9.6762,  9.4218,  9.1246,  8.7744,
          9.0863,  8.7488,  8.4695,  8.6710,  8.7728,  8.7405,  8.9824]])

You can compute the fbank feature for the same wave with Kaldi using the following commands:

test.scp compute-fbank-feats --dither=0 scp:test.scp ark,t:test.txt head -n4 test.txt ">
echo "1 test.wav" > test.scp
compute-fbank-feats --dither=0 scp:test.scp ark,t:test.txt
head -n4 test.txt

The output is:

1  [
  15.00744 21.17303 25.52861 24.46438 16.69938 13.84804 11.2087 11.79517 10.3911 10.44909 10.30123 9.874329 9.699727 9.37509 9.347578 9.355928 9.107419 9.00323 9.031268 8.839916 9.082197 8.744139 8.40221
  13.87853 20.56466 25.49562 24.69662 16.9541 13.91633 11.33638 11.84495 10.25656 10.58718 10.34841 9.747416 9.612316 9.39642 9.06955 9.117751 8.913527 8.842571 8.59212 8.831518 8.622513 8.86048 8.976251
  13.94753 19.94101 25.4494 24.90511 17.00044 13.92074 11.66673 11.82172 10.34108 10.72575 10.09829 9.810879 9.676199 9.421767 9.124647 8.774353 9.086291 8.74897 8.469534 8.670973 8.772754 8.740549 8.982433

You can see that kaldifeat produces the same output as Kaldi (within some tolerance due to numerical precision).

HINT: Download test.scp and test.txt.

To use GPU, you can use:

import torch

opts = kaldifeat.FbankOptions()
opts.device = torch.device("cuda", 0)

fbank = kaldifeat.Fbank(opts)
features = fbank(wave.to(opts.device))

MFCC, PLP, Spectrogram

To compute MFCC features, please replace kaldifeat.FbankOptions and kaldifeat.Fbank with kaldifeat.MfccOptions and kaldifeat.Mfcc, respectively. The same goes for PLP and Spectrogram.

Please refer to

for more examples.

HINT: In the examples, you can find that

  • kaldifeat supports batch processing as well as chunk processing
  • kaldifeat uses the same options as Kaldi's compute-fbank-feats and compute-mfcc-feats

Installation

From PyPi with pip

If you install kaldifeat using pip, it will also install PyTorch 1.8.1. If this is not what you want (i.e, you have installed a different version of PyTorch and you don't want to replace it with PyTorch 1.8.1), please add an option --no-dependencies to pip install.

pip install kaldifeat                     # also installs torch 1.8.1
pip install --no-dependencies kaldifeat   # will NOT install torch 1.8.1

From source

The following are the commands to compile kaldifeat from source. We assume that you have installed cmake and PyTorch. cmake 3.11 is known to work. Other cmake versions may also work. PyTorch 1.8.1 is known to work. Other PyTorch versions may also work.

mkdir /some/path
git clone https://github.com/csukuangfj/kaldifeat.git
cd kaldifeat
python setup.py install

To test whether kaldifeat was installed successfully, you can run:

python3 -c "import kaldifeat; print(kaldifeat.__version__)"
Comments
  • Install error with both pip/conda (system MKL installed)

    Install error with both pip/conda (system MKL installed)

    I can not install kaldifeat alongside with k2. Steps:

    1. conda create --name k2 -c k2-fsa -c pytorch -c nvidia k2 pytorch pytorch-cuda=11.7 torchaudio => OK
    2. conda install -c kaldifeat kaldifeat => Conflicting/ERROR: kaldifeat-cuda-install-error.log

    Conda failed so I've tried to install via pip. Steps:

    1. pip install --verbose kaldifeat => Failed with message saying that it can't find cuDNN: kaldifeat-pip-install-error.log
    2. Try to install cuDNN from conda: conda install -c conda-forge cudnn => OK with cuDNN-conda-install-message:
    3. Retry install kaldifeat via pip: pip install --verbose kaldifeat: .......annd- it failed with message saying that can not find mkl: kaldifeat-pip-install-error-cudnn-installed.log
    4. Following the suggestion from #10, and it failed with problem of cuda: k2-pip-install-error-mkl-provided.log
    5. Asking myself: "I have installed/trained/ran k2, icefall, kaldifeat, sherpa in this way in a server running ubuntu 20.04 why I can't reproduce it in my local machine, so I exported conda environment of the server and my local machine.... And I still can not make it work :/". Here is the two conda environments: conda environments
    6. Trying everything for a day without hope and go here to call for help....

    cuDNN-conda-install-message:

    (k2) conda install -c conda-forge cudnn
    Collecting package metadata (current_repodata.json): done
    Solving environment: done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 22.9.0
      latest version: 22.11.1
    
    Please update conda by running
    
        $ conda update -n base -c conda-forge conda
    
    
    
    ## Package Plan ##
    
      environment location: /home/trungle/opt/anaconda3/envs/k2
    
      added / updated specs:
        - cudnn
    
    
    The following NEW packages will be INSTALLED:
    
      cudatoolkit        conda-forge/linux-64::cudatoolkit-10.2.89-h713d32c_10 None
      cudnn              conda-forge/linux-64::cudnn-7.6.5.32-h01f27c4_1 None
    
    The following packages will be UPDATED:
    
      ca-certificates    pkgs/main::ca-certificates-2022.10.11~ --> conda-forge::ca-certificates-2022.12.7-ha878542_0 None
      certifi            pkgs/main/linux-64::certifi-2022.9.24~ --> conda-forge/noarch::certifi-2022.12.7-pyhd8ed1ab_0 None
    
    
    opened by trunglebka 28
  • Cannot find `kaldifeat` after install (make wheel failed)

    Cannot find `kaldifeat` after install (make wheel failed)

    Hi, I am facing an installation issue. I first set the KALDIFEAT_CMAKE_ARGS to point to my CUDNN_LIBRARY_PATH and CUDNN_INCLUDE_PATH (as suggested in this issue), and the installation went through. However, it seems there was a problem creating the wheel for the package, so the installation used setup.py install. See the logs below:

    Using pip 21.2.4 from /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/pip (python 3.8)
    Collecting kaldifeat
      Using cached kaldifeat-1.12.tar.gz (471 kB)
        Running command python setup.py egg_info
        running egg_info
        creating /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info
        writing /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/PKG-INFO
        writing dependency_links to /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/dependency_links.txt
        writing top-level names to /tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/top_level.txt
        writing manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
        reading manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no previously-included files found matching 'pyproject.toml'
        adding license file 'LICENSE'
        writing manifest file '/tmp/pip-pip-egg-info-llwy3qa0/kaldifeat.egg-info/SOURCES.txt'
    Building wheels for collected packages: kaldifeat
      Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __fil
    e__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import
     setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6uer8h92
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.8
      creating build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-3.8/kaldifeat
      copying kaldifeat/python/kaldifeat/__init__.py -> build/lib.linux-x86_64-3.8/kaldifeat
      running build_ext
      For fast compilation, run:
      export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install
      Setting PYTHON_EXECUTABLE to /home/draj/anaconda3/envs/scale/bin/python
      build command is:
    
                  cd build/temp.linux-x86_64-3.8
    
                  cmake -DCUDNN_LIBRARY_PATH=/home/smielke/cuda-cudnn/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/home/smielke/cuda-cudnn/include -DPYTHON_EXECUTABLE=/home/draj/anaconda3/envs/scale/bin/python /tmp/pip-install-
    vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36
    
    
                  make  _kaldifeat
    
      -- The C compiler identification is GNU 6.3.0
      -- The CXX compiler identification is GNU 6.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- No CMAKE_BUILD_TYPE given, default to Release
      -- C++ Standard version: 14
      -- Downloading pybind11
      -- pybind11 is downloaded to /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/pybind11-src
      -- pybind11 v2.6.0
    CMake Warning (dev) at /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/cmake/data/share/cmake-3.22/Modules/CMakeDependentOption.cmake:84 (message):
        Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
        Syntax.  Run "cmake --help-policy CMP0127" for policy details.  Use the
        cmake_policy command to set the policy and suppress this warning.
      Call Stack (most recent call first):
        build/temp.linux-x86_64-3.8/_deps/pybind11-src/CMakeLists.txt:91 (cmake_dependent_option)
      This warning is for project developers.  Use -Wno-dev to suppress it.
    
      -- Found PythonInterp: /home/draj/anaconda3/envs/scale/bin/python (found version "3.8.12")
      -- Found PythonLibs: /home/draj/anaconda3/envs/scale/lib/libpython3.8.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Python executable: /home/draj/anaconda3/envs/scale/bin/python
      -- Looking for pthread.h
      -- Looking for pthread.h - found
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - found
      -- Found Threads: TRUE
      -- Found CUDA: /usr/local/cuda (found version "10.2")
      -- Caffe2: CUDA detected: 10.2
      -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
      -- Caffe2: CUDA toolkit directory: /usr/local/cuda
      -- Caffe2: Header version is: 10.2
      -- Found CUDNN: /home/smielke/cuda-cudnn/lib64/libcudnn.so
      -- Found cuDNN: v7.6.5  (include: /home/smielke/cuda-cudnn/include, library: /home/smielke/cuda-cudnn/lib64/libcudnn.so)
      -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 08c4863f
      -- Autodetected CUDA architecture(s):  6.1 6.1 6.1 6.1
      -- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
      -- Found Torch: /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/torch/lib/libtorch.so
      -- PyTorch version: 1.8.1
      -- Downloading googletest
      -- googletest is downloaded to /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/googletest-src
      -- googletest's binary dir is /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8/_deps/googletest-build
      CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/CMakeLists.txt:4 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.
    
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
    
    
      CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.
    
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
    
    
      CMake Deprecation Warning at build/temp.linux-x86_64-3.8/_deps/googletest-src/googletest/CMakeLists.txt:56 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.
    
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
    
    
      -- Configuring done
      CMake Warning at build/temp.linux-x86_64-3.8/_deps/pybind11-src/tools/pybind11Tools.cmake:147 (add_library):
        Cannot generate a safe runtime search path for target _kaldifeat because
        files in some directories may conflict with libraries in implicit
        directories:
    
          runtime library [libcublas.so.10] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
            /home/draj/anaconda3/envs/scale/lib
    
        Some of these libraries may not be found correctly.
      Call Stack (most recent call first):
        kaldifeat/python/csrc/CMakeLists.txt:2 (pybind11_add_module)
    
    
      -- Generating done
      -- Build files have been written to: /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/build/temp.linux-x86_64-3.8
      [  5%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
      In file included from /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-fbank.h:13:0,
                       from /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-fbank.cc:7:
      /tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/kaldifeat/csrc/feature-common.h:11:43: fatal error: kaldifeat/csrc/feature-window.h: No such file or directory
      compilation terminated.
      kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/build.make:75: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o' failed
      make[3]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o] Error 1
      CMakeFiles/Makefile2:340: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all' failed
      make[2]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all] Error 2
      CMakeFiles/Makefile2:427: recipe for target 'kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule' failed
      make[1]: *** [kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule] Error 2
      Makefile:225: recipe for target '_kaldifeat' failed
      make: *** [_kaldifeat] Error 2
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py", line 32, in <module>
          include_dirs=[np.get_include()],
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 340, in run
          self.build_extensions()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/home/draj/anaconda3/envs/scale/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/cmake/cmake_extension.py", line 86, in build_extension
      Exception:
      Build kaldifeat failed. Please check the error message.
      You can ask for help by creating an issue on GitHub.
    
      Click:
            https://github.com/csukuangfj/kaldifeat/issues/new
    
      Building wheel for kaldifeat (setup.py) ... error
      ERROR: Failed building wheel for kaldifeat
      Running setup.py clean for kaldifeat
      Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
      running clean
      removing 'build/temp.linux-x86_64-3.8' (and everything under it)
      'build/lib' does not exist -- can't clean it
      'build/bdist.linux-x86_64' does not exist -- can't clean it
      'build/scripts-3.8' does not exist -- can't clean it
    Failed to build kaldifeat
    Installing collected packages: kaldifeat
        Running command /home/draj/anaconda3/envs/scale/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vti1wm7p/kaldifeat_083a70eca61440dbbddb6865da1e6d36/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wxsok0gm/install-record.txt --single-version-externally-managed --compile --install-headers /home/draj/anaconda3/envs/scale/include/python3.8/kaldifeat
        running install
        running build
        running install_egg_info
        running egg_info
        creating UNKNOWN.egg-info
        writing UNKNOWN.egg-info/PKG-INFO
        writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
        writing top-level names to UNKNOWN.egg-info/top_level.txt
        writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
        reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
        writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
        removing '/home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info' (and everything under it)
        Copying UNKNOWN.egg-info to /home/draj/anaconda3/envs/scale/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8.egg-info
        running install_scripts
        writing list of installed files to '/tmp/pip-record-wxsok0gm/install-record.txt'
        Running setup.py install for kaldifeat ... done
      DEPRECATION: kaldifeat was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
    Successfully installed kaldifeat
    

    However, the installation is not successful since import kaldifeat cannot find the module. Could you please suggest what may be going wrong?

    opened by desh2608 20
  • Centos: ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file: No such file or directory

    Centos: ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file: No such file or directory

    I installed kaldifeat by using python setup.py install. But it came out the follow errors:

    ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file: No such file or directory

    But i do found libmkl_intel_ilp64.so in the environment lib directory.

    opened by PPGGG 16
  • Taking forever to install kaldifeat

    Taking forever to install kaldifeat

    Hello, I am trying to install kaldifeat following the instructions in the doc, with this command: pip install --verbose kaldifeat

    It's been ~8 hours now but it seems the program got stuck at this place: image

    I was wondering if this is normal or if perhaps I should ask for some help now. Thank you!

    opened by huangruizhe 11
  • Failed to install kaldifeat from source

    Failed to install kaldifeat from source

    I tried installing kaldifeat from source since both pip and conda installation failed. I followed the suggestions in FAQ

    export KALDIFEAT_CMAKE_ARGS="-DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/include"
    

    Then I run the python setup.py install

    I could not capture the complete logs but I attached the CMakeOutput.log, and CMakeError.log

    The installation started well but then failed for some reason, I managed to capture some of the logs manually because The command python setup.py install &>log gave file with unreadable data.

    opying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
    copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
    copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
    copying kaldifeat/python/kaldifeat/__init__.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
    copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat
    running build_ext
    Setting PYTHON_EXECUTABLE to /home/local/QCRI/ahussein/anaconda3/envs/k2/bin/python
    For fast compilation, run:
    export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install
    Setting make_args to '-j4'
    build command is:
    
                    cd build/temp.linux-x86_64-cpython-38
    
                    cmake -DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/lib64/libcudnn.so -DCUDNN_INCLUDE_PATH=/cm/shared/apps/cudnn8.0-cuda10.2/8.0.5.39/include -DPYTHON_EXECUTABLE=/home/local/QCRI/ahussein/anaconda3/envs/k2/bin/python -Dkaldifeat_BUILD_TESTS=OFF  -DCMAKE_INSTALL_PREFIX=/alt-arabic/speech/amir/k2/tmp/kaldifeat/build/lib.linux-x86_64-cpython-38/kaldifeat  /alt-arabic/speech/amir/k2/tmp/kaldifeat
    
    
                    make  -j4  _kaldifeat install
    
    -- The C compiler identification is GNU 8.4.0
    -- The CXX compiler identification is GNU 8.4.0
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /cm/shared/apps/gcc8/8.4.0/bin/gcc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /cm/shared/apps/gcc8/8.4.0/bin/g++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- C++ Standard version: 14
    -- Downloading pybind11
    
      Cannot generate a safe runtime search path for target _kaldifeat because
      files in some directories may conflict with libraries in implicit
      directories:
    
        runtime library [libcufft.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
        runtime library [libcurand.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
        runtime library [libcublas.so.10] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
        runtime library [libnvrtc.so.10.2] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
        runtime library [libnvToolsExt.so.1] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
        runtime library [libcudart.so.10.2] in /cm/shared/apps/cuda10.2/toolkit/10.2.89/lib64 may be hidden by files in:
          /home/local/QCRI/ahussein/anaconda3/envs/k2/lib
    
      Some of these libraries may not be found correctly.
    Call Stack (most recent call first):
      kaldifeat/python/csrc/CMakeLists.txt:2 (pybind11_add_module)
    
    
    -- Generating done
    -- Build files have been written to: /alt-arabic/speech/amir/k2/tmp/kaldifeat/build/temp.linux-x86_64-cpython-38
    Scanning dependencies of target kaldifeat_core
    [ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
    [ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
    [ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
    [ 20%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
    [ 25%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
    [ 30%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
    [ 35%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
    [ 40%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
    [ 45%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/online-feature.cc.o
    [ 50%] Linking CXX shared library ../../lib/libkaldifeat_core.so
    [ 50%] Built target kaldifeat_core
    Scanning dependencies of target _kaldifeat
    [ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-fbank.cc.o
    [ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-mfcc.cc.o
    [ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-plp.cc.o
    [ 70%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/feature-spectrogram.cc.o
    [ 80%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/kaldifeat.cc.o
    [ 85%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/mel-computations.cc.o
    [ 90%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/online-feature.cc.o
    [ 95%] Building CXX object kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/utils.cc.o
    [100%] Linking CXX shared module ../../../lib/_kaldifeat.cpython-38-x86_64-linux-gnu.so
    [100%] Built target _kaldifeat
    [ 45%] Built target kaldifeat_core
    Scanning dependencies of target test_kaldifeat
    [ 90%] Built target _kaldifeat
    [ 95%] Building CXX object kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/test_kaldifeat.cc.o
    
     File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run
        self.build()
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
        self.run_command('build_ext')
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
        self.distribution.run_command(command)
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
        super().run_command(command)
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
    
        _build_ext.run(self)
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
        _build_ext.build_ext.run(self)
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
        self.build_extensions()
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
        _build_ext.build_ext.build_extensions(self)
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
        self._build_extensions_serial()
      File "/home/local/QCRI/ahussein/anaconda3/envs/k2/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
        self.build_extension(ext)
      File "/alt-arabic/speech/amir/k2/tmp/kaldifeat/cmake/cmake_extension.py", line 124, in build_extension
        raise Exception(
    Exception:
    Build kaldifeat failed. Please check the error message.
    
    

    CMakeOutput.log CMakeError.log

    opened by AmirHussein96 9
  • Build error: /usr/bin/ld: cannot find /path/to/miniconda3/envs/k2s/lib: File format not recognized

    Build error: /usr/bin/ld: cannot find /path/to/miniconda3/envs/k2s/lib: File format not recognized

    Hi,

    I am trying to install package from source. I set cmake arguments like this*. However I got "" error.

    -- Generating done
    -- Build files have been written to: /path/to/k2/kaldifeat/build/temp.linux-x86_64-3.8
    Scanning dependencies of target kaldifeat_core
    [  5%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
    [ 11%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
    [ 16%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
    [ 22%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
    [ 27%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
    [ 33%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
    [ 38%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
    [ 44%] Building CXX object kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
    [ 50%] Linking CXX shared library ../../lib/libkaldifeat_core.so
    /usr/bin/ld: cannot find /path/to/miniconda3/envs/k2s/lib: File format not recognized
    collect2: error: ld returned 1 exit status
    kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/build.make:203: recipe for target 'lib/libkaldifeat_core.so' failed
    make[3]: *** [lib/libkaldifeat_core.so] Error 1
    CMakeFiles/Makefile2:393: recipe for target 'kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all' failed
    make[2]: *** [kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/all] Error 2
    CMakeFiles/Makefile2:454: recipe for target 'kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule' failed
    make[1]: *** [kaldifeat/python/csrc/CMakeFiles/_kaldifeat.dir/rule] Error 2
    Makefile:220: recipe for target '_kaldifeat' failed
    make: *** [_kaldifeat] Error 2
    
    
    • export KALDIFEAT_CMAKE_ARGS='-DCMAKE_BUILD_TYPE=Release -DCUDNN_LIBRARY_PATH="path/to/miniconda3/envs/k2s/lib" -DCUDNN_INCLUDE_PATH="path/to/miniconda3/envs/k2s/include"'
    opened by EmreOzkose 8
  • changes for using C++ API by other projects

    changes for using C++ API by other projects

    I use the C++ API in wenet runtime with FetchContent. But kaldifeat's CMakeLists.txt use CMAKE_SOURCE_DIR, which is path to wenet/runtime/libtorch, then kaldifeat couldn't find its own cmake module. so, I make two changes to make kaldifeat working fine as a dependenty project to other projects.:

    1. change it to CMAKE_CURRENT_SOURCE_DIR.
    2. add option kaldifeat_BUILD_PYMODULE, which can be set OFF by other projects. we only use C++ api, but kaldifeat default to build python module by add_subdirectory(python)

    But, without these changes, kaldifeat works fine with k2, why? Because there're cmake modules in k2/cmake/ used by kaldifeat, in other words, kaldifeat use k2's cmake modules not its own.

    ~~With the 2nd change, k2 needs set kaldifeat_BUILD_PYMODULE to OFF in k2/cmake/kaldifeat.cmake.~~ k2 needs no change.

    opened by veelion 6
  • fix tensor device type incompatibility

    fix tensor device type incompatibility

    When I tried to extract fbank with this code:

    opts = kaldifeat.FbankOptions()
    opts.device = torch.device('cuda', 0)
    opts.frame_opts.window_type = 'povey'
    fbank = kaldifeat.Fbank(opts)
    features = fbank(wave)
    

    I got this error

    Traceback (most recent call last):
      File "extract_feature.py", line 15, in <module>
        features = fbank(wave)
      File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/kaldifeat-1.10-py3.8-linux-x86_64.egg/kaldifeat/offline_feature.py", line 75, in forward
        features = self.compute(strided, vtln_warp)
      File "/path/to/anaconda3/envs/general/lib/python3.8/site-packages/kaldifeat-1.10-py3.8-linux-x86_64.egg/kaldifeat/offline_feature.py", line 111, in compute
        features = self.computer.compute_features(x, vtln_warp)
    RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
    

    Hence I added .to(device) to related tensor

    opened by EmreOzkose 5
  • _kaldifeat.get_strided  got error

    _kaldifeat.get_strided got error

    hi, when i run a pretrained model under icefall, i used this command : python ./pruned_transducer_stateless2/pretrained.py --decoding-method greedy_search --lang-dir ./luomingshuang_pruned_transducer_stateless2/data/lang_char --max-sym-per-frame 1 --sample-rate 16000 --checkpoint ./luomingshuang_pruned_transducer_stateless2/exp/pretrained_epoch_10_avg_2.pt ./luomingshuang_pruned_transducer_stateless2/test_wavs/VID_175648.wav

    and i got this error: File "D:\ProgramData\Anaconda3\envs\python36\lib\site-packages\kaldifeat-1.18-py3.6-win-amd64.egg\kaldifeat\offline_feature.py", line 155, in convert_samples_to_frames return _kaldifeat.get_strided(wave, self.opts.frame_opts) RuntimeError: setStorage: sizes [1504, 400], strides [320, 2], storage offset 0, and itemsize 4 requiring a storage size of 1927036 are out of bounds for storage of size 963520

    i searched everywhere and got no idea, please help, thank you!

    opened by taotaoyuhust 4
  • Ubuntu/Debian `pip install kaldifeat`: cannot find CUDNN

    Ubuntu/Debian `pip install kaldifeat`: cannot find CUDNN

    either doing pip install kaldifeat, or cloning source and doing python setup.py install, yields the following message:

    ...
    -- Found CUDA: /usr/local/cuda (found version "10.2")
    -- Caffe2: CUDA detected: 10.2
    -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
    -- Caffe2: CUDA toolkit directory: /usr/local/cuda
    -- Caffe2: Header version is: 10.2
    -- Could NOT find CUDNN (missing: CUDNN_LIBRARY_PATH CUDNN_INCLUDE_PATH)
    CMake Warning at /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:109 (message):
      Caffe2: Cannot find cuDNN library.  Turning the option off
    Call Stack (most recent call first):
      /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
      /home/pzelasko/miniconda3/envs/lhotse/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
      cmake/torch.cmake:13 (find_package)
      CMakeLists.txt:41 (include)
    

    I am able to build it by doing this:

    <inside sources root>
    mkdir build
    cd build
    cmake .. -DCUDNN_LIBRARY_PATH="/home/pzelasko/miniconda3/envs/lhotse/lib" -DCUDNN_INCLUDE_PATH="/home/pzelasko/miniconda3/envs/lhotse/include"
    

    But there seems to be no way to pass these values through env vars when doing pip install kaldifeat.

    opened by pzelasko 4
  • FbankOpts (and others) conversion to/from dict

    FbankOpts (and others) conversion to/from dict

    Would it be possible to add a conversion mechanism for the Opts classes to/from dict? It would make it much easier for me to integrate kaldifeat in Lhotse. I'm using dataclasses for configuration of all feature extractors, like this:

    https://github.com/lhotse-speech/lhotse/blob/f5bd2063291b7d789d4b8a2a50396d2ffa76eb2c/lhotse/features/fbank.py#L9-L31

    Example API could look like this:

    # Read a dict
    config = {"frame_shift": 0.012, "use_energy": True}
    extractor = kaldifeat.Fbank(kaldifeat.FbankOptions.from_dict(config))
    
    # Store to a dict
    config2 = extractor.opts.to_dict()
    # config2 = {... a list of all settings ...}
    
    opened by pzelasko 4
  •       cl: 命令行 error D8021 :无效的数值参数“/Wno-register”

    cl: 命令行 error D8021 :无效的数值参数“/Wno-register”

    c /Tpextensions/align.cpp /Fobuild\temp.win-amd64-3.9\Release\extensions/align.obj -std=c++11 -Wno-register -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char
          cl: 命令行 error D8021 :无效的数值参数“/Wno-register”
          error: command 'D:\\apps\\VisualStudio\\VC\\Tools\\MSVC\\14.33.31629\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
          [end of output]
    

    windows无法安装。

    opened by jinfagang 15
  • /lib64/libm.so.6: version `GLIBC_2.27' not found (required by python3.7/site-packages/libkaldifeat_core.so

    /lib64/libm.so.6: version `GLIBC_2.27' not found (required by python3.7/site-packages/libkaldifeat_core.so

    import kaldifeat Traceback (most recent call last): File "", line 1, in File "/home/cyn244124/espnet/espnet/tools/anaconda/envs/espnet/lib/python3.7/site-packages/kaldifeat/init.py", line 2, in from _kaldifeat import FbankOptions, FrameExtractionOptions, MelBanksOptions ImportError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /lib/python3.7/site-packages/libkaldifeat_core.so)

    opened by maxandchen 3
  • python3 setup.py install error

    python3 setup.py install error

    running install /root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( /root/anaconda3/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( running bdist_egg running egg_info writing kaldifeat.egg-info/PKG-INFO writing dependency_links to kaldifeat.egg-info/dependency_links.txt writing top-level names to kaldifeat.egg-info/top_level.txt reading manifest file 'kaldifeat.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'kaldifeat.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py copying kaldifeat/python/kaldifeat/torch_version.py -> build/lib.linux-x86_64-3.9/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-3.9/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /root/anaconda3/bin/python3 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

                cd build/temp.linux-x86_64-3.9
    
                cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/root/anaconda3/bin/python3 -Dkaldifeat_BUILD_TESTS=OFF  -DCMAKE_INSTALL_PREFIX=/root/kaldifeat/build/lib.linux-x86_64-3.9/kaldifeat  /root/kaldifeat
    
    
                make  _kaldifeat install
    

    -- C++ Standard version: 14 -- Downloading pybind11 -- pybind11 is downloaded to /root/kaldifeat/build/temp.linux-x86_64-3.9/_deps/pybind11-src -- pybind11 v2.9.2 -- Python executable: /root/anaconda3/bin/python3 -- TORCH_DIR: /root/anaconda3/lib/python3.9/site-packages/torch -- Caffe2: CUDA detected: 11.3 -- Caffe2: CUDA nvcc is: /usr/local/cuda-11.3/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.3 -- Caffe2: Header version is: 11.3 -- Found cuDNN: v8.2.1 (include: /usr/local/cuda-11.3/include, library: /usr/local/cuda-11.3/lib64/libcudnn.so) -- /usr/local/cuda-11.3/lib64/libnvrtc.so shorthash is 8aa72235 -- Autodetected CUDA architecture(s): 6.0 6.0 6.0 -- Added CUDA NVCC flags for: -gencode;arch=compute_60,code=sm_60 CMake Warning at /root/anaconda3/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message): static library kineto_LIBRARY-NOTFOUND not found. Call Stack (most recent call first): /root/anaconda3/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found) cmake/torch.cmake:14 (find_package) CMakeLists.txt:55 (include)

    -- PyTorch version: 1.12.1 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -- CMAKE_INSTALL_PREFIX: /root/kaldifeat/build/lib.linux-x86_64-3.9/kaldifeat -- All headers: /root/kaldifeat/kaldifeat/csrc/feature-common-inl.h;/root/kaldifeat/kaldifeat/csrc/feature-common.h;/root/kaldifeat/kaldifeat/csrc/feature-fbank.h;/root/kaldifeat/kaldifeat/csrc/feature-functions.h;/root/kaldifeat/kaldifeat/csrc/feature-mfcc.h;/root/kaldifeat/kaldifeat/csrc/feature-plp.h;/root/kaldifeat/kaldifeat/csrc/feature-spectrogram.h;/root/kaldifeat/kaldifeat/csrc/feature-window.h;/root/kaldifeat/kaldifeat/csrc/log.h;/root/kaldifeat/kaldifeat/csrc/matrix-functions.h;/root/kaldifeat/kaldifeat/csrc/mel-computations.h;/root/kaldifeat/kaldifeat/csrc/online-feature-itf.h;/root/kaldifeat/kaldifeat/csrc/online-feature.h;/root/kaldifeat/kaldifeat/csrc/pitch-functions.h -- Configuring done -- Generating done -- Build files have been written to: /root/kaldifeat/build/temp.linux-x86_64-3.9 [ 50%] Built target kaldifeat_core [100%] Built target _kaldifeat [ 45%] Built target kaldifeat_core [ 50%] Linking CXX executable ../../bin/test_kaldifeat /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to [email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to[email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to [email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to[email protected]' /usr/bin/ld: /root/anaconda3/lib/python3.9/site-packages/torch/lib/libtorch_cuda_cu.so: undefined reference to `[email protected]' collect2: error: ld returned 1 exit status make[2]: *** [kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/build.make:98: bin/test_kaldifeat] Error 1 make[1]: *** [CMakeFiles/Makefile2:172: kaldifeat/csrc/CMakeFiles/test_kaldifeat.dir/all] Error 2 make: *** [Makefile:130: all] Error 2 Traceback (most recent call last): File "/root/kaldifeat/setup.py", line 32, in setuptools.setup( File "/root/anaconda3/lib/python3.9/site-packages/setuptools/init.py", line 87, in setup return distutils.core.setup(**attrs) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup return run_commands(dist) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands dist.run_commands() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands self.run_command(cmd) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 74, in run self.do_egg_install() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 123, in do_egg_install self.run_command('bdist_egg') File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 165, in run cmd = self.call_command('install_lib', warn_dir=0) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command self.run_command(cmdname) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install_lib.py", line 107, in build self.run_command('build_ext') File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command super().run_command(command) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command cmd_obj.run() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run _build_ext.run(self) File "/root/anaconda3/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run self.build_extensions() File "/root/anaconda3/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions self._build_extensions_serial() File "/root/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial self.build_extension(ext) File "/root/kaldifeat/cmake/cmake_extension.py", line 122, in build_extension raise Exception( Exception: Build kaldifeat failed. Please check the error message. You can ask for help by creating an issue on GitHub.

    Click: https://github.com/csukuangfj/kaldifeat/issues/new

    opened by farmer21cn 13
  • fail to install kaldifeat in conda

    fail to install kaldifeat in conda

    pip install kaldifeat Looking in indexes: http://mirrors.cloud.tencent.com/pypi/simple Collecting kaldifeat Using cached http://mirrors.cloud.tencent.com/pypi/packages/2d/7b/0a5c9254de6a62cdca7bea829bd8d5d646e18508d34f9214f048ee003fed/kaldifeat-1.17.tar.gz (477 kB) Preparing metadata (setup.py) ... done Building wheels for collected packages: kaldifeat Building wheel for kaldifeat (setup.py) ... -

    error error: subprocess-exited-with-error

    × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [184 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-38 creating build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-cpython-38/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

                      cd build/temp.linux-x86_64-cpython-38
      
                      cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783
      
      
                      make  _kaldifeat
      
      -- The C compiler identification is GNU 11.3.0
      -- The CXX compiler identification is GNU 11.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- C++ Standard version: 14
      -- Downloading pybind11
      -- pybind11 is downloaded to /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/_deps/pybind11-src
      -- pybind11 v2.9.2
      -- Found PythonInterp: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 (found version "3.8.13")
      -- Found PythonLibs: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libpython3.8.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Python executable: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8
      -- Looking for pthread.h
      -- Looking for pthread.h - found
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - found
      -- Found Threads: TRUE
      -- Found CUDA: /cfs/user/burkliu/dev/anaconda3/envs/k2_m (found version "11.7")
      -- The CUDA compiler identification is NVIDIA 11.7.64
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- Caffe2: CUDA detected: 11.7
      -- Caffe2: CUDA nvcc is: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc
      -- Caffe2: CUDA toolkit directory: /cfs/user/burkliu/dev/anaconda3/envs/k2_m
      CMake Error at /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:88 (message):
        Caffe2: Couldn't determine version from header: Change Dir:
        /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp
      
      
      
      
        Run Build Command(s):/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
        Makefile cmTC_e257c/fast &&
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
        CMakeFiles/cmTC_e257c.dir/build.make CMakeFiles/cmTC_e257c.dir/build
      
        make[1]: Entering directory
        '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
      
      
        Building CXX object CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
        -I/cfs/user/burkliu/dev/anaconda3/envs/k2_m/include -std=c++14 -o
        CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o -c
        /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/detect_cuda_version.cc
      
      
        Linking CXX executable cmTC_e257c
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cmake -E cmake_link_script
        CMakeFiles/cmTC_e257c.dir/link.txt --verbose=1
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
        CMakeFiles/cmTC_e257c.dir/detect_cuda_version.cc.o -o cmTC_e257c
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so
      
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
        reference to `memcpy@GLIBC_2.14'
      
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
        reference to `clock_gettime@GLIBC_2.17'
      
        collect2: error: ld returned 1 exit status
      
        make[1]: *** [CMakeFiles/cmTC_e257c.dir/build.make:100: cmTC_e257c] Error 1
      
        make[1]: Leaving directory
        '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
      
      
        make: *** [Makefile:127: cmTC_e257c/fast] Error 2
      
      
      
      Call Stack (most recent call first):
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        cmake/torch.cmake:13 (find_package)
        CMakeLists.txt:48 (include)
      
      
      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeOutput.log".
      See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeError.log".
      make: *** No rule to make target '_kaldifeat'.  Stop.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/setup.py", line 32, in <module>
          setuptools.setup(
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
          super().run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
          self.build_extensions()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/cmake/cmake_extension.py", line 110, in build_extension
          raise Exception(
      Exception:
      Build kaldifeat failed. Please check the error message.
      You can ask for help by creating an issue on GitHub.
      
      Click:
          https://github.com/csukuangfj/kaldifeat/issues/new
      
      [end of output]
    

    note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for kaldifeat Running setup.py clean for kaldifeat Failed to build kaldifeat Installing collected packages: kaldifeat Running setup.py install for kaldifeat ... error error: subprocess-exited-with-error

    × Running setup.py install for kaldifeat did not run successfully. │ exit code: 1 ╰─> [188 lines of output] running install /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.linux-x86_64-cpython-38 creating build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/offline_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/online_feature.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/fbank.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/mfcc.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/plp.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/init.py -> build/lib.linux-x86_64-cpython-38/kaldifeat copying kaldifeat/python/kaldifeat/spectrogram.py -> build/lib.linux-x86_64-cpython-38/kaldifeat running build_ext Setting PYTHON_EXECUTABLE to /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 For fast compilation, run: export KALDIFEAT_MAKE_ARGS="-j"; python setup.py install build command is:

                      cd build/temp.linux-x86_64-cpython-38
      
                      cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783
      
      
                      make  _kaldifeat
      
      -- The C compiler identification is GNU 11.3.0
      -- The CXX compiler identification is GNU 11.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- C++ Standard version: 14
      -- Downloading pybind11
      -- pybind11 is downloaded to /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/_deps/pybind11-src
      -- pybind11 v2.9.2
      -- Found PythonInterp: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8 (found version "3.8.13")
      -- Found PythonLibs: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libpython3.8.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Python executable: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/python3.8
      -- Looking for pthread.h
      -- Looking for pthread.h - found
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - found
      -- Found Threads: TRUE
      -- Found CUDA: /cfs/user/burkliu/dev/anaconda3/envs/k2_m (found version "11.7")
      -- The CUDA compiler identification is NVIDIA 11.7.64
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- Caffe2: CUDA detected: 11.7
      -- Caffe2: CUDA nvcc is: /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/nvcc
      -- Caffe2: CUDA toolkit directory: /cfs/user/burkliu/dev/anaconda3/envs/k2_m
      CMake Error at /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:88 (message):
        Caffe2: Couldn't determine version from header: Change Dir:
        /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp
      
      
      
      
        Run Build Command(s):/cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
        Makefile cmTC_a8777/fast &&
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/make -f
        CMakeFiles/cmTC_a8777.dir/build.make CMakeFiles/cmTC_a8777.dir/build
      
        make[1]: Entering directory
        '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
      
      
        Building CXX object CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
        -I/cfs/user/burkliu/dev/anaconda3/envs/k2_m/include -std=c++14 -o
        CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o -c
        /tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/detect_cuda_version.cc
      
      
        Linking CXX executable cmTC_a8777
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/cmake -E cmake_link_script
        CMakeFiles/cmTC_a8777.dir/link.txt --verbose=1
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/c++
        CMakeFiles/cmTC_a8777.dir/detect_cuda_version.cc.o -o cmTC_a8777
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so
      
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
        reference to `memcpy@GLIBC_2.14'
      
      
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/libcudart.so: undefined
        reference to `clock_gettime@GLIBC_2.17'
      
        collect2: error: ld returned 1 exit status
      
        make[1]: *** [CMakeFiles/cmTC_a8777.dir/build.make:100: cmTC_a8777] Error 1
      
        make[1]: Leaving directory
        '/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeTmp'
      
      
        make: *** [Makefile:127: cmTC_a8777/fast] Error 2
      
      
      
      Call Stack (most recent call first):
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
        /cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        cmake/torch.cmake:13 (find_package)
        CMakeLists.txt:48 (include)
      
      
      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeOutput.log".
      See also "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeError.log".
      make: *** No rule to make target '_kaldifeat'.  Stop.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/setup.py", line 32, in <module>
          setuptools.setup(
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/install.py", line 698, in run
          self.run_command('build')
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
          super().run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
          self.build_extensions()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/cfs/user/burkliu/dev/anaconda3/envs/k2_m/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-wu0b8kwx/kaldifeat_ce98eebe94604b818d6f906974358783/cmake/cmake_extension.py", line 110, in build_extension
          raise Exception(
      Exception:
      Build kaldifeat failed. Please check the error message.
      You can ask for help by creating an issue on GitHub.
      
      Click:
          https://github.com/csukuangfj/kaldifeat/issues/new
      
      [end of output]
    

    note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

    × Encountered error while trying to install package. ╰─> kaldifeat

    note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.

    opened by boji123 25
  • potential memory leak

    potential memory leak

    Hi~

    I use valgrind to test the below line:

    import kaldifeat
    

    The log is here: https://1drv.ms/t/s!AhtoTbISXXlSgSLtqDWjo8RP_ouA?e=Sn1cd0

    Log summary:

    ==20510== LEAK SUMMARY:
    ==20510==    definitely lost: 0 bytes in 0 blocks
    ==20510==    indirectly lost: 0 bytes in 0 blocks
    ==20510==      possibly lost: 271,997 bytes in 129 blocks
    ==20510==    still reachable: 5,881,625 bytes in 22,065 blocks
    ==20510==                       of which reachable via heuristic:
    ==20510==                         stdstring          : 521,021 bytes in 6,954 blocks
    ==20510==         suppressed: 58,517 bytes in 5 blocks
    ==20510==
    ==20510== For lists of detected and suppressed errors, rerun with: -s
    ==20510== ERROR SUMMARY: 17745 errors from 434 contexts (suppressed: 14 from 2)
    

    Reproduce

    valgrind --leak-check=full --show-reachable=yes --trace-children=yes --track-origins=yes --trace-children=yes --log-file='log.txt' python3 test.py
    

    test.py

    import kaldifeat
    

    Could you help check if these are potential issues? Thanks!

    opened by Slyne 3
Releases(v1.22)
Owner
Fangjun Kuang
Was vorbei ist, ist vorbei.
Fangjun Kuang
PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations

PyTorch Sparse This package consists of a small extension library of optimized sparse matrix operations with autograd support. This package currently

Matthias Fey 757 Jan 4, 2023
PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

Cong Cai 12 Dec 19, 2021
Tez is a super-simple and lightweight Trainer for PyTorch. It also comes with many utils that you can use to tackle over 90% of deep learning projects in PyTorch.

Tez: a simple pytorch trainer NOTE: Currently, we are not accepting any pull requests! All PRs will be closed. If you want a feature or something does

abhishek thakur 1.1k Jan 4, 2023
A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-precision, and PyTorch extensions.

A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-precision, and PyTorch extensions.

Fidelity Investments 56 Sep 13, 2022
A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

null 878 Dec 30, 2022
null 270 Dec 24, 2022
Unofficial PyTorch implementation of DeepMind's Perceiver IO with PyTorch Lightning scripts for distributed training

Unofficial PyTorch implementation of DeepMind's Perceiver IO with PyTorch Lightning scripts for distributed training

Martin Krasser 251 Dec 25, 2022
The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

News March 3: v0.9.97 has various bug fixes and improvements: Bug fixes for NTXentLoss Efficiency improvement for AccuracyCalculator, by using torch i

Kevin Musgrave 5k Jan 2, 2023
A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch

Torchmeta A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch. Torchmeta contains popular meta-learning bench

Tristan Deleu 1.7k Jan 6, 2023
PyTorch extensions for fast R&D prototyping and Kaggle farming

Pytorch-toolbelt A pytorch-toolbelt is a Python library with a set of bells and whistles for PyTorch for fast R&D prototyping and Kaggle farming: What

Eugene Khvedchenya 1.3k Jan 5, 2023
Fast, general, and tested differentiable structured prediction in PyTorch

Torch-Struct: Structured Prediction Library A library of tested, GPU implementations of core structured prediction algorithms for deep learning applic

HNLP 1.1k Jan 7, 2023
A simplified framework and utilities for PyTorch

Here is Poutyne. Poutyne is a simplified framework for PyTorch and handles much of the boilerplating code needed to train neural networks. Use Poutyne

GRAAL/GRAIL 534 Dec 17, 2022
A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

?? Accelerate was created for PyTorch users who like to write the training loop of PyTorch models but are reluctant to write and maintain the boilerplate code needed to use multi-GPUs/TPU/fp16.

Hugging Face 3.5k Jan 8, 2023
A very simple and small path tracer written in pytorch meant to be run on the GPU

MentisOculi Pytorch Path Tracer A very simple and small path tracer written in pytorch meant to be run on the GPU Why use pytorch and not some other c

Matthew B. Mirman 222 Dec 1, 2022
A pure Python implementation of Compact Bilinear Pooling and Count Sketch for PyTorch.

Compact Bilinear Pooling for PyTorch. This repository has a pure Python implementation of Compact Bilinear Pooling and Count Sketch for PyTorch. This

Grégoire Payen de La Garanderie 234 Dec 7, 2022
PyTorch Lightning Optical Flow models, scripts, and pretrained weights.

PyTorch Lightning Optical Flow models, scripts, and pretrained weights.

Henrique Morimitsu 105 Dec 16, 2022
PyTorch implementations of normalizing flow and its variants.

PyTorch implementations of normalizing flow and its variants.

Tatsuya Yatagawa 55 Dec 1, 2022
Pretrained ConvNets for pytorch: NASNet, ResNeXt, ResNet, InceptionV4, InceptionResnetV2, Xception, DPN, etc.

Pretrained models for Pytorch (Work in progress) The goal of this repo is: to help to reproduce research papers results (transfer learning setups for

Remi 8.7k Dec 31, 2022
Model summary in PyTorch similar to `model.summary()` in Keras

Keras style model.summary() in PyTorch Keras has a neat API to view the visualization of the model which is very helpful while debugging your network.

Shubham Chandel 3.7k Dec 29, 2022