:speech_balloon: SpeechPy - A Library for Speech Processing and Recognition: http://speechpy.readthedocs.io/en/latest/

Currently, passing a signal size equal to or less than the frame length throws an exception. This changes it so that when a signal of the same length is given, it gives a result like normal, and when a signall smaller than that is given, it outputs an empty array of the correct dimensions.

With change:

import numpy as np
from speechpy.main import mfcc
mfcc(np.ones((999)), 1000, 1, 1, 2)  # array([], shape=(0, 2), dtype=float64)
mfcc(np.ones((1000)), 1000, 1, 1, 2)  # array([[ 6.23832463,  0.        ]])
mfcc(np.ones((1999)), 1000, 1, 1, 2)  # array([[ 6.23832463,  0.        ]])
mfcc(np.ones((2000)), 1000, 1, 1, 2)  # array([[ 6.23832463,  0.        ], [ 6.23832463,  0.        ]])

Before:

import numpy as np
from speechpy.main import mfcc
mfcc(np.ones((999)), 1000, 1, 1, 2)  # UnboundLocalError: local variable 'signal' referenced before assignment
mfcc(np.ones((1000)), 1000, 1, 1, 2)  # UnboundLocalError: local variable 'signal' referenced before assignment
mfcc(np.ones((1999)), 1000, 1, 1, 2)  # array([[ 6.23832463,  0.        ]])
mfcc(np.ones((2000)), 1000, 1, 1, 2)  # array([[ 6.23832463,  0.        ], [ 6.23832463,  0.        ]])

Hi tryinig to use this in windows 7 64 bits 3.5 python by pip or by git no errors during the pip install

when i import pyspeech got this one :

import speechpy

ImportError Traceback (most recent call last) in () ----> 1 import speechpy

c:\anaconda3\lib\site-packages\speechpy_init_.py in () ----> 1 from main import * 2 from processing import *

ImportError: No module named 'main'

In encountered the following warning during the variance normalization of the speech features:

RuntimeWarning: divide by zero encountered in true_divide

cmvnw This is probably not the desired behavior, I don't know what the best solution in this case is though.

when you use the derivative_extraction function, you are likely to concatenate it with your features, and your features' type might be chosen carefully in a manner sensitive to memory usage.

for example in my project i have a large dataset (which is normal for the use case of this library) which i use float32 as the datatype of it's values, since it's accuracy is enough and it reduces the memory footprint by half compared to float64, but when i used the derivative_extraction it calculated it's values in float64 and when i concatenated the derivatives and the original features all the values where converted to float64 and the system's memory usage was doubled, which wasn't immediately apparent.

this is not that critical and it could be fixed by converting types from outside the library call, but why bother with the inconvenience when changing the datatype of the derivatives array from the incoming feature's datatype is not needed.

This pulls in #11 again, but disabling the caching in Python 2 rather than trying to use a backport of it. The reason is that in some installations, despite being installed, Python 2 will fail on the backports.functools_lru_cache import. This fixes #15.

@astorfi Let me know if you still see the error with this. Thanks

• Add function caching to speed up computation of functions called with the same parameters
• Remove use of np.round since it's slower for primitives

From testing in my use case, I get the following speed increases relative to the given function:

stack_frames()

• Removing np.round: 20%
• Adding create_indices lru: 60%

mfcc()

• Adding lru to filterbank: 310%

I have installed SpeechPy as part of my review of the package. Installing the package, I found a minor issue with the version number: I explicitly checked out the '2.3' release for installation whereas the install script output refers to version 2.2:

$ python setup.py develop
running develop
running egg_info
creating speechpy.egg-info
writing speechpy.egg-info/PKG-INFO
writing dependency_links to speechpy.egg-info/dependency_links.txt
writing requirements to speechpy.egg-info/requires.txt
writing top-level names to speechpy.egg-info/top_level.txt
writing manifest file 'speechpy.egg-info/SOURCES.txt'
reading manifest file 'speechpy.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'speechpy.egg-info/SOURCES.txt'
running build_ext
Creating /home/tha/.conda/envs/joss-review-tmp/lib/python3.6/site-packages/speechpy.egg-link (link to .)
Adding speechpy 2.2 to easy-install.pth file

Installed /home/tha/other-repo/speechpy
Processing dependencies for speechpy==2.2
Searching for numpy==1.14.3
Best match: numpy 1.14.3
Adding numpy 1.14.3 to easy-install.pth file

Using /home/tha/.conda/envs/joss-review-tmp/lib/python3.6/site-packages
Searching for scipy==1.1.0
Best match: scipy 1.1.0
Adding scipy 1.1.0 to easy-install.pth file

Using /home/tha/.conda/envs/joss-review-tmp/lib/python3.6/site-packages
Finished processing dependencies for speechpy==2.2

I suspect this is just some configuration text string that was not properly updated for the 2.3 release?

Im trying to extract mfcc features from audio of a video file.

I tried FFMPEG:

def extractAudioFromVideo(video, audio_out="out.wav"):
	cmd="ffmpeg -i {} -acodec pcm_s16le -ac 1 -ar 16000 {}".format(video, audio_out)
	os.system(cmd)
	return audio_out

def extractAudioMFCC(file_name="out.wav"):
	fs, signal = wav.read(file_name)
	signal = signal[:,0]

	############# Extract MFCC features #############
	mfcc = speechpy.feature.mfcc(signal, sampling_frequency=fs, frame_length=0.020, frame_stride=0.01,num_filters=40, fft_length=512, low_frequency=0, high_frequency=None)
	mfcc_cmvn = speechpy.processing.cmvnw(mfcc,win_size=301,variance_normalization=True)
	print('mfcc(mean + variance normalized) feature shape=', mfcc_cmvn.shape)


extractAudioMFCC("test.mp4", audio_out="out.wav")
extractAudioMFCC("out.wav")

The error I get:

Traceback (most recent call last): File "TWK.py", line 99, in extractAudioMFCC() File "TWK.py", line 22, in extractAudioMFCC signal = signal[:,0] IndexError: too many indices for array

Am I using the wrong wav format?

As per issue #2. The citation text content is from https://zenodo.org/record/840395/export/hx via https://zenodo.org/badge/latestdoi/87262342, which is the link of the button you mentioned in your comment on issue #2.

Respected Sir, Greetings of the day !!!

Sir first of all thank you so much for such amazing library you shared with us.

Sir I am using SpeechPy library for extracting the MFCC of audio signal.

Sir I have an audio signal of 16kHz, 32bit float PCM, Mono channel. I am using framelength 100ms with 50% overlapping.

I used below code for extraction of MFCC,

fs, signal = wav.read("b0.wav") signal = signal / abs(max(signal)) #Convert into double mfcc = speechpy.feature.mfcc (signal , sampling_frequency=fs, frame_length=0.1, frame_stride=0.05, num_filters=40, fft_length=2048, low_frequency=0, high_frequency=None)

Respected Sir, I got confusion because I used python_speech_features library also to extract mfcc and for verification of my result. But both are giving different result.

mfcc1 = python_speech_features.base.mfcc(signal, samplerate=fs, winlen=0.1, winstep=0.05, numcep=13, nfilt=26, nfft=2048, lowfreq=0, highfreq=None, preemph=0.97, ceplifter=22, appendEnergy=True)

I wanted to know where I am doing mistake.

My Questions Are:

1. Is the above code sequence is correct to extract mfcc using speechpy library ?

2. While using speechpy.feature.mfcc function, preemphasis operation is not performed? That is the reason both library are giving different result.

Should we have to perform seprately preemphasis using below code then we have to give the output of preemphasis to mfcc?

signal_preemphasized = speechpy.processing.preemphasis(signal, cof=0.98)

3. Why both library are giving different result ?

Its my humble request respected Sir Please response to my query. I am not getting clarification. What to use and which is correct.

I am sorry for my poor English.

Hello， I have a need for speech recognition now, and I have read many documents of this project, but I am still not sure whether this project can meet my need：

Now I have hundreds of thousands of wav audio files, which are only one to five seconds and divided into two categories, one is positive answer, the other is negative answer, but I do not have the text information corresponding to each wav file, now my demand is whether I can use this project to make intention judgment？

For example, if I input an audio data, then I can get the intention expressed by this audio, but there is no text corresponding to this audio

Any help will be greatly appreciated！

I wrote some code to compare the mel filterbanks in librosa, python speech feature and speechpy, and found two problems.

• 1. The initialization of the band edge of the Mel filterbanks may be wrong.
• 2. The calculation to convert frequency to fft bin number is wrong.

import matplotlib.pyplot as plt
import numpy as np
import librosa
import python_speech_features as psf
import speechpy

n_fft = 256        # The number of FFT components
n_filter = 20      # The number of filters in the filterbank
samplerate = 16000 # The samplerate of the signal
low_freq = 0       # The lowest band edge of the filters
high_freq = 8000   # The highest band edge of the filters

librosa_fbanks = librosa.filters.mel(
    sr=samplerate, n_fft=n_fft, n_mels=n_filter, fmin=low_freq, fmax=high_freq, norm=None)
print("Librosa mel fbanks shape:{}".format(librosa_fbanks.shape))

psf_fbanks = psf.base.get_filterbanks(
    nfilt=n_filter, nfft=n_fft, samplerate=samplerate, lowfreq=low_freq, highfreq=high_freq)
print("PSF mel fbanks shape:{}".format(psf_fbanks.shape))

coefficients = int(n_fft/2 + 1)
speechpy_fbanks = speechpy.feature.filterbanks(
    n_filter, coefficients, sampling_freq=samplerate, low_freq=low_freq, high_freq=high_freq)
print("Speechpy mel fbanks shape:{}".format(speechpy_fbanks.shape))

fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(10, 10))

x = np.array(list(range(speechpy_fbanks.shape[1])))
x = x * (samplerate / (n_fft + 1))

for i in range(librosa_fbanks.shape[0]):
    axes[0].plot(x, librosa_fbanks[i])
axes[0].set_title("librosa mel fbanks")

for i in range(psf_fbanks.shape[0]):
    axes[1].plot(x, psf_fbanks[i])
axes[1].set_title("psf mel fbanks")

for i in range(speechpy_fbanks.shape[0]):
    axes[2].plot(x, speechpy_fbanks[i])
axes[2].set_title("speechpy mel fbanks")

plt.show()

As shown in the figure, the parameter setting of low_freq of filterbanks of speechpy is invalid, and the filterbanks only covers half of the frequency band.

The first problem is caused by

low_freq = low_freq or 300.

When low_freq is 0, low_freq or 300 will return 300 instead of 0.

The second problem is a calculation error.

freq_index = (
    np.floor(
        (coefficients +
         1) *
        hertz /
        sampling_freq)).astype(int)

coefficients is equal to fftpoints/2 +1, which cannot cover the complete frequency band. We should use fftpoints instead of coefficients for calculation.

As shown in my code，I have fixed the above two bugs and hope to get your review and merge. Thank you!

Hi Amirsina,

First of all, great project!

I noticed in the mfcc the last frame_length of the signal buffer is always missing. When the number of stack frames is calculated (in the function stack_frames), the sample_buffer is decreased with the frame_length before it is divided in a number of stack frames.

See snippet: https://github.com/astorfi/speechpy/blob/4ece793cc52e36decd60dd89aca25233d773afe6/speechpy/processing.py#L103-L104

On a 1 second sample buffer this is hardly noticeable, but if we run the mfcc on smaller buffers this becomes significant.

If the calculation is done in this way:

    numframes = (int(math.ceil((length_signal
                                  - (frame_sample_length - frame_stride)) / frame_stride)))

The full sample buffer is used if frame_sample_length equals the frame_stride and adjusted correctly on differences between the frame_length and frame_stride.