As an FYI, pyfasttext is no longer maintained and the author recommends using the official Python bindings.

I'm not sure if this is worth updating with a PR, but wanted to at least raise a potential issue.

You are using Fasttext for language classification. While examining the README example, I see the input text is given in corresponding unicode characters. I'm being curious that, how it differentiate itself from a logic that uses character parser to determine language?

Hi,

Can you change this dependency to something that uses another binding of fasttext? It looks like pyfasttext is using old version of fasttext (pre 0.8.2) and so mac os users get the symbol not found error: https://github.com/vrasneur/pyfasttext/issues/46. So I cannot use this on my mac..

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-7VI_fe/pyfasttext/setup.py", line 2, in <module>
    from Cython.Build import cythonize
ImportError: No module named Cython.Build

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-7VI_fe/pyfasttext/

Found this issue while passing input as unicode string.

Example

wtl.predict_lang(u"nike running shoes string")
['CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT', 'CANT_PREDICT']

This can be resolved by wrapping all text including unicode text in list.

>>> wtl.predict_lang([u"nike running shoes"])
[u'en']

• Unicode text handled differently in Python 2 and 3 and this fixes the unicode input issue.
• Added test for unicode
• bumping up the version
• Let travis build in py3 also

Hi, as I tried to pip install pyfasttext I always fail, but I did successfully installed the new version of fasttext as mentioned in a previous issue, the error message is as follows, could anyone please help on this?

`pip install pyfasttext==0.4.3 Collecting pyfasttext==0.4.3 Using cached https://files.pythonhosted.org/packages/6a/64/7a06c6bf18a34f6530d1bb07cb98ab56f6f21edf0129c9ea8baf0d666f4a/pyfasttext-0.4.3.tar.gz Requirement already satisfied: cysignals in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pyfasttext==0.4.3) (1.10.2) Requirement already satisfied: future in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pyfasttext==0.4.3) (0.17.1) Requirement already satisfied: numpy in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pyfasttext==0.4.3) (1.16.4) Requirement already satisfied: Cython>=0.28 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from cysignals->pyfasttext==0.4.3) (0.29.13) Building wheels for collected packages: pyfasttext Building wheel for pyfasttext (setup.py) ... error ERROR: Command errored out with exit status 1: command: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"'; file='"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-wheel-prgzb0bt --python-tag cp37 cwd: /private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/ Complete output (450 lines): running bdist_wheel running build running build_ext building 'pyfasttext' extension creating build creating build/temp.macosx-10.9-x86_64-3.7 creating build/temp.macosx-10.9-x86_64-3.7/src creating build/temp.macosx-10.9-x86_64-3.7/src/fastText creating build/temp.macosx-10.9-x86_64-3.7/src/fastText/src gcc -DNDEBUG -g -fwrapv -O3 -Wall -iquote . -include src/custom_exit.h -Isrc -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cysignals -I. -Isrc/variant/include -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/pyfasttext.cpp -o build/temp.macosx-10.9-x86_64-3.7/src/pyfasttext.o -std=c++0x -Wno-sign-compare In file included from src/pyfasttext.cpp:680: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings] #warning "Using deprecated NumPy API, disable it with "
^ src/pyfasttext.cpp:6523:4: warning: array index 1 is past the end of the array (which contains 1 element) [-Warray-bounds] (pyx_v_shape[1]) = ((npy_intp)pyx_v_dim); ^ ~ src/pyfasttext.cpp:6331:3: note: array 'pyx_v_shape' declared here npy_intp pyx_v_shape[1]; ^ 2 warnings generated. gcc -DNDEBUG -g -fwrapv -O3 -Wall -iquote . -include src/custom_exit.h -Isrc -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cysignals -I. -Isrc/variant/include -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/fasttext_access.cpp -o build/temp.macosx-10.9-x86_64-3.7/src/fasttext_access.o -std=c++0x -Wno-sign-compare src/fasttext_access.cpp:33:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, args); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:33:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, args); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:34:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, dict); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:34:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, dict); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:35:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, input_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:35:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, input_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:36:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, output_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:36:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, output_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:37:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qinput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:37:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qinput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:38:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qoutput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:38:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qoutput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:39:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, model_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:39:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, model_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:40:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::atomic<int64_t>, tokenCount); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:40:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::atomic<int64_t>, tokenCount); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:41:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, bool, quant_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:41:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, bool, quant_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:42:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, int32_t, version); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:42:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, int32_t, version); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:43:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(FastText, bool, checkModel, std::istream&); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:43:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(FastText, bool, checkModel, std::istream&); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:45:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, std::vector<int32_t>, word2int_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:45:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, std::vector<int32_t>, word2int_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:46:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, std::vector, words_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:46:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, std::vector, words_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:47:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:47:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:48:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nwords_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:48:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nwords_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:49:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nlabels_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:49:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nlabels_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:50:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int64_t, ntokens_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:50:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int64_t, ntokens_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:51:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int64_t, pruneidx_size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:51:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int64_t, pruneidx_size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:53:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, pruneidx_type, pruneidx_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:53:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, pruneidx_type, pruneidx_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:54:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:54:1: note: did you mean struct here? ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:54:48: error: 'find' is a private member of 'fasttext::Dictionary' ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~ src/private_access.h:59:45: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ~~~~~~~^~~~~~ src/fastText/src/dictionary.h:41:13: note: declared private here int32_t find(const std::string&) const; ^ src/fasttext_access.cpp:55:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_CONST_METHOD_ACCESS(Dictionary, void, pushHash, std::vector<int32_t>&, int32_t); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:55:1: note: did you mean struct here? ALLOW_CONST_METHOD_ACCESS(Dictionary, void, pushHash, std::vector<int32_t>&, int32_t); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:56:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(Dictionary, void, initTableDiscard, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:56:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(Dictionary, void, initTableDiscard, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:57:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(Dictionary, void, initNgrams, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:57:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(Dictionary, void, initNgrams, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ 23 warnings and 1 error generated. error: command 'gcc' failed with exit status 1

ERROR: Failed building wheel for pyfasttext Running setup.py clean for pyfasttext Failed to build pyfasttext Installing collected packages: pyfasttext Running setup.py install for pyfasttext ... error ERROR: Command errored out with exit status 1: command: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"'; file='"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-record-xjlblv7z/install-record.txt --single-version-externally-managed --compile cwd: /private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/ Complete output (450 lines): running install running build running build_ext building 'pyfasttext' extension creating build creating build/temp.macosx-10.9-x86_64-3.7 creating build/temp.macosx-10.9-x86_64-3.7/src creating build/temp.macosx-10.9-x86_64-3.7/src/fastText creating build/temp.macosx-10.9-x86_64-3.7/src/fastText/src gcc -DNDEBUG -g -fwrapv -O3 -Wall -iquote . -include src/custom_exit.h -Isrc -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cysignals -I. -Isrc/variant/include -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/pyfasttext.cpp -o build/temp.macosx-10.9-x86_64-3.7/src/pyfasttext.o -std=c++0x -Wno-sign-compare In file included from src/pyfasttext.cpp:680: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12: In file included from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings] #warning "Using deprecated NumPy API, disable it with "
^ src/pyfasttext.cpp:6523:4: warning: array index 1 is past the end of the array (which contains 1 element) [-Warray-bounds] (pyx_v_shape[1]) = ((npy_intp)pyx_v_dim); ^ ~ src/pyfasttext.cpp:6331:3: note: array 'pyx_v_shape' declared here npy_intp pyx_v_shape[1]; ^ 2 warnings generated. gcc -DNDEBUG -g -fwrapv -O3 -Wall -iquote . -include src/custom_exit.h -Isrc -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cysignals -I. -Isrc/variant/include -I/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/fasttext_access.cpp -o build/temp.macosx-10.9-x86_64-3.7/src/fasttext_access.o -std=c++0x -Wno-sign-compare src/fasttext_access.cpp:33:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, args); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:33:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, args); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:34:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, dict); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:34:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, dict); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:35:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, input_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:35:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, input_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:36:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, output_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:36:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, output_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:37:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qinput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:37:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qinput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:38:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qoutput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:38:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, qoutput_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:39:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, model_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:39:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::shared_ptr, model_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:40:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, std::atomic<int64_t>, tokenCount); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:40:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, std::atomic<int64_t>, tokenCount); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:41:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, bool, quant_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:41:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, bool, quant_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:42:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(FastText, int32_t, version); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:42:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(FastText, int32_t, version); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:43:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(FastText, bool, checkModel, std::istream&); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:43:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(FastText, bool, checkModel, std::istream&); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:45:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, std::vector<int32_t>, word2int_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:45:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, std::vector<int32_t>, word2int_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:46:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, std::vector, words_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:46:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, std::vector, words_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:47:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:47:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:48:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nwords_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:48:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nwords_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:49:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nlabels_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:49:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int32_t, nlabels_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:50:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int64_t, ntokens_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:50:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int64_t, ntokens_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:51:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, int64_t, pruneidx_size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:51:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, int64_t, pruneidx_size_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:53:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_MEMBER_ACCESS(Dictionary, pruneidx_type, pruneidx_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:53:1: note: did you mean struct here? ALLOW_MEMBER_ACCESS(Dictionary, pruneidx_type, pruneidx_); ^ src/private_access.h:51:12: note: expanded from macro 'ALLOW_MEMBER_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:54:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:54:1: note: did you mean struct here? ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:54:48: error: 'find' is a private member of 'fasttext::Dictionary' ALLOW_CONST_METHOD_ACCESS(Dictionary, int32_t, find, const std::string&); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~ src/private_access.h:59:45: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ~~~~~~~^~~~~~ src/fastText/src/dictionary.h:41:13: note: declared private here int32_t find(const std::string&) const; ^ src/fasttext_access.cpp:55:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_CONST_METHOD_ACCESS(Dictionary, void, pushHash, std::vector<int32_t>&, int32_t); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:55:1: note: did you mean struct here? ALLOW_CONST_METHOD_ACCESS(Dictionary, void, pushHash, std::vector<int32_t>&, int32_t); ^ src/private_access.h:59:12: note: expanded from macro 'ALLOW_CONST_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:56:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(Dictionary, void, initTableDiscard, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:56:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(Dictionary, void, initTableDiscard, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:57:1: warning: class template 'rob' was previously declared as a struct template; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags] ALLOW_METHOD_ACCESS(Dictionary, void, initNgrams, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ src/fasttext_access.cpp:31:1: note: previous use is here ACCESS_INIT; ^ src/private_access.h:39:12: note: expanded from macro 'ACCESS_INIT' struct rob : result {
^ src/fasttext_access.cpp:57:1: note: did you mean struct here? ALLOW_METHOD_ACCESS(Dictionary, void, initNgrams, ); ^ src/private_access.h:55:12: note: expanded from macro 'ALLOW_METHOD_ACCESS' template class rob<Only_##MEMBER, &CLASS::MEMBER> ^ 23 warnings and 1 error generated. error: command 'gcc' failed with exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"'; file='"'"'/private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-install-b72ncbjt/pyfasttext/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/hj/9jxvqvrs2xq4h9rj1p288lf40000gn/T/pip-record-xjlblv7z/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.`

When attempting to install:

Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\ser\AppData\Local\Temp\pip-req-build-jkgwiajx\setup.py", line 8, in <module> readme = f.read() File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 661: character maps to <undefined>

Fixed by changing the following lines from setup.py:

with open('README.md'') as f:
    readme = f.read()
with open('LICENSE') as f:
    license = f.read()

To the following:

with open('README.md', encoding='utf-8') as f:
    readme = f.read()
with open('LICENSE', encoding='utf-8') as f:
    license = f.read()

I'm using version 1.0.2 - fails when I use the single word "hallo" - here is a quick repro:

$ python
Python 3.7.1 (default, Dec 14 2018, 19:28:38)
>>> from whatlang import WhatLang
>>> wl = WhatLang()
>>> wl.predict_lang("hallo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "........../site-packages/whatlang/predict_lang.py", line 32, in predict_lang
    if len(pred[0]) == 0:
IndexError: list index out of range

Think the results below speak for themselves. It would be helpful to know what sort of normalisation is expected since it seems WTL does not do any. Mostly seems to be newlines influencing the results.

from whatthelang import WhatTheLang as WTL
p = WTL()
s = 'Chapter 14: Person of Interest\n Chapter 16: The Debt\n'
p.predict_lang(s)  # fr
s = 'to actual persons, living or dead, \n business establishments, events,\n'
p.predict_lang(s)  # es
s = 'Chapter 11: Freedom\n Chapter 12: Everyday Life for Five Years\n'
p.predict_lang(s)  # fr
s = 'Prologue\n They were two beautiful people. Both strong and healthy, exactly what she was looking for.\n'
p.predict_lang(s)  # fr
s = '###\n By Heather Graham\n'
p.predict_lang(s)  # de
s = 'Raymond Stocker – Owner/operator of Nicoll’s Island amusement park\n Jasmine Stocker – wife of Raymond\n'
p.predict_lang(s)  # de
s = 'Prologue\n Flirtation lasts the brief flutter of a butterfly’s wing.\n'
p.predict_lang(s)  # fr
s = 'Any resemblance to places or actual persons,\n living or dead is entirely coincidental.\n'
p.predict_lang(s)  # es
s = 'Dream of the Fir Bolg\n Eochaidh mac Eirc, Ard Ri, High King of Ireland and leader of the Fir Bolg, stood on the cliff edge and looked out to sea. The night was clear, the water was calm, and the moon and stars observed the scene like many bright eyes.\n'
p.predict_lang(s)  # af