I have some files that are causing a MagicException to be thrown with no details. This is on Ubuntu 12.04 LTS, with all packages up to date, and using the latest version of python-magic installed with pip.

Stack trace is below. I have stepped through the code, and it appears that it's dying somewhere in the ctypes error handler:

Traceback (most recent call last):
  File "./test_magic.py", line 35, in <module>
    mime_type = magic.from_file(filename, mime=True)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 121, in from_file
    return m.from_file(filename)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 78, in from_file
    return magic_file(self.cookie, filename)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 208, in magic_file
    return _magic_file(cookie, coerce_filename(filename))
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 169, in errorcheck_null
    raise MagicException(err)
magic.MagicException: None

I used the Python debugger, and in the errorcheck_null() method, the "result" parameter is "None", the "func" argument has a dict with "name==magic_file", and the "args" argument is a tuple with the numeric value 22135440 and the filename of the file I'm checking.

Since this is Ubuntu 12.04 LTS, the Python version is 2.7.3 (2.7.3-0ubuntu3.4) and the libmagic1 version is 5.09-2. I have been unable to reproduce this issue on Ubuntu 13.04 or Ubuntu 13.10, so I think it's something specific to either Python 2.7.3 or the version of libmagic1. Unfortunately I don't have the luxury of upgrading the Ubuntu version, since this is for a commercial production use.

What does the numeric value 22135440 mean? I realize that this may be a bug with either Python or libmagic1, but I'm not sure what the issue is to report it.

For what it's worth, if I use "/usr/bin/file" on the file, it works fine, and returns:

Composite Document File V2 Document, Little Endian, Os: Windows, Version 5.1, Code page: 936, Name of Creating Application: Microsoft Excel, Create Time/Date: Fri Sep 15 01:00:00 2006, Last Saved Time/Date: Mon Jan 17 19:59:34 2011, Security: 0

This is the same as https://github.com/ahupp/python-magic/issues/9 BUT I cannot encode to utf-8 as I do not know which encoding was used to create that filename. The file was created on some linux and is extracted on windows.

A test file is in this archive: https://github.com/pombredanne/python-magic/blob/issue_42/testdata/a.zip

When extracted on windows I can os.walk or os.listdir(u'some dir') (using a unicode string for the input dir) where this file has been extracted and open the file in Python alright. ie os.path.exists is True and the file can be opened.

Yet magic.from_file fails when passing the filename whether as a string or unicode

The point being that you cannot rely on my file system encoding to convert the file name, because it was created elsewhere, and the encoding is arbitrary, eventually unknown.

The best explanation of the problem is here IMHO: http://web.archive.org/web/20081219193532/http://boodebr.org/main/python/all-about-python-and-unicode#UNI_FILENAMES

NB: FWIW, I cannot get run the file command line to work on that path either... so it could be a problem with the file code itself, not your ctypes wrapper

Hi,

I use python-magic in eyeD3 and am quite happy with it, thanks!

On gentoo, at least, there is a package called sys-apps/file that provides libmagic.so and lays down a magic.py in site-packages. This collides with python-magic's file.. Which was first? I'm thinking of patching python-magic at install time to have a different module name. Or.. is python-magic going to replace what is in the 'file' pkg someday?

Hi,

We're facing a issue when trying to install thumbor using python-magic version 0.4.7 via pip. Not sure if it's related to python-magic itself or something else, but when I rollback to version 0.4.6 thumbor installs without an issue.

Here's pip log: https://gist.github.com/mjsilva/4d43075e707f999c3c42

Could this be related to recent changes done here?

Appreciate your help. Feel free to ask more debug questions, I can replicate the issue by switching between python-magic versions.

Thanks!

I have installed the latest Python-magic version on a Windows 7 64 bits virtual machine using the pip command.

I have also put the following files to my C:\Windows\System32 folder: magic1.dll, regex2.dll and zlib1.dll.

But when i try to import the magic lib, i have the following error:

Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import magic

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import magic
  File "C:\Python27\lib\site-packages\magic.py", line 115, in <module>
    libmagic = ctypes.CDLL(dll)
  File "C:\Python27\lib\ctypes\__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 193] %1 is not a valid Win32 application

Any idea ?

Since I upgraded to the latest version 0.4.18 of python-magic, my appveyor build keeps crashing after running my tests (although my tests are all successful). Build just before upgrade: https://ci.appveyor.com/project/h3llrais3r/auto-subliminal/builds/33085854/job/fps9g20few2tihb0 Build just after upgrade: https://ci.appveyor.com/project/h3llrais3r/auto-subliminal/builds/33085978/job/pohlxi9iga7ekr0f

When I'm running it on my own machine (Windows 10 x64), I don't have the issue. It seems to be only when running it on appveyor running Windows Server 2012 R2 (travis builds on unix/linux are running without any problems).

Any idea what could be wrong? Sadly I can't find any additional info in the build itself.

Hi. In dependency section mention that "magic1.dll, regex2.dll, and zlib1.dll onto your PATH from the Binaries " where exactly PATH is? ( System Environment PATH? )

I was trying to run coverage on your unittests to find out how much was being tested. I found that when doing coverage run test/test.py one of the tests fail. (test_descriptions for 'test.gz')

$ coverage run test/test.py
F....
======================================================================
FAIL: test_descriptions (__main__.MagicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test.py", line 62, in test_descriptions
    'text.txt': 'ASCII text',
  File "test/test.py", line 34, in assert_values
    self.assertTrue(False, "no match for " + repr(expected_value))
AssertionError: no match for ('gzip compressed data, was "test", from Unix, last modified: Sun Jun 29 01:32:52 2008', 'gzip compressed data, was "test", last modified: Sun Jun 29 01:32:52 2008, from Unix')

----------------------------------------------------------------------
Ran 5 tests in 0.080s

FAILED (failures=1)
Coverage.py warning: No data was collected.

Note that running it with python works.

$ python test/test.py
.....
----------------------------------------------------------------------
Ran 5 tests in 0.074s

OK

Hello, my simple code written in Python 3 doesn't work on Windows 10:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import argparse
import magic
import os
import os.path

# Argparse starts here
parser = argparse.ArgumentParser()								
parser.add_argument('-input', dest='input',help="input one or more files",metavar=None)			
args = parser.parse_args()

input = args.input

file = magic.from_file(input)
print (file)

If I test it against a normal TXT file with

py Test.py -input TestMe.txt

it returns me:

_cffi__xa0d5132dx54cebdac.c
C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\magic\__pycache__\_cffi__xa0d5132dx54cebdac.c(208): fatal error C1083: Cannot open include file: 'magic.h': No such file or directory
Traceback (most recent call last):
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\_msvccompiler.py", line 423, in compile
    self.spawn(args)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\_msvccompiler.py", line 542, in spawn
    return super().spawn(cmd)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\spawn.py", line 38, in spawn
    _spawn_nt(cmd, search_path, dry_run=dry_run)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\spawn.py", line 81, in _spawn_nt
    "command %r failed with exit status %d" % (cmd, rc))
distutils.errors.DistutilsExecError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\cl.exe' failed with exit status 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\ffiplatform.py", line 55, in _build
    dist.run_command('build_ext')
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\setuptools\command\build_ext.py", line 75, in run
    _build_ext.run(self)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\command\build_ext.py", line 339, in run
    self.build_extensions()
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Cython\Distutils\old_build_ext.py", line 194, in build_extensions
    self.build_extension(ext)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\setuptools\command\build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\command\build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\distutils\_msvccompiler.py", line 425, in compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\cl.exe' failed with exit status 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Test.py", line 5, in <module>
    import magic
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\magic\__init__.py", line 1, in <module>
    from . import ffi
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\magic\ffi.py", line 27, in <module>
    ext_package="magic")
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\api.py", line 437, in verify
    lib = self.verifier.load_library()
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\verifier.py", line 113, in load_library
    self._compile_module()
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\verifier.py", line 210, in _compile_module
    outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\ffiplatform.py", line 29, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose)
  File "C:\Users\franc\AppData\Local\Programs\Python\Python36-32\lib\site-packages\cffi\ffiplatform.py", line 62, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.ffiplatform.VerificationError: CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\cl.exe' failed with exit status 2

Yes, Python Magic looks correctly installed:

My Python version is the latest:

I initially encountered this as an apparent issue with eyed3, but have been able to isolate it to magic, which is used by eyed3.

Python script calling magic is non-responsive, even to a CTRL-C; Process Explorer sees CPU to the python task go to nearly 25% (i.e., one core).

Simple script to reproduce:

import sys
print "OS info:\n ", sys.getwindowsversion()
print "Python info:\n ", sys.version
import magic
print "Calling magic..."
print magic.from_file("test.py")
print "returned from magic call."

Output:

OS info:
  sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1')
Python info:
  2.7.8 (default, Jul  2 2014, 19:50:44) [MSC v.1500 32 bit (Intel)]
Calling magic...

... at which point the script hangs, pegging CPU at ~25% (one of four cores), the call never completing. I can kill the Python task from Process Explorer.

based on some print statements, it appears to be hanging in class Magic, __init__, at the statement

self.cookie = magic_open(self.flags)

I used cygmagic-1.dll from Cygwin package "file-5.22-1" (of course, with all dependent dlls) on Windows 7 x64. python-magic lastest from master.

test.py

import magic

magic_detect = magic.Magic()

Result:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    magic_detect = magic.Magic()
  File "D:\Dev\magic.py", line 59, in __init__
    self.cookie = magic_open(self.flags)
WindowsError: exception: access violation writing 0x00000000
Exception AttributeError: "Magic instance has no attribute 'cookie'" in <bound method Magic.__del__ of <magic.Magic instance at 0x024058F0>> ignored

Heads-up: This commit in file:

Author: Christos Zoulas <[email protected]>
Date:   Wed Dec 21 15:55:52 2022 +0000

    Add more extensions and remove truncated stuff from size (Joerg Jenderek)

introduced more extensions for gz-compressed files, breaking the test suite. Consider relaxing the test as for example:

--- a/test/python_magic_test.py
+++ b/test/python_magic_test.py
@@ -134,7 +134,7 @@
             self.assert_values(m, {
                 # some versions return '' for the extensions of a gz file,
                 # including w/ the command line.  Who knows...
-                'test.gz': ('gz/tgz/tpz/zabw/svgz', '', '???'),
+                'test.gz': ('gz/tgz/tpz/zabw/svgz/adz/kmy/xcfgz', 'gz/tgz/tpz/zabw/svgz', '', '???'),
                 'name_use.jpg': 'jpeg/jpg/jpe/jfif',
             })
         except NotImplementedError:

Kind regards,

Windows Server 2016, Python 10.0 python-magic 0.4.27 python-magic-bin 0.4.14

Code:

import magic

def main():
    ft = magic.Magic()
    file = 'memblock.h'
    r = ft.from_file(file)
    print(r)

if __name__ == '__main__':
    main()

Exception raised:

Traceback (most recent call last):
  File "c:\win-checkout\dev1.py", line 10, in <module>
    main()
  File "c:\win-checkout\dev1.py", line 6, in main
    r = ft.from_file(file)
  File "c:\Python3\lib\site-packages\magic\magic.py", line 91, in from_file
    return self._handle509Bug(e)
  File "c:\Python3\lib\site-packages\magic\magic.py", line 100, in _handle509Bug
    raise e
  File "c:\Python3\lib\site-packages\magic\magic.py", line 89, in from_file
    return maybe_decode(magic_file(self.cookie, filename))
  File "c:\Python3\lib\site-packages\magic\magic.py", line 255, in magic_file
    return _magic_file(cookie, coerce_filename(filename))
  File "c:\Python3\lib\site-packages\magic\magic.py", line 196, in errorcheck_null
    raise MagicException(err)
magic.magic.MagicException: b"line I64u: regex error 14 for `^[[:space:]]*class[[:space:]]+[[:digit:][:alpha:]:_]+[[:space:]]*\\{(.*[\n]*)*\\}(;)?$', (failed to get memory)"

The problematic file attached (extension changed to txt). memblock.txt

On Windows 10 a plain text buffer is classified as application/octet-stream which is indicative of a binary file (I think). Ubuntu Linux is correctly identifying as "text/plain".

Python3.9, python-magic-bin as the driver for magiclib

How to reproduce

content = "Bunch of fake content" * 8192

magic = Magic(mime=True)
mimetype = magic.from_buffer(content[0:8192]) # Index to replicate my environment

I'm getting this error of Magic version on my Virtual Environment when I run python3.8 manage.py migrate on my Django REST API.

I've already done the following after activating Virtual Environment:

pip3 install python-magic --upgrade Here is what exactly is shown at the very end.

magic.MagicException: b"File 5.39 supports only version 16 magic files.`myproject/misc/magic.mgc' is version 14>

Here is the detailed error log.

myproject_common/misc/magic.mgc, 2607: Warning: offset [' invalid myproject_common/misc/magic.mgc, 2608: Warning: offset[' invalid myproject_common/misc/magic.mgc, 2617: Warning: offset .' invalid myproject_common/misc/magic.mgc, 2619: Warning: offset.' invalid myproject_common/misc/magic.mgc, 2635: Warning: offset ' invalid myproject_common/misc/magic.mgc, 2653: Warning: offset.' invalid Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/earthling/myEnv/lib/python3.8/site-packages/django/core/management/init.py", line 381, in execute_from_command_line utility.execute() File "/home/earthling/myEnv/lib/python3.8/site-packages/django/core/management/init.py", line 357, in execute django.setup() File "/home/earthling/myEnv/lib/python3.8/site-packages/django/init.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/home/earthling/myEnv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate app_config.import_models() File "/home/earthling/myEnv/lib/python3.8/site-packages/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/home/earthling/myprojects/MyAPI/myproject_auth/models.py", line 31, in from myproject_common.utils.helpers import delete_file_field File "/home/earthling/myprojects/MyAPI/myproject_common/utils/helpers.py", line 131, in magic = magic.Magic(magic_file='myproject_common/misc/magic.mgc', mime=True) File "/home/earthling/myEnv/lib/python3.8/site-packages/magic.py", line 66, in init magic_load(self.cookie, magic_file) File "/home/earthling/myEnv/lib/python3.8/site-packages/magic.py", line 291, in magic_load return _magic_load(cookie, coerce_filename(filename)) File "/home/earthling/myEnv/lib/python3.8/site-packages/magic.py", line 217, in errorcheck_negative_one raise MagicException(err) magic.MagicException: b"File 5.41 supports only version 16 magic files. `myproject_common/misc/magic.mgc' is version 14"