ImportError: failed to find libmagic. Check your installation

[lucas-rudd]

I believe this may be an issue with the magic package used by thehive4py/api.py

After installing elastalert v 0.1.3.6 I get the following stacktrace when I try to run an elastalert test with a folder of rules

Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
RMTMACSIERRA01:infrastructure-cm lucas.rudd$ docker run --rm -it -e ES_HOST=search-pss-pf-prod-es-51-gcm4nbuhxjuo6telgbszkp3i7u.us-east-1.es.amazonaws.com -e AWS_PROFILE=asurion-soluto-nonprod.dev -v ~/.aws:/home/elastalert/.aws  -v ~/Desktop/rules/:/home/elastalert/rules --entrypoint elastalert asurionpss/elastalert:test --verbose --es_debug
Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation

After manually installing libmagic the issue persists.

Has anyone else experienced this issue?

[synhershko]

Here too

[synhershko]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

[lsoumille]

@synhershko Your solution doesn't work for me because python-magic requirement is >=0.4.15. Here is the error message:

pkg_resources.DistributionNotFound: The 'python-magic>=0.4.15' distribution was not found and is required by elastalert

[primeroz]

I just installed libmagic and it fixed that error. I am running an alpine container

[eliasbrange]

+1. Our elastalert suddenly started failing in our CI due to libmagic not being installed. Might want to add it to the docs that it is required. Even better, don't try to import it unless a Hive rule is configured.

[wuxinwei]

+1. I got the same problem when I was run elastalert at alpine container

[wuxinwei]

@primeroz , Can you tell me that how do you solve this problem, just install libmagic when build docker image?

[eliasbrange]

@primeroz , Can you tell me that how do you solve this problem, just install libmagic when build docker image?

We've just added apk add libmagic.

[primeroz]

@wuxinwei as @eliasbrange mentioned just apk add libmagic

[wuxinwei]

@eliasbrange ,thank you, the solution which you mentioned solved my problem

[Samuel-Gong]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

[skatakar]

Observed the same issue while running elastalert
1)D:\elastalert>elastalert-test-rule "D:\elastalert\example_rules\example_frequenc
y.yaml"
Traceback (most recent call last):
File "D:\Python27\Scripts\elastalert-test-rule-script.py", line 11, in <module

load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert-test-r

ule')()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 480, in l
oad_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2693, in
load_entry_point
return ep.load()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2324, in
load
return self.resolve()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2330, in
resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "D:\Python27\Lib\site-packages\elastalert\test_rule.py", line 20, in
import elastalert.config
File "D:\Python27\Lib\site-packages\elastalert\config.py", line 9, in

import alerts

File "D:\Python27\Lib\site-packages\elastalert\alerts.py", line 31, in <module

from thehive4py.api import TheHiveApi

File "D:\Python27\Lib\site-packages\thehive4py\api.py", line 7, in
import magic
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation

D:\elastalert>

C:\Users\u3xlbes>elastalert --start NOW --verbose
Traceback (most recent call last):
File "D:\Python27\Scripts\elastalert-script.py", line 11, in
load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 480, in l
oad_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2693, in
load_entry_point
return ep.load()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2324, in
load
return self.resolve()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2330, in
resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "D:\Python27\Lib\site-packages\elastalert\elastalert.py", line 21, in
from alerts import DebugAlerter
File "D:\Python27\Lib\site-packages\elastalert\alerts.py", line 31, in <module

from thehive4py.api import TheHiveApi

File "D:\Python27\Lib\site-packages\thehive4py\api.py", line 7, in
import magic
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation

C:\Users\u3xlbes>

we are using python 2.7.15 . any workaround for this issue?

[skatakar]

Do we have any workaround or solution for this issue?

[eliasbrange]

Do we have any workaround or solution for this issue?

Check out https://github.com/pidydx/libmagicwin64 for installing libmagic on windows.

[skatakar]

Do we have any workaround or solution for this issue?

Check out https://github.com/pidydx/libmagicwin64 for installing libmagic on windows.

I have downloaded all 4 files which are mentioned in the above link and placed under C:\Windows\System32.. But when I try to import magic or import liblogic, its giving same error.

import magic
Traceback (most recent call last):
File "", line 1, in
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
import libmagic
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named libmagic

Do we need to run/configure any thing after download those 4 files?

we have python 32bit on windows 2012 64 bit

platform.architecture()
('32bit', 'WindowsPE')