Multiple startup errors and no SimpleITK in May 1 nightly on mac

Startup error:

code for hash md5 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
code for hash sha1 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
code for hash sha224 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha224
code for hash sha256 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha256
code for hash sha384 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha384
code for hash sha512 was not found.
Traceback (most recent call last):
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha512

SimpleITK import error:

>>> import SimpleITK
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/site-packages/SimpleITK/__init__.py", line 1, in <module>
    from .SimpleITK import *
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/site-packages/SimpleITK/SimpleITK.py", line 21, in <module>
    _SimpleITK = swig_import_helper()
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/site-packages/SimpleITK/SimpleITK.py", line 20, in swig_import_helper
    return importlib.import_module('_SimpleITK')
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _SimpleITK

The dashboard looks quite good.
http://slicer.cdash.org/index.php?project=Slicer4

Does this error come up on other computers, too?

I confirm I am getting this error even after removing ~/.config.

Do other Mac users have this problem, too?

Have you installed Python or OpenSSL lately? (it might be related, see https://github.com/Homebrew/legacy-homebrew/issues/22816)

Does it work with earlier Slicer versions? There were Python packaging related changes and OpenSSL library install location change in the past few weeks.

@lassoan I have not changed anything in OpenSSL (not to my knowledge, at least!). I did install python via pyenv recently.

I do not use homebrew.

If I disable the new python I installed (it is no longer in my path), I still get the same startup errors.

@pieper and other Mac users: can you reproduce this, too?

@Fedorov Does the problem occur only with the latest version or earlier nightly or stable version, too?

Stable works fine for me. Never had this problem with nightlies before. I tried on another Mac system, different (latest) macOS version, no updates to system python, and I am getting exact same errors.

Remarkable how with all the glorious downloads we get every day not a single Mac user to test this anywhere else.

It seems that something happened between r25945 (2017-04-25 nightly, works) and r25971 (2017-04-26 nightly, doesn’t work).

As part of this process, I also discovered a new “feature” of macOS + Slicer: apparently, it is no longer possible to allow launching apps from unidentified developers, so every time new Slicer is installed, it has to be launched twice, second time allowing to launch in the security panel. Quite annoying, considering initial launch of Slicer on mac is super slow, and macOS verifies package on each of the 2 launches…

image

@jcfr this regression may be related to your Python updates/packaging changes between 25945 and 25971. Could you please check what could be the problem?

I see. It may be related to the following commit done between 2.7.11 and 2.7.13.

There is a big storm going on in the area and I lost the ssh connection to the macosx. I will have a look later.

I tested the May 1 nightly on a mac with 10.9.5 and it started with no
problem (right click icon and pick open from menu). I can try other macs
later.

I get the same failures to import hashlib on max os 10.12.4. Otherwise the
startup process is the same as on older macs (only need to open from menu
and pick okay) probably because I already have the security preference
selected.

Fails for me on older mac too.

Discussed this with Jc today and he will try going back on python version to use the older OpenSSL for now.

@pieper no, startup process is not the same, since in the older mac you could choose “allow from anywhere” (as shown below), and in the newer macs this option is gone (as shown in a post above).

Right-click+Open (or Ctrl-click+Open) still works fine for any application. You can restore the “Anywhere” setting in Gatekeeper with a command line switch, see: http://www.imore.com/how-open-apps-unidentified-developers-mac (not really recommended…)

I had no idea. I guess this proves that I remain a novice Mac user!

@jcfr @pieper I took a quick look at the 5/1 nightly, and the underlying issue is the failure to import hashlib:

error message
import _hashlib
Traceback (most recent call last):
  File "<console>", line 1, in <module>
ImportError: dlopen(/opt/worksw/Slicer-4.7.0-20170501.app/Contents/lib/Python/lib/python2.7/lib-dynload/_hashlib.so, 2): Library not loaded: libcrypto.dylib
  Referenced from: /opt/worksw/Slicer-4.7.0-20170501.app/Contents/lib/Python/lib/python2.7/lib-dynload/_hashlib.so
  Reason: Incompatible library version: _hashlib.so requires version 1.0.0 or later, but libcrypto.0.9.8.dylib provides version 0.9.8
>>> 

There is a try block around this import, when it fails then Python falls back to looking for the generic versions of the hash functions (which aren’t compiled/available) instead of using the ones from OpenSSL.

Looking at vmmap and dyld debug trace, Slicer is loading both the Slicer version and the system version (/usr/lib/libcrypto.0.9.8.dylib) of libcrypto (and libssl). Slicer version of libcrypto/libssl is opened first, but the system version is preferred when later dlopen’ing the Python _hashlib. (maybe Slicer version is only opened RTLD_LOCAL, while system version is opened RTLD_GLOBAL. no simple way to check this)

It can be fixed locally by correcting the dependency paths, e.g. install_name_tool -change /Users/kitware/Dashboards/Nightly/Slicer-0-build/OpenSSL/libssl.dylib /opt/worksw/Slicer-4.7.0-20170501.app/Contents/lib/Slicer-4.7/libssl.dylib /opt/worksw/Slicer-4.7.0-20170501.app/Contents/lib/Python/lib/python2.7/lib-dynload/_hashlib.so (and likewise for libcrypto).

So possible fixes would be

  • correct ®PATHs for _hashlib on the build system
  • figure out why we are loading/preferring the system libcrypto at all.

@ihnorton Thanks for the detailed report

The following code is/was intended to take care of this, it basically ensures the full path is associated with the libraries which then allow the fixup script to work as expected.

https://github.com/Slicer/Slicer/blob/fba6424b7d48fb4418295105894e317de6e850bf/SuperBuild/External_OpenSSL.cmake#L123-L131

I suspect the regression was introduced here:

https://github.com/Slicer/Slicer/blob/1e9cc24cb026db42f36da9914308366a2d643405/CMake/SlicerCPackBundleFixup.cmake.in#L186-L192

I think I identified the issue.

After installing the module into <root>/python-install/lib/python2.7.lib-dynload the full path to the module is stripped and this prevent the fixup package from working.

It was not an issue until now because I suspect the version 0.9.8 of ssl was available on the target system.

Currently working on a fix.

1 Like