I have discovered that the python in bin/python-real is different from the python interactor compiled into Slicer
Using the pre-combiled binaries on linux (both 4.9.0 and 4.8.1) - I have this problem.
For instance in 4.9.0
For the interactor I have: python 2.7.13, which is compiled with ucs2 encoding
For python-real I have: python 2.7.12, compiled with ucs4 encoding
For those unaware the ucs encoding can be discovered as follows:
import sys
sys.maxunicode
This difference (the ucs encoding) makes many packages working with python-real not work with the python interactor. It would be really great to have access to the interactor python binary - for making use of virtualenv and the like to install python packages.
Right now my workaround is I compile a python binary with the same version and ucs encoding as the interactor, setup a virtualenv, and use this to install the python packages I desire / need.
My questions are:
Where is the actual python binary that the interactor uses?
Can this issue be fixed in the next release?
(I am not familiar enough with the slicer build system to fix this problem myself)
As you can see, the python-real executable can not be started directly. Otherwise it will resolve its symbols against a python library available somewhere else on your system.
launcher
SlicerPython is a convenience launcher allowing to set the environment. The same one is used to start Slicer.
./python-install/bin/SlicerPython --launcher-help
Usage
SlicerPython [options]
Options
--launcher-help Display help
--launcher-version Show launcher version information
--launcher-verbose Verbose mode
--launch Specify the application to launch
--launcher-detach Launcher will NOT wait for the application to finish
--launcher-no-splash Hide launcher splash
--launcher-timeout Specify the time in second before the launcher kills the application. -1 means no timeout (default: -1)
--launcher-load-environment Specify the saved environment to load.
--launcher-dump-environment Launcher will print environment variables to be set, then exit
--launcher-show-set-environment-commands Launcher will print commands suitable for setting the parent environment (i.e. using 'eval' in a POSIX shell), then exit
--launcher-additional-settings Additional settings file to consider
--launcher-ignore-user-additional-settings Ignore additional user settings
--launcher-generate-exec-wrapper-script Generate executable wrapper script allowing to set the environment
--launcher-generate-template Generate an example of setting file
Using --launcher-show-set-environment-commands you could conveniently update the current environment and then start python-real directly.
I really appreciate your informative and prompt reply.
Based on your suggestion I used SlicerPython --launcher-show-set-environment-commands to update the shell environment variables and run python-real directly
This did bring up the same python version and ucs encoding as the interpreter with python-real. Unfortunately, within the shell environment, virtualenv wouldn’t work anymore failing with the error:
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Current thread 0x00007fd9b9a95700 (most recent call first):
Aborted
I haven’t done a lot of digging into what is causing this, but will post again if I figure it out. In any case, using virtualenv with the workaround I described above seems a useful / flexible way to install python packages and import them into slicer (including the pre-compiled binary). Maybe that will be helpful to some, as with it I can bring in sklearn, numba, pycuda, scipy, etc…
For now, I haven’t been able to setup a virtualenv with the Slicer’s own python and am stuck having to compile a compatible version from source for the the virtualenv. It is not a terrible solution…
Would you mind sharing the output of SlicerPython --launcher-show-set-environment-commands, I suspect a system path is prepended to either PATH or LD_LIBRARY_PATH causing your system library to be picked up instead of the one buitd by Slicer.
Also, did you build against Qt5 or Qt4 ?
This used to happen when building against a system Qt4, but we added logic to account for this. I suspect we need to add a similar logic for Qt5_DIR.
the workaround I described above seems a useful / flexible way to install python packages and import them into slicer
I haven’t been able to setup a virtualenv with the Slicer’s own python
Consider Slicer python is itself an isolated environment and installing python package (on Linux) is already possible, we haven’t done much testing using virtualenv in this context.
To support virtualenv, we need to make sure that the PYTHONPATH used when starting Slicer is also updated when switching virtual env.
I have been able to install many python packages to slicer from the python interpreter using eg pip.main(['install','scipy']). However, having a virtualenv allowed me to compile / install packages from source and import packages (numba, pycuda for instance) that weren’t available or didn’t work from me when installing with pip from the interpreter. Bringing in the packages to slicer with sys.path.append to the virtualenv package directory means that the slicer build is never harmed when things are going badly…
I am not sure if others are having the same limitations I am using pip from the interpreter, or if a better solution exists. Anyhow, hopefully that explains my rationale for wanting to use a virtualenv.