Extension adding an entry to Slicer-XYZ.ini [PYTHONPATH]

adamrankin · January 30, 2018, 4:18pm

Hello,

SlicerOpenCV currently installed cv2.pyd to the root directory, and currently import cv2 fails. A fix to this is to install it to the qt-scripted-modules folder, but I was wondering if there was any interest in adding the ability for an extension to add entries to the Slicer-XYZ.ini sections?

For example, the extension could add a path entry to the [PYTHONPATH] section.

Thoughts?
Adam

code of interest:
qSlicerExtensionsManagerModelPrivate::addExtensionPathToLauncherSettings
qSlicerExtensionsManagerModelPrivate::extensionPythonPaths

adamrankin · January 30, 2018, 4:33pm

Alternately, an extension could do this manually in a module’s init function, accessing the qSlicerExtensionsModelManager functionality.

Edit: uninstalling the extension wouldn’t remove the entry though, so this is sub-optimal.

lassoan · January 30, 2018, 5:12pm

Any scripted module that calls import cv2 could add the dll’s path to Python paths, but this, too, would be just a workaround.

For me, the main question is:

why cv2.pyd is installed into 26876-win-amd64-SlicerOpenCV-git8ea8ff1-2018-01-29\lib\Slicer-4.9\cv2.pyd instead of in a scripted module directory? (scripted module directories are already added to PYTHONPATH section)
if cv2.pyd current location is preferable for some reason, then why not add that location to PYTHONPATH section as well?

@jcfr Could you comment on these?

adamrankin · January 30, 2018, 5:30pm

Not a perfect answer to your first point, but the extension packaging would have to assume that the directory is “qt-scripted-modules”. Probably a reasonable assumption, but if the scripted directory ever changed, the extension would break (extremely unlikely scenario, I agree).

At the moment, point two is not possible, and was the goal of this discussion.

adamrankin · January 30, 2018, 5:31pm

@fedorov @Nicole_Aucoin any thoughts?

Nicole_Aucoin · January 30, 2018, 5:55pm

I recall working with @jcfr on this problem during OpenCV integration and I think that this bug was the reason why the opencv library was installed where it was. I can’t dig up the details though.

Nicole

fedorov · January 30, 2018, 6:03pm

Adam, I don’t have answers to your questions. But to me it seems suboptimal to modify those variables from the module code.

adamrankin · January 30, 2018, 6:08pm

The alternative is installing cv2.pyd to qt-scripted-modules.

Edit: or implementing a system for extensions to install/uninstall entries to the settings file.

ihnorton · January 30, 2018, 6:25pm

Maybe it could be simpler to have settings aggregated from a folder: Slicer.(app,exe) would look in that folder and append search settings from all the .ini files contained within. I think this would mesh well with the extension manager, and could also make local developer workflow nicer: just symlink .ini files into the aggregate folder from local extension build trees.

Nicole_Aucoin · January 30, 2018, 6:33pm

Here’s the commit where I talked about trying to get python wrapping and importing to work:

github.com/Slicer/SlicerOpenCV

ENH: Add python wrapping to OpenCV

committed 02:17PM - 13 Sep 16 UTC

+372 -3

Turn on wrapping OpenCV in python. This produces the cv2.so python package that …can be imported. Use the OpenCV CMake var -DPYTHON2_PACKAGES_PATH to specify that the OpenCV python package needs to be installed in the third party library directory, it will be found there by changes made to the AdditionalLauncherSettings.ini file in Slicer svn 25305. To get packaged, it has to be in the Slicer third party library install directory. Adapted the opencv python example that computes the histogram of an input image to provide a self test. Uploaded a copy of one of the Slicer Pathology sample slices to midas and download it for this test. Don't install the ctest command line test on windows as it's not finding the python wrapped library when the test is run from the build directory. It still runs from the Testing module gui. Issue #12

It referenced:

fedorov · January 30, 2018, 9:21pm

Thanks @Nicole_Aucoin! I added those pointers to the issue.

jcfr · January 31, 2018, 4:14am

Since r25959 introduced in April 2017, extension can already package python modules and packages using PYTHON_SITE_PACKAGES_SUBDIR CMake variable to specify the install destination.

The packages found in this sub directory will be importable.

github.com

Slicer/Slicer/blob/5d294412119bf51f14c3a19ec98f055c1cbe30fe/Base/QTCore/qSlicerExtensionsManagerModel.cxx#L735-L751


#ifdef Slicer_USE_PYTHONQT
// --------------------------------------------------------------------------
QStringList qSlicerExtensionsManagerModelPrivate::extensionPythonPaths(const QString& extensionName)const
{
Q_Q(const qSlicerExtensionsManagerModel);
if (this->SlicerVersion.isEmpty())
  {
  return QStringList();
  }
QString path = q->extensionInstallPath(extensionName);
return appendToPathList(QStringList(), QStringList()
                        << path + "/" + QString(Slicer_QTSCRIPTEDMODULES_LIB_DIR).replace(Slicer_VERSION, this->SlicerVersion)
                        << path + "/" + QString(Slicer_QTLOADABLEMODULES_PYTHON_LIB_DIR).replace(Slicer_VERSION, this->SlicerVersion)
                        << path + "/" + QString(PYTHON_SITE_PACKAGES_SUBDIR)
                        );
}
#endif

I suggest to change this line:

github.com

SBU-BMI/SlicerOpenCV/blob/74e0f836bdc284e3b73e541c8d770808ccadec55/SuperBuild/External_OpenCV.cmake#L103


    -DWITH_OPENCL:BOOL=OFF
    # Disable find_package(Java) so that java wrapping is not done
    -DCMAKE_DISABLE_FIND_PACKAGE_JAVA:BOOL=ON
    # Add Python wrapping, use Slicer's python
    -DBUILD_opencv_python2:BOOL=ON
    -DPYTHON_EXECUTABLE:FILEPATH=${PYTHON_EXECUTABLE}
    -DPYTHON_INCLUDE_DIR:PATH=${PYTHON_INCLUDE_DIR}
    -DPYTHON_LIBRARY:FILEPATH=${PYTHON_LIBRARY}
    -DINSTALL_PYTHON_EXAMPLES:BOOL=OFF
    # install the python package in the third party lib dir
    -DPYTHON2_PACKAGES_PATH:PATH=${Slicer_INSTALL_THIRDPARTY_LIB_DIR}
	  ${ADDITIONAL_OPENCV_ARGS}
  DEPENDS
    ${${proj}_DEPENDENCIES}
  )
set(OpenCV_DIR ${${proj}_INSTALL_DIR})
if(UNIX)
  set(OpenCV_DIR ${${proj}_INSTALL_DIR}/share/OpenCV)
endif()
else()
# The project is provided using OpenCV_DIR, nevertheless since other projects

fedorov · January 31, 2018, 10:39pm

Thank you @jcfr for the suggestion, and @adamrankin for the PR!

The fix is now merged in https://github.com/SBU-BMI/SlicerOpenCV/pull/50. We shall see, tomorrow.

fedorov · February 19, 2018, 6:14pm

As a followup, the suggestion above was implemented, but did not fix the import cv2 issue.

github.com

SBU-BMI/SlicerOpenCV/blob/master/SuperBuild/External_OpenCV.cmake#L103


    -DWITH_OPENCL:BOOL=OFF
    # Disable find_package(Java) so that java wrapping is not done
    -DCMAKE_DISABLE_FIND_PACKAGE_JAVA:BOOL=ON
    # Add Python wrapping, use Slicer's python
    -DBUILD_opencv_python2:BOOL=ON
    -DPYTHON_EXECUTABLE:FILEPATH=${PYTHON_EXECUTABLE}
    -DPYTHON_INCLUDE_DIR:PATH=${PYTHON_INCLUDE_DIR}
    -DPYTHON_LIBRARY:FILEPATH=${PYTHON_LIBRARY}
    -DINSTALL_PYTHON_EXAMPLES:BOOL=OFF
    # install the python package in the third party lib dir
    -DPYTHON2_PACKAGES_PATH:PATH=${PYTHON_SITE_PACKAGES_SUBDIR}
    ${ADDITIONAL_OPENCV_ARGS}
  DEPENDS
    ${${proj}_DEPENDENCIES}
  )
set(OpenCV_DIR ${${proj}_INSTALL_DIR})
if(UNIX)
  set(OpenCV_DIR ${${proj}_INSTALL_DIR}/share/OpenCV)
endif()
else()
# The project is provided using OpenCV_DIR, nevertheless since other projects

lassoan · February 19, 2018, 11:05pm

What did you do exactly?

Do you create an extension package and install it or you run from the build tree? If you run from the build tree: did you specify the additional launcher settings ini file on the command line when you started Slicer?

fedorov · February 19, 2018, 11:21pm

What did you do exactly?

I downloaded the nightly package, installed SlicerOpenCV extension, and did import cv2 in the python console.

lassoan · February 20, 2018, 1:16am

import cv2 works well on Windows (latest nightly).

Do you have this problem on Mac? Maybe it’s the same Apple SIP policy issue that plagues so many other things - see for example Tutorial for using pyradiomics, no module named _collections.

fedorov · February 20, 2018, 4:11am

Yes, I tested on mac

Topic		Replies	Views
Programatically install extensions Development extensions-manager , python	1	601	August 18, 2021
How can I modify Slicer's PYTHONPATH variable? Support	4	1216	June 5, 2018
Installing Slicer Extensions from cmd with python Support	7	356	October 26, 2023
Install extension module in built Slicer Development extensions	10	498	October 7, 2023
Custom module does not appear in 'modules' section in Settings Support	11	74	September 18, 2024

Extension adding an entry to Slicer-XYZ.ini [PYTHONPATH]

Related topics