I noticed that there is something wrong with the order of when which dependency is loaded. It’s either a problem with my extensions or Slicer. Not sure about this.
We have a module Quantitative Reporting [1] which depends (not much longer) on SlicerProstate [2]. I have been working on factoring out some code into another repository and I noticed that the dependency is loaded to late into Slicer (even when I compile the python scripted module).
Here an example for reproduction:
Execute the following in a directory that you want to use for testing this issue:
Add the {SlicerProstate_Build_Directory}/lib/Slicer4.7/qt-scripted-modules
Restart Slicer
>>> Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/christian/sources/py/Reporting/QuantitativeReporting.py", line 8, in <module>
from SlicerProstateUtils.mixins import *
ImportError: No module named SlicerProstateUtils.mixins
When I add mpReview[3] (both depend on SlicerProstate) I only get errors from QuantitativeReporting but not form mpReview displayed.
@che85 wow, I have to say I had to re-read this many times!
I don’t know what’s going on, but maybe the problem can be narrowed down a bit better:
I think it would be more consistent (and that is how the factories work) to build each of the extensions, and add paths to the built directories for all of the modules
can you reproduce the problem if you just take SlicerProstate, and then try to import from python console SlicerProstateUtils.mixins first compiling SlicerProstate with DistanceMapRegistration “added”, and then cleaning it up and building it without that directory?
It is a bit difficult to follow because of many pieces involved.
I thought about this to, but this would make it even more complicated, right?
I don’t know how dependencies are resolved right now when starting Slicer, but when there is a dependency defined for a module and the dependency hasn’t been loaded yet, the load process of that module should be moved to the end of the list for modules that need to be loaded. In the end if the dependency still couldn’t be resolved, it should simply throw an error.
Maybe JC can tell us more about that loading process?
I think it would be more consistent (and that is how the factories work) to build each of the extensions, and add paths to the built directories for all of the modules
Loading SlicerProstateUtils.mixins from the python console works in any case.
can you reproduce the problem if you just take SlicerProstate, and then try to import from python console SlicerProstateUtils.mixins first compiling SlicerProstate with DistanceMapRegistration “added”, and then cleaning it up and building it without that directory?
@che85 there are multiple stages in the module initialization process: registration, instantiation, and loading. Instantiation is when the scripted module is parsed, so any import dependencies must be resolveable at that time. However, instantiation is done alphabetically, with no dependency resolution – because the module factory does not know anything about dependencies (the module itself supplies the dependencies function). QuantitativeReporting works “by accident” when you have DistanceMapBasedRegistration in place because DMBR is in the same directory where SlicerProstateUtils is installed, so that the parent directory is added to PYTHONPATH before attempting to instantiate QuantitativeReporting (During instantiation, the path for each module is added to PYTHONPATH). When you remove it, SlicerProstate comes later alphabetically, so the path is not available when QR is instantiated – as @fedorov noted, the import works fine in the python console once Slicer starts.
Anyway, there’s probably a few ways this could be fixed, including:
add all the additionalModulePaths to PYTHONPATH (not sure what this would break, there’s a lot of existing logic for adding to PYTHONPATH in various places, including for regular loadable modules).
add the path during registration rather than instantiation (simplest fix I can think of). Try this patch:
patch to test
diff --git a/Base/QTGUI/qSlicerScriptedLoadableModuleFactory.cxx b/Base/QTGUI/qSlicerScriptedLoadableModuleFactory.cxx
index 9ca65a503..939160247 100644
--- a/Base/QTGUI/qSlicerScriptedLoadableModuleFactory.cxx
+++ b/Base/QTGUI/qSlicerScriptedLoadableModuleFactory.cxx
@@ -43,6 +43,24 @@
//----------------------------------------------------------------------------
bool ctkFactoryScriptedItem::load()
{
+#ifdef Slicer_USE_PYTHONQT
+ if (!qSlicerCoreApplication::testAttribute(qSlicerCoreApplication::AA_DisablePython))
+ {
+ // By convention, if the module is not embedded, "<MODULEPATH>" will be appended to PYTHONPATH
+ if (!qSlicerCoreApplication::application()->isEmbeddedModule(this->path()))
+ {
+ QDir modulePathWithoutIntDir = QFileInfo(this->path()).dir();
+ QString intDir = qSlicerCoreApplication::application()->intDir();
+ if (intDir == modulePathWithoutIntDir.dirName())
+ {
+ modulePathWithoutIntDir.cdUp();
+ }
+ qSlicerCorePythonManager * pythonManager = qSlicerCoreApplication::application()->corePythonManager();
+ pythonManager->appendPythonPaths(QStringList() << modulePathWithoutIntDir.absolutePath());
+ }
+ }
+#endif
+
return true;
}
@@ -59,24 +77,6 @@ qSlicerAbstractCoreModule* ctkFactoryScriptedItem::instanciator()
module->setInstalled(qSlicerUtils::isPluginInstalled(this->path(), app->slicerHome()));
module->setBuiltIn(qSlicerUtils::isPluginBuiltIn(this->path(), app->slicerHome()));
-#ifdef Slicer_USE_PYTHONQT
- if (!qSlicerCoreApplication::testAttribute(qSlicerCoreApplication::AA_DisablePython))
- {
- // By convention, if the module is not embedded, "<MODULEPATH>" will be appended to PYTHONPATH
- if (!qSlicerCoreApplication::application()->isEmbeddedModule(module->path()))
- {
- QDir modulePathWithoutIntDir = QFileInfo(module->path()).dir();
- QString intDir = qSlicerCoreApplication::application()->intDir();
- if (intDir == modulePathWithoutIntDir.dirName())
- {
- modulePathWithoutIntDir.cdUp();
- }
- qSlicerCorePythonManager * pythonManager = qSlicerCoreApplication::application()->corePythonManager();
- pythonManager->appendPythonPaths(QStringList() << modulePathWithoutIntDir.absolutePath());
- }
- }
-#endif
-
bool ret = module->setPythonSource(this->path());
if (!ret)
{
@ihnorton, @fedorov I applied the patch and compiled Slicer under Ubuntu and indeed it works! I don’t get those errors anymore. When can I expect this to be available in the nightly?
from Jc I’ll commit tomorrow if there’s no objection.