Build on macOS 11 fails in the middle (after building vtkpython)

Hi! I’m having trouble trying to build Slicer on macOS 11 (Intel)

  • Source code from git (master branch)
  • Qt 5.15.0
  • cmake version 3.19.3
  • Apple clang version 12.0.0 (clang-1200.0.32.28)

I configure the build with this command:

cmake \
    -DCMAKE_OSX_DEPLOYMENT_TARGET=11 \
    -DCMAKE_BUILD_TYPE:STRING=Debug \
    -DQt5_DIR:PATH=/Users/p2p/Qt/5.15.0/clang_64/lib/cmake/Qt5 \
    ../Slicer

The build continues for somewhat 30 minutes to stop with a message that’s not that informative to me

 [100%] Linking CXX shared library ../../lib/libvtkViewsInfovisPython36D-8.2.dylib
 [100%] Built target vtkViewsInfovisPythonD
 Scanning dependencies of target vtkViewsInfovisPython
 [100%] Building CXX object Wrapping/Python/CMakeFiles/vtkViewsInfovisPython.dir/vtkViewsInfovisPythonInit.cxx.o
 [100%] Linking CXX shared module ../../lib/python3.6/site-packages/vtkmodules/vtkViewsInfovisPython.so
 [100%] Built target vtkViewsInfovisPython
 Scanning dependencies of target vtk_python_package
 [100%] Built target vtk_python_package
 Scanning dependencies of target vtkpython
 [100%] Building CXX object Wrapping/Python/CMakeFiles/vtkpython.dir/vtkPythonAppInit.cxx.o
 [100%] Linking CXX executable ../../bin/vtkpython
 [100%] Built target vtkpython
 [ 81%] No install step for 'VTK'
 [ 81%] Creating '/Users/p2p/GitClones/CMI/Slicer-SuperBuild/python-install/lib/python3.6/site-packages/vtk-8.2.0-py3.6.egg-info' directory
 [ 81%] Completed 'VTK'
 [ 81%] Built target VTK
 make: *** [all] Error 2

The CMakeFiles/CMakeError.log has the following lines

Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: /Applications/DevelopmentTools/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
Build flags:
Id flags:
The output was:
1
ld: library not found for -lSystem
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /Applications/DevelopmentTools/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
Build flags:
Id flags:
The output was:
1
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Am I missing something in the configuration? Are there any mac-heads here who can advice how to debug this?

If this might help suggesting a debug path, the last modified error log file in the superbuild folder is ./VTK-build/CMakeFiles/CMakeError.log and it ends with the following error message:


int main(int argc, char** argv)
{
  (void)argv;
#ifndef _stat
  return ((int*)(&_stat))[argc];
#else
  (void)argc;
  return 0;
#endif
}
Checking for DIR in sys/ndir.h failed to compile with the following output:
Change Dir: /tmp/sb/VTK-build/ThirdParty/libxml2/vtklibxml2/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_99e16/fast && /Applications/DevelopmentTools/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/cmTC_99e16.dir/build.make CMakeFiles/cmTC_99e16.dir/build
Building C object CMakeFiles/cmTC_99e16.dir/platformTestsC.c.o
/Applications/DevelopmentTools/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DTEST_HAVE_SYS_NDIR_H  -w -w  -isysroot /Applications/DevelopmentTools/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11 -o CMakeFiles/cmTC_99e16.dir/platformTestsC.c.o -c /tmp/sb/VTK/ThirdParty/libxml2/vtklibxml2/platformTestsC.c
/tmp/sb/VTK/ThirdParty/libxml2/vtklibxml2/platformTestsC.c:82:10: fatal error: 'sys/ndir.h' file not found
#include <sys/ndir.h>
         ^~~~~~~~~~~~
1 error generated.
make[4]: *** [CMakeFiles/cmTC_99e16.dir/platformTestsC.c.o] Error 1
make[3]: *** [cmTC_99e16/fast] Error 2

I don’t get these errors on my mac builds. I’ll bet your problem is using /tmp as the build tree, since files may get deleted from there as it fills. On mac I now use /opt/s to keep the path short.

Thanks, will try that. I was looking through error logs that are left after the build and noticed several odd things:

  • The python-numpy error log says that the hash of the downloaded package doesn’t match the hash that’s in the repo
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    numpy==1.19.2 from https://files.pythonhosted.org/packages/bf/e8/15aea783ea72e2d4e51e3ec365e8dc4a1a32c9e5eb3a6d695b0d58e67cdd/numpy-1.19.2.zip#sha256=0d310730e1e793527065ad7dde736197b705d0e4c9999775f212b03c44a8484c (from -r /tmp/sb/python-numpy-requirements.txt (line 8)):
        Expected sha256 967c92435f0b3ba37a4257c48b8715b76741410467e2bdb1097e8391fccfae15
        Expected     or b594f76771bc7fc8a044c5ba303427ee67c17a09b36e1fa32bde82f5c419d17a
        Expected     or 3733640466733441295b0d6d3dcbf8e1ffa7e897d4d82903169529fd3386919a
        Expected     or 7c6646314291d8f5ea900a7ea9c4261f834b5b62159ba2abe3836f4fa6705526
        Expected     or 7118f0a9f2f617f921ec7d278d981244ba83c85eea197be7c5a4f84af80a9c3c
             Got        0d310730e1e793527065ad7dde736197b705d0e4c9999775f212b03c44a8484c

After I manually add the hash of the package that I get I ran into a different error saying that pip is looking for a MACOSX_DEPLOYMENT_TARGET that should be a string, not an int. I got your message when I just started the build specifying both the hash and -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=11.1 but before I continue my experiments I’ll try to move to /opt/.

The numpy install issue is probably because Slicer is currently using pip 20.2.4 and in the pip 20.3 release notes it includes “Add support for MacOS Big Sur compatibility tags.” This might be solved by just updating the pip version.

I see python package hash issues with numpy, scipy and pillow that comes with python-dicom.
If I try to specify the package hashes that I get, I end up with scipy not being able to install because of

numpy.distutils.system_info.NotFoundError: No lapack/blas resources found. Note: Accelerate is no longer supported.

I’ll try to update the version of pip along with some other stuff that it advised in the scipy’s github issue (like installing openblas from brew), but I’m not sure this is the reason for my build to fail.

Just to check, @pieper is the Slicer source code also required to be in a folder with a short path, or only the build dir? If it is this might be my problem.

No, just the build tree. And typically on mac the long path is only a problem at runtime (on windows it can be a compile time issue).

I am about to give up=)
What CMAKE_OSX_DEPLOYMENT_TARGET do you use in your builds?

Also can anyone advice some secret command I can type into the black hole of the terminal to see what exactly caused the make error?=)

I issued COMP: Update python pip to latest by jamesobutler · Pull Request #5393 · Slicer/Slicer · GitHub to attempt to address that pip install issue when using macOS Big Sur.

2 Likes

Thanks everyone for taking the time to investigate :pray: and thanks @jamesobutler for submitting a pull request, it has been integrated :rocket:

Don’t give up!

I use this command:

BUILD_TYPE=Debug

cmake \
	-DCMAKE_BUILD_TYPE:STRING=${BUILD_TYPE} \
	-DCMAKE_OSX_DEPLOYMENT_TARGET=10.15 \
	-DQt5_VERSION:STRING=5.15 \
	-DQt5_DIR:PATH=/usr/local/Cellar/qt/5.15.0/lib/cmake/Qt5 \
	/Users/pieper/slicer/latest/Slicer

I can’t recall if I’ve build from scratch on mac since updating the OS. If you are still having trouble tomorrow I’ll try a fresh build.

Not exactly a secret, but I often do something like:

make -j10 -k; make 2>&1 | tee /tmp/build.log

Just let this run and the -k will build everything it can on the first round, then the second make will stop at the real error. The 2>&1 sends errors to stdout and tee saves a file you can search through but also shows progress in the terminal.

Hey :wave:

Just to follow up on this one. First of all, thanks everyone for looking into this and thank you @jamesobutler for the PR. The upgrade of pip version did solve the issue I had yesterday.

Here’s a small log of my endeavours.

As discussed on the Weekly Hangout the core of the problems might have been that I’ve specified CMAKE_OSX_DEPLOYMENT_TARGET=11.1 while everyone specifies 10.15 or earlier. Not that this is a problem, but this could have popped up later while creating the build for Apple Silicon.

Despite that I’ve succeeded with the build there are a couple things that seem strange.

After my first attempt to build with the latest version of pip I ended up with logs full of errors saying that OpenBLAS could not be found and the app didn’t launch. I understand that OpenBLAS is not a requirement for Slicer, though it’s needed for scipy. So I’ve installed openblas from homebrew and the build went smooth without a single error.
But the Slicer app still didn’t launch. Only a distorted splashscreen and the app freezes after that.

So I decided to run the tests.
When I launched the tests the widgets started popping up as expected so it seemed that Slicer is working. And only after I ran the tests launching the application would produce a distorted splashscreen followed by the UI.

Functionally Slicer is working as expected.

Regarding the distorted splashscreen, I think this has something to do with an HDPI display settings (like the AA_UseHighDpiPixmaps attribute is missing somewhere).

P.S. Some tests failed

Total Test time (real) = 3712.16 sec

The following tests FAILED:
    184 - qMRMLLayoutManagerTest4 (Failed)
    373 - py_MarkupsCurveMeasurementsTest (Failed)
    407 - py_nomainwindow_SegmentationsModuleTest2 (Failed)
    524 - ModelMakerGenerateAllOneLabelTest (Failed)
    647 - py_WebEngine (Failed)
    651 - py_SlicerRestoreSceneViewCrashIssue3445 (Failed)

When you start Slicer, you see two splash images: one rendered immediately by the launcher, then some time later a second one rendered by the Slicer application. Which splash images are scaled incorrectly: the first, the second, or both?

I only see one. It’s rendered immediately, then it goes away and in a couple of seconds I see the UI.

Did you run Slicer.app from within a package? On the mac that will only have one splash screen because there is no launcher for that case.

1 Like

I would’ve probably cleared out any “python-scipy*” directories or project files. Possibly just cleared out all the “python-*” type directories to rebuild the python stuff.

What could be happening is that the original build did not find a Scipy wheel that fit the os platform, and therefore download the Scipy source which might require OpenBLAS as a build requirement. Then even with pip upgraded it might’ve attempted to still build from source which is why I suggest the clearing out of directories. This would make sure that it does not try to build Scipy from source and picks an appropriate wheel.

No, I didn’t package the app