Long path using ninja

Development
1

Configuring Slicer with ninja as the make program, and VS2015 cl.exe as compiler, I run into the error below. This doesn’t happen with pure VS2015 build in one letter longer build dir (C:/Dev/Sn/ vs C:/Dev/Svs/). Is ninja merely more careful, or is this specific to ninja?

CMake Warning in Modules/Loadable/VolumeRendering/SubjectHierarchyPlugins/CMakeLists.txt:
  The object file directory

    C:/Dev/Sn/Slicer-build/Modules/Loadable/VolumeRendering/SubjectHierarchyPlugins/CMakeFiles/qSlicerVolumeRenderingSubjectHierarchyPluginsPythonQt.dir/

  has 149 characters.  The maximum full path to an object file is 250
  characters (see CMAKE_OBJECT_PATH_MAX).  Object file

    generated_cpp/osm_qSlicerVolumeRenderingSubjectHierarchyPlugins/osm_qSlicerVolumeRenderingSubjectHierarchyPlugins_module_init.cpp.obj

  cannot be safely placed under this directory.  The build may not work
  correctly.
2

This is a warning only. Probably you just haven’t noticed it when you’ve built Slicer with the default generator.

3

On a related note, has anyone tried enabling the new long path behavior in Windows 10?

From Naming Files, Paths, and Namespaces - Win32 apps | Microsoft Learn

Starting in Windows 10, version 1607, MAX_PATH limitations have been removed from common Win32 file and directory functions. However, you must opt-in to the new behavior.
…
To enable long path behavior set the registry key at HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD).

1 Like
4

The build stopped soon afterwards. I am now re-running ninja to see what happens.

I could try it at home!

5

After re-running ninja, I get the same error:

-- Configuring done
CMake Warning in Modules/Loadable/VolumeRendering/SubjectHierarchyPlugins/CMakeLists.txt:
  The object file directory

    C:/Dev/Sn/Slicer-build/Modules/Loadable/VolumeRendering/SubjectHierarchyPlugins/CMakeFiles/qSlicerVolumeRenderingSubjectHierarchyPluginsPythonQt.dir/

  has 149 characters.  The maximum full path to an object file is 250
  characters (see CMAKE_OBJECT_PATH_MAX).  Object file

    generated_cpp/osm_qSlicerVolumeRenderingSubjectHierarchyPlugins/osm_qSlicerVolumeRenderingSubjectHierarchyPlugins_module_init.cpp.obj

  cannot be safely placed under this directory.  The build may not work
  correctly.


-- Generating done
-- Build files have been written to: C:/Dev/Sn/Slicer-build
[214/217] Performing build step for 'Slicer'
FAILED: Slicer-prefix/src/Slicer-stamp/Slicer-build
cmd.exe /C "cd /D C:\Dev\Sn\Slicer-build && "C:\Program Files\CMake\bin\cmake.exe" --build . && "C:\Program Files\CMake\bin\cmake.exe" -E touch C:/Dev/Sn/Slicer-prefix/src/Slicer-stamp/Slicer-build"
ninja: build stopped: subcommand failed.

C:\Dev\Sn>
6

Trying to build Slicer in a long path build directory, the build fails when it comes turn to configure ITK. Then it snowballs from there. I guess we need to add extra logic to this check to handle the possibility of longer paths which now exists in Windows.

50>  CMake Error at CMakeLists.txt:30 (message):
50>    ITK source code directory path length is too long (65 > 50).Please move the
50>    ITK source code directory to a directory with a shorter path.
50>
50>
50>  -- Configuring incomplete, errors occurred!
50>  See also "C:/Dev/Slicer-buildVS2015x64-super-long-path-without-spaces/ITKv4-build/CMakeFiles/CMakeOutput.log".
7

It’ll take several years until all build tools will be updated to take advantage of new file path lengths. Also note that often long paths cause build failures by making environment variables too long.

It would be easier to limit file and directory name lengths in VTK and ITK. File path length issues popped up when modularization was introduced in VTK and ITK: it added one more level of directory for all files and increased number of libraries by a factor of 10x - which threw off Visual Studio.

8

Refactoring has already been done, and there is no going back. The support for longer max path was just added to ITK.

9

Modularization is good, I’m not suggesting to going back, but it is implemented in a way that it puts significant burden on build systems. Hopefully, ITK&VTK build system and/or Visual Studio will be improved at some point, because in large projects like Slicer we are near the limit of having a feasible build.

1 Like
10

VTK developers plan to overhaul their build system. Once they are done, we plan to plagiarize that for ITK.

11

Do you mind to link to the discussion, if available?

12

I wanted to, but I don’t remember where I saw that.

1 Like
13

We could make various changes to use the modularized build more effectively (e.g. not passing the full paths to every single library). But we effectively dump all VTK_LIBRARIES to the linker anyway, so modularization was kind of a no-op and comes at the cost of startup time. Processing >600 dynamic libraries is really slow on mac (and to a lesser degree linux): a debug build takes close to 1 min to start on mac, the majority of which is in strcmp during symbol relocations. Release build is somewhat faster of course, but still spends most time doing relocation.

(Apple’s code dump for 10.13 shows big changes in dyld including some files about “caching”, so maybe this improves in High Sierra – but I still haven’t upgraded)

14

Just to clarify, that comment was intended as an observation rather than a complaint per se :slight_smile:. There is enough other low-hanging fruit to go after next time we do build system optimization, and in the worst case we can relink VTK and ITK into single libraries ourself if it is demonstrably faster.

2 Likes