Static build of Qt5 to update the application launcher and support interactive process

Development
1

Short version

Help is needed to do a 32-bit static build of Qt5 on Windows. See link error at the end of this post.

Indeed, building against a static build of Qt4 is not sufficient anymore because the function QProcess::setInputChannelMode added in Qt 5.2 is needed on Windows by the applauncher.

Long version

What is this “interactive support” ?

Thanks to the suggestion of @ihnorton and help from @lassoan, we recently improved the launcher to support interactive process.

It basically makes the following possible:

before

macbook-pro:bin inorton$ ./SlicerPython
                              ---> nothing happens, no display or response
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyboardInterrupt

after:

macbook-pro:bin inorton$ ./SlicerPython
Python 2.7.13 (default, Apr  5 2018, 16:41:55)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

What is the launcher ?

The launcher is statically built executable allowing to prepare the environment before starting the “real” Slicer executable.

The associated source code is maintained at https://github.com/commontk/AppLauncher

How do you build the launcher ?

Each time we integrate changes to the master branch or tag a new release, binaries for Linux, macOS and Windows are automatically uploaded here.

To efficiently support building the launcher statically against a static build of Qt4, we have pre-packaged build tree for the three operating system we support. See https://github.com/jcfr/qt-static-build

We now need to update these and have Qt5 build tree starting with Windows. But we have issue doing so.

Problem building statically Qt5 on windows ?

References:

Step by step

Step 1: Install Perl to C:\Perl64

Step 2: Open Git Bash and download sources

Note: we should probably only download qtbase module

mkdir /c/Qt-static
cd /c/Qt-static
git clone git://github.com/qt/qt5/ --branch v5.11.1 --depth 1 --recursive 5.11.1

Step 3: Create C:\Qt-static\qt5vars.cmd and associated desktop shortcut

Note: we on purpose selected x86 so that we have a 32-bit build tree that will allow to create binaries working on both 32-bit and 64-bit windows install.

REM Set up \Microsoft Visual Studio 2017, where <arch> is \c amd64, \c x86, etc.
REM CALL "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat" x86
CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86
SET _ROOT=C:\Qt-static\5.11.1
SET PATH=%_ROOT%\qtbase\bin;%_ROOT%\gnuwin32\bin;%PATH%
REM Uncomment the below line when using a git checkout of the source repository
SET PATH=%_ROOT%\qtrepotools\bin;%PATH%
SET _ROOT=

SET PATH=C:\Perl64\bin;%PATH%
SET PATH=C:\Python37-x64;%PATH%

In C:\Qt-static\, create a shortcut link named qt5-shell by specifying the command %SystemRoot%\system32\cmd.exe /E:ON /V:ON /k C:\Qt-static\qt5vars.cmd as application and C:\Qt-static as working directory.

Step 4: Download jom

Download http://download.qt.io/official_releases/jom/jom.zip and copy the jom.exe into C:\Qt-static\5.11.1

Step 5: Start terminal using the Desktop link created in a previous step and configure Qt

cd C:\Qt-static\5.11.1
configure.bat -platform win32-msvc -release ^
-nomake examples ^
-nomake tests ^
-opensource ^
-confirm-license ^
-static ^
-qt-zlib ^
-qt-pcre ^
-qt-libpng ^
-qt-libjpeg ^
-qt-freetype ^
-no-ssl ^
-no-opengl ^
-skip qt3d ^
-skip qtactiveqt ^
-skip qtandroidextras ^
-skip qtcanvas3d ^
-skip qtcharts ^
-skip qtconnectivity ^
-skip qtdatavis3d ^
-skip qtdeclarative ^
-skip qtdoc ^
-skip qtdocgallery ^
-skip qtenginio ^
-skip qtfeedback ^
-skip qtgamepad ^
-skip qtgraphicaleffects ^
-skip qtimageformats ^
-skip qtlocation ^
-skip qtmacextras ^
-skip qtmultimedia ^
-skip qtnetworkauth ^
-skip qtpim ^
-skip qtpurchasing ^
-skip qtqa ^
-skip qtquickcontrols ^
-skip qtquickcontrols2 ^
-skip qtremoteobjects ^
-skip qtrepotools ^
-skip qtscript ^
-skip qtscxml ^
-skip qtsensors ^
-skip qtserialbus ^
-skip qtserialport ^
-skip qtspeech ^
-skip qtsvg ^
-skip qtsystems ^
-skip qttools ^
-skip qttranslations ^
-skip qtvirtualkeyboard ^
-skip qtwayland ^
-skip qtwebchannel ^
-skip qtwebengine ^
-skip qtwebglplugin ^
-skip qtwebsockets ^
-skip qtwebview ^
-skip qtwinextras ^
-skip qtx11extras ^
-skip qtxmlpatterns

Step 6: Build

C:\Qt-static\5.11.1> jom

jom 1.1.2 - empower your cores

        cd qtbase\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefile C
:\Qt-static\5.11.1\qtbase\qtbase.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        cd src\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefile C:\Q
t-static\5.11.1\qtbase\src\src.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        cd corelib\ && ( if not exist Makefile.qtzlib C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Ma
kefile.qtzlib C:\Qt-static\5.11.1\qtbase\src\corelib\qtzlib.pro ) && C:\Qt-static\5.11.1\jom.exe -f
Makefile.qtzlib

[...]

        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
        cd sql\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefile C:\Q
t-static\5.11.1\qtbase\src\sql\sql.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
        cd 3rdparty\harfbuzz-ng\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe
 -o Makefile C:\Qt-static\5.11.1\qtbase\src\3rdparty\harfbuzz-ng\harfbuzz-ng.pro ) && C:\Qt-static\5
.11.1\jom.exe -f Makefile
        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
        cd tools\uic\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefil
e C:\Qt-static\5.11.1\qtbase\src\tools\uic\uic.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
        cd network\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefile
C:\Qt-static\5.11.1\qtbase\src\network\network.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
        cd dbus\ && ( if not exist Makefile C:\Qt-static\5.11.1\qtbase\bin\qmake.exe -o Makefile C:\
Qt-static\5.11.1\qtbase\src\dbus\dbus.pro ) && C:\Qt-static\5.11.1\jom.exe -f Makefile
        C:\Qt-static\5.11.1\jom.exe -f Makefile.Release
Qt5Core.lib(qstring.obj) : error LNK2019: unresolved external symbol "public: class QFlags<enum QReg
ularExpression::PatternOption> __thiscall QRegularExpression::patternOptions(void)const " (?patternO
ptions@QRegularExpression@@QBE?AV?$QFlags@W4PatternOption@QRegularExpression@@@@XZ) referenced in fu
nction "public: class QString __thiscall QString::section(class QRegularExpression const &,int,int,c
lass QFlags<enum QString::SectionFlag>)const " (?section@QString@@QBE?AV1@ABVQRegularExpression@@HHV
?$QFlags@W4SectionFlag@QString@@@@@Z)
Qt5Core.lib(qstringlist.obj) : error LNK2001: unresolved external symbol "public: class QFlags<enum
QRegularExpression::PatternOption> __thiscall QRegularExpression::patternOptions(void)const " (?patt
ernOptions@QRegularExpression@@QBE?AV?$QFlags@W4PatternOption@QRegularExpression@@@@XZ)
Qt5Core.lib(qstring.obj) : error LNK2019: unresolved external symbol "public: void __thiscall QRegul
arExpression::setPatternOptions(class QFlags<enum QRegularExpression::PatternOption>)" (?setPatternO
ptions@QRegularExpression@@QAEXV?$QFlags@W4PatternOption@QRegularExpression@@@@@Z) referenced in fun
ction "public: class QString __thiscall QString::section(class QRegularExpression const &,int,int,cl
ass QFlags<enum QString::SectionFlag>)const " (?section@QString@@QBE?AV1@ABVQRegularExpression@@HHV?
$QFlags@W4SectionFlag@QString@@@@@Z)
Qt5Core.lib(qstring.obj) : error LNK2019: unresolved external symbol "public: __thiscall QRegularExp
ression::QRegularExpression(class QRegularExpression const &)" (??0QRegularExpression@@QAE@ABV0@@Z)
referenced in function "public: class QString __thiscall QString::section(class QRegularExpression c
onst &,int,int,class QFlags<enum QString::SectionFlag>)const " (?section@QString@@QBE?AV1@ABVQRegula
rExpression@@HHV?$QFlags@W4SectionFlag@QString@@@@@Z)
[...]

FAQ

How do you create the list of -skip option to give to configure.bat ?

Copy list of modules from https://github.com/qt/qt5 into a text file (e.g /tmp/qt-modules.txt):

 	qt3d @ 3a3fa5d 	Update submodules on '5.11' in qt5 	a day ago
	qtactiveqt @ e977cfd 	Update submodules on '5.11' in qt5 	13 days ago
[...]

then run the following command:

for module in $(cat /tmp/qt-modules.txt  | cut -d"@" -f1 | tr -d "\t" | tr -d " "); do echo "-skip $module ^"; done

In the command we used, we removed -skip qtbase

1 Like
2

Have you tried to build with default options (not skipping any packages)? I’ve noticed that you skipped pcre package and you have an unresolved symbol related to regular expressions - maybe they are related.

1 Like
3

I have built static Qt in the past, I don’t remember it being much different from regular build.

As @lassoan pointed out, you are skipping a whole lot of components. To check whether it is disabling of components or the static build that is causing the error, you could try the same build command except you now remove the -static option.

Edit: it might also be a consequence of using parallel build tool, which might not honor all inter-module dependencies. I remember running into problems building Qt with /MP option.

4

Have you tried to build with default options (not skipping any packages)?

You actually will have to skip some modules like QtWebEngine if python 3 is being used in the build process as indicated.

SET PATH=C:\Python37-x64;%PATH%

QtWebengine requires python >2.7.5 but not python 3. Unsure if there are any other modules like that.
Plus static builds aren’t supported either.

1 Like
5

Could we just use a static pre-built Qt, such as this one?

6

That looks great.

And it seems there are no script to build Qt 5.11 in static with msvc, only Qt 5.10.1 … that is probably why it failed.

Would appreciate if someone could test this script: https://github.com/martinrotter/qt5-minimalistic-builds/blob/master/msvc-build5101-static.ps1 (ideally with VS2015 … which is the version we have available on the factory)

7

I started the script but the first issue it runs into is SSL/TLS secure download. I’m investigating

8

Update: Had to add

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

before line https://github.com/martinrotter/qt5-minimalistic-builds/blob/master/msvc-build5101-static.ps1#L26
Now the download is running.

9

Nice.

In scikit-ci-addons, we set the following:

[System.Net.ServicePointManager]::SecurityProtocol = 3072 -bor 768 -bor 192 -bor 48

See https://scikit-ci-addons.readthedocs.io/en/latest/addons.html#addressing-the-underlying-connection-was-closed-error

10

The security protocol you mention also works here.

Apparently the script doesn’t run with VS2015. @lassoan has VS2017, he’s running it now.

11

Thanks for the help.

I also forgot to mention:

  • the build should be done in a 32-bit console. That way we can use the libraries to build a launcher working for both platform and easily support 32-bit tablets.
  • since usually the archived build tree is not relocatable, the build directory should be something generic enough. Here is what I used for the last build tree C:\D\Support\qt-static-release-i386-4.8.6

After the build is done, it would be ideal to create a release like this one: https://github.com/jcfr/qt-static-build/releases/tag/4.8.6-vs2008

12

There have been some issues with the build:

  • need to apply Qt patch due to this bug that affects latest
    VS2017 with older Qt versions => this patch fixed it
  • OpenSSL was not detected (even though it was installed at the specified location) => removing -openssl -openssl-linked flags from msvc-build5101-static.ps1 fixed it

The build is still in progress.

Note that pre-built static binaries are already available, although it is 64-bit only.

13

need to apply Qt patch

Good catch

OpenSSL was not detected

We don’t need OpenSSL for the launcher. simpler is better

Note that pre-built static binaries are already available, although it is 64-bit only

Considering we use these launcher for commercial projects, the associated library binaries need to come from a trusted and known party.

Thanks for the help

14

I’ve managed to build static Qt 5.10.1 and use it to build a launcher (64-bit, VS2017 toolset) with a couple of changes to Qt build and AppLauncher. There were many test errors but Slicer started successfully using this launcher. I haven’t tried to build a 32-bit launcher. It would be great if somebody could take over from here (reproduce the build, determine what is causing test failures, replacing hardcoded paths in CMakeLists.txt, etc).

15

I’ve uploaded build script and additional patch for building 32-bit static Qt5 launcher using MSVC 2017.

App launcher build completes and the launcher seemingly works but there are many test errors. Maybe there are issues in the test application.

@jcfr Can you take it over from here? I’m happy to help out but don’t have time to properly work out all the details.

16

Will do. Thanks a lot for your help, this was very helpful :+1:

17

To follow up on this, updated launcher is available at https://github.com/commontk/AppLauncher/releases/tag/v0.1.25

Script used to generate the 32-bit Qt5 static build used for building the launcher is here: https://github.com/jcfr/qt5-minimalistic-builds/compare/master...jcfr:add-msvc-build5112-static-x86-applauncher

Associated archive is available at https://github.com/jcfr/qt-static-build/releases/tag/applauncher-5.11.2-vs2017

Finally, the set of flags and list of libraries to specify to ensure the launcher is statically built and the universal C runtime and the C++ standard library are included in the executable is here:

Useful references are:

3 Likes