@randomizedid C++ code needs to be compiled and this cannot be avoided. qt-loadable-modules usually are not that time consuming when compiling them, if you can structure your development in this way you will experience shorter compilation times.
If you cannot structure your development in a qt-loadable-module, you can still speed up the building process. The Slicer building system is structured in many build targets. The default compilation is all, which will try to build everything (in practice, this will go through the files and check for changes, so it is not as time consuming as a fresh compilation…still, it takes a while). However, you can build a specific target (typically that part of Slicer where you have made changes; this should already avoid building code that is not related). There is even a /fast version of targets, which I believe won’t resolve dependencies (this can be risky if you don’t know what you are doing). Finally, you can use ccache (Speeding up Slicer builds with CCache and ccache tip in GNU/Linux systems — 3D Slicer documentation). I’m not sure ccache is available for Windows and MAC, but it is possible to use it on GNU/Linux systems.