Of course everything is doable in all computing environments, the question is really just the amount of effort needed to develop and maintain it. Creating a decent 4D volume rendering shader alone may probably take several months (maybe much more, if you want to have similar quality and performance as the one in Slicer, with corrent interaction with non-volume-rendered elements, interaction with the volume, etc.). Implementing slice views that you can conveniently browse, basic annotations (points, lines, curves), segmentation tools, adjust visualization options in immersive view, … each may take several months to develop and much more to refine and continuous effort to maintain (user support, bugfixes, platform updates, etc.).
So, while everything is doable, it is probably not a good candidate for a successful one-man-project, because by the time you finish, you already run out of time/funding and/or others have implemented something that is much better than what you have.
I can see a few approaches that could make sense:
- Implement a minimal viewer in Unity (reimplementing as few features as possible) and leverage as much as possible what Slicer provides. You can do all your data import and pre-processing in Slicer, have all the easy-to-use desktop widgets (buttons, sliders, etc.) for going through complex workflows and setting advanced options. We currently use this approach for some of our HoloLens projects, for example this pedicle screw planner prototype. This project is very simple, very small amount of code, no volume rendering, just a single slice for reformatting, rendered and sent in real-time to Unity by Slicer. Transforms are also synchronized between Unity and Slicer.
- Join one of the large projects that reimplement medical applications in Unity. I don’t know about open-source projects but there are a number for companies in this space. They work mostly with HoloLens, but some of them also use virtual reality. Unity is not a good platform for medical application development, as lots of things are missing and needs and priorities of game developers are very different from medical device developers, but if enough resources are poured into a project like this then eventually something usable gets out from it.