I’ve come across a serious limitation in our current database structure: the frame set (see also here) level is completely missing. We treat series as displayable set of frames, which is wrong for image modalities such as ultrasound and not always optimal for fluoroscopy and in many other cases. It is common to have dozens of ultrasound acquisitions in a single series that we currently cannot browse. We can switch to advanced mode and browser loadables (results of “Examine” step) but that’s very limited (e.g., no thumbnails) and slow.
I’m wondering if we should add a “FrameSet” table to the database. We would store each loadables item in a table, provide thumbnail and metadata for those and not for each series.
We can discuss this, but I agree we could implement a similar/enhanced UI of the default browser in the visual one with thumbnails.
Probably for the user knowing the concept of FrameSet is not important, so we could just provide a simple UI with thumbnails of the selected series where each new thumbnails/element comes either from:
differences in geometry of the instances, … etc… (i.e. the currrent examine operation)
I’m only thinking about improving the visual browser, as I expect that it can replace the old browser in the future.
Examine step provides the frame sets, so they are not two different things. I would not spend time with the old browser, we can leave it as is (show the list of series and generate framesets on-the-fly).
In the visual browser, the thumbnail list should always show the frameset list to the user, never the series list, so there is no need to ask the user. For this, we probably need a FrameSets table and the visual browser would display records of this table instead of the series table. The table would be populated by running “examine” for each imported (or updated) study automatically. We could store output of all plugins in the framelists table, but by default we would only show the results provided by the plugin with the highest confidence. We could add an option to the right-click menu of the thumbnail to choose outputs from a different plugin.
Yes, I think the browser should expose the logical structure of the acquisition as captured in the dicom data and support previews that make sense for the modality and use case. I think you should do what is needed for the visual browser.
Longer term we should rethink the database entirely, and define a new API that better refects framesets and other DICOM concepts that aren’t currently well represented.
In my experiments I found that the dicomDatabase can be completely replaced by a DICOMweb client so we should factor the database out from the plugins (which currently treat it as a global singleton). In this branch I made the database an optional argument for the ScalarVolumePlugin, and I think we can go a lot further to clean things up. This could be a good deliverable for some funding proposal.
the UI probably will need only minor refactoring (we could simply convert the series widget into a Frame sets one)
however for the logic (Examine step, which would be run as soon as we fetch the series and instances metadata), we would need some refactoring and probably move (at least part of) the plugins logic into CTK (C++).
Yes, the logic part is the difficult one. Unfortunately, it is not an option to move all the plugin logic to C++ because many people who have the domain knowledge to create plugins will provide it in their language of preference (now mostly Python, but soon also Rust, etc.).
It should not be a big issue, except we need to find a robust and fast way of running these plugins in parallel in the background. Multithreading in Python is rather limited, so we may need to run Python plugins in separate processes.
We will have to rework the plugin/database interface anyway to be able to run the plugins in the background. I’m not sure if DICOMweb REST API is optimal or sufficient, as we still need to store additional data, such as framesets. There might be some overhead due to HTTP and fully text-based communication, which might be perceivable when inspecting geometry of large series, such as high-resolution 4D CTs. But the API could be perhaps very similar to DICOMweb so that it would be trivial to use it with a DICOMweb server.
Yes, the database API should not assume DICOMweb, but it also should not assume DIMSE or filesystem either, but rather expose the metadata in a clean and efficient way, possibly pre-cached and possibly fetched on the fly.
Exposing the plugins to the C++ layer should not mean migrating the code to C++, it should mean providing a clean abstract C++ class that can be specialized to use the plugins in whatever language they come in. Running in threads or processes would be options to explore.