Do you like this tool better than DVTk? (DICOM Validation Toolkit - www.dvtk.org)
Also, do you know where I can find valid DICOM file containing Encapsulated format Pixel Data? Is there a site where we can access standard DICOM data with which we can compare our data?
Thanks
I’ve never used dvtk, thanks for pointing it out.
dciodvfy is written by David Clunie, editor of the dicom standard, so it’s usually a pretty rigorous and informative test. In the end if your goal is to make things interoperable you probably want to use anything and everything you can get your hands on to make sure the data you are generating is as standards compliant as possible. This is usually the best way to avoid surprises down the line.
I wish there were a definitive collection of valid example dicom objects demonstrating the full range of legal variants, but I’ve never found such a collection. Best I can suggest is the TCIA collections (or the subset hosted through IDC).
Good Morning,
I’m confused about how to interpret the log files. Here’s an example of the heap logs…
[DEBUG][Qt] 28.07.2021 16:12:06 [] (unknown:0) - “DICOM indexer has successfully inserted 929 files [0.56s]”
[DEBUG][Qt] 28.07.2021 16:12:06 [] (unknown:0) - “DICOM indexer has successfully processed 929 files [2.71s]”
[DEBUG][Qt] 28.07.2021 16:12:06 [] (unknown:0) - "DICOM indexer has updated display fields for 929 files [0.22s]"
[INFO][Python] 28.07.2021 16:15:17 [Python] (C:/Program Files/Mayo Foundation/QREADS_3D_MPR/bin/../lib/SlicerQReads-4.13/qt-scripted-modules/DICOMScalarVolumePlugin.py:384) - Loading with imageIOName: GDCM
[INFO][Stream] 28.07.2021 16:15:17 [] (unknown:0) - Loading with imageIOName: GDCM
[INFO][Stream] 28.07.2021 16:15:42 [] (unknown:0) - default Window / level = 1673.0 / -187.5
You can see a 3 minute difference between the 1st and 2nd lines. There are 23,000 lines in this log file and during the whole three minutes we can see the log file growing, and when it stops at around 3000KB, the app displays the images. Yet in the smaller Stack logs the time between the 1st and second lines in about 25 seconds. Is all of that time spent writing out the log file?
Thanks
Yes, if you are really getting 23,000 lines of output then the logging system is likely to get choked. It would be good if slicer’s logging was faster, but it’s still better to fix the dicom instead.
Corrected the main cause that produced the most log lines. But now it’s putting out thousands of these…
[CRITICAL][Stream] 29.07.2021 17:40:48 [] (unknown:0) - W: DcmMetaInfo: No Group Length available in Meta Information Header
[CRITICAL][Stream] 29.07.2021 17:40:48 [] (unknown:0) - W: DcmItem: Length of element (7fe0,0010) is not a multiple of 2 (VR=OW)
I really wish there was a way to turn of logging or control the severity. It’s really affecting my load times. Minutes to load 925 images. The Docs will just stop using 3D MPR.
Are you sure there’s no way to turn it off? Should I be able to find the code to shut it off? I know you don’t recommend it, but is the less of the 2 evils because tomorrow is my dev cutoff day.
Thanks!
Since you are building from source, you have the option to change dcmtk’s behavior here. If you can’t find a way to change the logging, you can search for the substrings of the error message and comment out the messages you don’t want (again, it’s not what I’d recommend, but it would be a workaround).
OK, these files have no “0002,0000) - File Meta Information Group Length” elements. I’ll try to insert it. That may get rid of the “No Group Length available in Meta Information Header” warning.
But, do you allow the length of the Pixel data to be -1 or Undefined as listed here?..
Here is our [7FE0, 0010] pixel data tags for Implicit VR Little Endian encapsulated pixel data…
00008a66 ( 35430) (7FE0,0010) - Pixel Data [OW][UNDEFN][ 1]:
00008a72 ( 35442) (FFFE,E000) - Item [][ 4][ 1]:
00008a7e ( 35454) (FFFE,E000) - Item [][183848][ 0]:
000358ae ( 219310) |--->(FFFE,E0DD) - Sequence Delimitation Item [][ 0][ 0]:
Is the “UNDEFN” is actually -1 which is not a multiple of 2 and it’s legal DICOM I believe. Is this why the “Length of element (7fe0,0010) is not a multiple of 2” problem? Do you think it’s a Slicer bug?
Thanks
I believe all those warnings originate from DCMTK so it’s really not Slicer that is in control of determining the logic. Slicer is just reporting the warnings of DCMTK.
OK thanks a lot. I wonder if there is a way to configure DCMTK to control logging. I’ll look into it. Thanks for all of your help.
I will look for the code that generates the logs. Any hints would be greatly appreciated. Thanks. 
I believe you are using a SlicerCustomApplicationTemplate, which I haven’t used myself fir this but you should have a way to access the superbuild cmake script for DCMTK corresponding to this one for Slicer itself. DCMTK uses log4cpp and probably the configuration variables for that are exposed in CMake, but you’ll need to do some research to figure out what options there are for turning off warning messages. HTH.
I see absolutely nothing in there that gives the slightest hint of controlling error or warning messages.
Are the logs written out by ITK? I found C:.…\TK\Modules\IO\DCMTK\src\itkDCMTKFileReader.cxx that seems to deal with the reading of DCMs, But I can’t find where it’s actually writing to the log files. Are you familiar enough in this space to know what Slicer Code Repository search string I can use to find the code that writes to the log files?
Thank you.
Most of the messages you copied here come from dcmtk. For Slicer builds we use this fork of dcmtk:
You can search for the warning messages from the log file and see what code is generating the messages.
But again, from what I can see these indicate errors in the format of the data. It sounds from your messages that you have control over the way these files are written and that is the correct place to fix them. Do you not agree?
You can control log level in DCMTK, in CTK, and in Slicer and we could add some convenient and easier-to-discover API for changing log levels at runtime.
However, these are invalid DICOM files, so suppressing logs does not sound like a good idea. You want to be able to see warnings and report the issues to developers of the software that created the invalid files (in more severe cases, even report to users to make them aware that the data may be corrupted).
What DICOM toolkit do you use that create pixel data with undefined length? That’s a major issue, because many DICOM toolkit may simply refuse to load such data sets.
As a corrective action, I would recommend fixing the DICOM files. As a preventive action, we could try to improve speed of logging in Slicer.
Log files are already rotated. The number of log files preserved is set to 15 or so, and you can configure the number in the Slicer application configuration file. Cutting off a log file in the middle of a session makes it harder to collect complete log files from customers and make it harder for developers to analyze the logs, so I would not recommend doing it.
OK, yes. I think you’re right.
After further review …
Maybe, (7FE0,0010) - Pixel Data [OW][UNDEFN][ 1] should be of type OB rather than OW, and maybe that’s why it’s flagged as not being a multiple of 2 bytes.
And yes, GEL (0002, 0000) is type 1.
We should try to fix this first.
Thanks!
@lassoan @pieper @jamesobutler\
Hi. Please forgive me for so many questions, but we are preparing for this major release of QREADS with the SlicerQREADS extension. For controlling the number of log files and the temporary database folders, we will use our QREADS launcher code to delete older files.
The question was asked by my lead of whether there is a way to set the path of logging folder different from C:\Users\userID\AppData\Local\Temp\SlicerQReads.
Thanks
Changing the type to OB fixed that error.
Now trying to figure out how to insert the mandatory (0002, 0000) into the header, and that should fix that error.
There is no need to manage the log files externally. You can set the number of log files to 1 if you only want to keep the log file of the last session.
The log file location is in the temporary folder specified by TemporaryPath in Slicer.ini. If needed, it would be easy to add an optional settings value to specify a custom location.
Thanks, how about the Temporary database folders? They are not so temporary. They don’t seem to be ever deleted.
Thanks