Issue Loading TensorFlow .h5 Model in 3D Slicer – "Unknown Layer: 'TensorFlowOpLayer

Support
1

Hi everyone,

I’m currently working on integrating an acute stroke detection model (originally trained in TensorFlow and saved as a .h5 file) into the 3D Slicer environment.

I tried running it within my Slicer module, but I’m getting the following error:
Processing failed: Error loading lesion model: Unknown layer: ‘TensorFlowOpLayer’. Please ensure you are using a keras.utils.custom_object_scope and that this object is included in the scope.
See: Save, serialize, and export models  |  TensorFlow Core
…
ValueError: Unknown layer: ‘TensorFlowOpLayer’
We only have the .h5 file, and unfortunately, we don’t have access to the model’s architecture code. I’ve looked into converting it to PyTorch, but as far as I know, it’s not feasible with just the .h5 file alone.

My Questions:

  • Is this a limitation of 3D Slicer’s Python environment (maybe TensorFlow isn’t fully supported)?
  • Or is it strictly a TensorFlow model serialization issue, requiring access to the original custom layer definitions?
  • Is there a workaround for this inside Slicer (e.g., registering custom layers or running externally)?

Any help or guidance would be really appreciated. Thanks in advance!

2

That sounds like a tensorflow version issue.

1 Like
3

Sir, is it possible to use a TensorFlow .h5 model in 3D Slicer?

4

An h5 file is just a container, like a zip file and tensorflow is a utility, so you can use both in Slicer but you need to know the software context to know if you can use it or not. I’m not a tensorflow expert, but the “unknown layer” looks like something that would happen if you have a version mis-match between the creator or the h5 file and the version of tensorflow you are tying to use.

These days people mostly use PyTorch for medical imaging applications, so probably you can study the original design and other example models and port it and re-train.

1 Like
5

Tensorflow is notoriously difficult to get it installed and working even in a normal python environment. That’s probably the number one reason people migrate to pytorch even though tf has been around longer and probably was more mature at the time.

Last I checked, tf worked fine in Slicer. But I would suggest trying to make it work in normal python first to rule out other issues and then try in Slicer. Having said that, I agree with @pieper. i don’t think this is an installation issue, but a model/version issue.

You might have better support if you ask this on tensorflow forum.

1 Like
6

The pretrained model I’m using is from the open-source GitHub repo Chin-Fu-Liu / Acute-stroke_Detection_Segmentation. According to the requirements, it’s built using TensorFlow 2.0.0 with Python 3.7.

I’ve set up a virtual environment in VS Code with the exact same dependencies:
pip install tensorflow==2.0.0
pip install tensorflow-gpu==2.0.0
pip install numpy nibabel scipy scikit-image scikit-learn
pip install dipy==1.4.0
pip install h5py==2.10.0
:white_check_mark: With this setup, the model loads and runs perfectly fine.
So now I’m wondering — is this issue specific to Slicer’s Python environment, or do I need to somehow register custom objects manually in the code?

Any suggestions on how to align Slicer’s environment to match the one where it’s working would be a huge help. :folded_hands:

7

I would suggest repeating this effort outside of Slicer, but using Python 3.9. This version of Python is what Slicer is currently using so if you can get ADS working with it outside of Slicer using Python 3.9 then you can then possibly get it working within Slicer’s environment.

1 Like
8

Sir, I have tested the model in a new environment with Python 3.9, and it is working fine outside of Slicer in a regular Python environment. The following requirements were used:
numpy==1.19.5
scipy==1.7.3
h5py==3.1.0
nibabel==3.2.1
tensorflow==2.7.0
scikit-image==0.18.1
scikit-learn==0.24.1
dipy==1.5.0

9

Based on the original error you ran into, I think it is likely tensorflow being an issue in Slicer. However, I don’t think many in this community use Tensorflow as it is almost exclusively PyTorch usage. You could try converting the keras h5 model to an onnx object with something like GitHub - onnx/tensorflow-onnx: Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX.

Based on the specific ADS model being GPL, there likely won’t be a lot of additional support here based on the type of discussion at:

10

ok. Try installing these packages using the exact versions in Slicer. Does that work?

11

For example on my mac with Slicer 5.8.1, tensorflow==2.7.0 supposedly installs, but then bring this error message when I import it:

>> pip_install('tensorflow==2.7.0')
Collecting tensorflow==2.7.0
  Downloading tensorflow-2.7.0-cp39-cp39-macosx_10_11_x86_64.whl.metadata (2.9 kB)
Requirement already satisfied: numpy>=1.14.5 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorflow==2.7.0) (1.26.4)
Collecting absl-py>=0.4.0 (from tensorflow==2.7.0)
  Downloading absl_py-2.2.2-py3-none-any.whl.metadata (2.6 kB)
Collecting astunparse>=1.6.0 (from tensorflow==2.7.0)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting libclang>=9.0.1 (from tensorflow==2.7.0)
  Downloading libclang-18.1.1-py2.py3-none-macosx_10_9_x86_64.whl.metadata (5.2 kB)
Collecting flatbuffers<3.0,>=1.12 (from tensorflow==2.7.0)
  Downloading flatbuffers-2.0.7-py2.py3-none-any.whl.metadata (872 bytes)
Collecting google-pasta>=0.1.1 (from tensorflow==2.7.0)
  Downloading google_pasta-0.2.0-py3-none-any.whl.metadata (814 bytes)
Collecting h5py>=2.9.0 (from tensorflow==2.7.0)
  Downloading h5py-3.13.0-cp39-cp39-macosx_10_9_x86_64.whl.metadata (2.5 kB)
Collecting keras-preprocessing>=1.1.1 (from tensorflow==2.7.0)
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl.metadata (1.9 kB)
Collecting opt-einsum>=2.3.2 (from tensorflow==2.7.0)
  Downloading opt_einsum-3.4.0-py3-none-any.whl.metadata (6.3 kB)
Collecting protobuf>=3.9.2 (from tensorflow==2.7.0)
  Downloading protobuf-6.30.2-cp39-abi3-macosx_10_9_universal2.whl.metadata (593 bytes)
Requirement already satisfied: six>=1.12.0 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorflow==2.7.0) (1.16.0)
Collecting termcolor>=1.1.0 (from tensorflow==2.7.0)
  Downloading termcolor-3.0.1-py3-none-any.whl.metadata (6.1 kB)
Requirement already satisfied: typing-extensions>=3.6.6 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorflow==2.7.0) (4.12.1)
Requirement already satisfied: wheel<1.0,>=0.32.0 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorflow==2.7.0) (0.43.0)
Requirement already satisfied: wrapt>=1.11.0 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorflow==2.7.0) (1.16.0)
Collecting gast<0.5.0,>=0.2.1 (from tensorflow==2.7.0)
  Downloading gast-0.4.0-py3-none-any.whl.metadata (1.1 kB)
Collecting tensorboard~=2.6 (from tensorflow==2.7.0)
  Downloading tensorboard-2.19.0-py3-none-any.whl.metadata (1.8 kB)
Collecting tensorflow-estimator<2.8,~=2.7.0rc0 (from tensorflow==2.7.0)
  Downloading tensorflow_estimator-2.7.0-py2.py3-none-any.whl.metadata (1.2 kB)
Collecting keras<2.8,>=2.7.0rc0 (from tensorflow==2.7.0)
  Downloading keras-2.7.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting tensorflow-io-gcs-filesystem>=0.21.0 (from tensorflow==2.7.0)
  Downloading tensorflow_io_gcs_filesystem-0.37.1-cp39-cp39-macosx_10_14_x86_64.whl.metadata (14 kB)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.7.0)
  Downloading grpcio-1.71.0-cp39-cp39-macosx_10_14_universal2.whl.metadata (3.8 kB)
Collecting markdown>=2.6.8 (from tensorboard~=2.6->tensorflow==2.7.0)
  Downloading markdown-3.8-py3-none-any.whl.metadata (5.1 kB)
Requirement already satisfied: packaging in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorboard~=2.6->tensorflow==2.7.0) (24.0)
Requirement already satisfied: setuptools>=41.0.0 in /Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages (from tensorboard~=2.6->tensorflow==2.7.0) (70.0.0)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard~=2.6->tensorflow==2.7.0)
  Downloading tensorboard_data_server-0.7.2-py3-none-macosx_10_9_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard~=2.6->tensorflow==2.7.0)
  Downloading werkzeug-3.1.3-py3-none-any.whl.metadata (3.7 kB)
Collecting importlib-metadata>=4.4 (from markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.7.0)
  Downloading importlib_metadata-8.6.1-py3-none-any.whl.metadata (4.7 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard~=2.6->tensorflow==2.7.0)
  Downloading MarkupSafe-3.0.2-cp39-cp39-macosx_10_9_universal2.whl.metadata (4.0 kB)
Collecting zipp>=3.20 (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.7.0)
  Using cached zipp-3.21.0-py3-none-any.whl.metadata (3.7 kB)
Downloading tensorflow-2.7.0-cp39-cp39-macosx_10_11_x86_64.whl (207.1 MB)

Downloading absl_py-2.2.2-py3-none-any.whl (135 kB)

Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Downloading flatbuffers-2.0.7-py2.py3-none-any.whl (26 kB)
Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)

Downloading grpcio-1.71.0-cp39-cp39-macosx_10_14_universal2.whl (11.3 MB)

Downloading h5py-3.13.0-cp39-cp39-macosx_10_9_x86_64.whl (3.4 MB)

Downloading keras-2.7.0-py2.py3-none-any.whl (1.3 MB)

Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)

Downloading libclang-18.1.1-py2.py3-none-macosx_10_9_x86_64.whl (26.5 MB)

Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)

Downloading protobuf-6.30.2-cp39-abi3-macosx_10_9_universal2.whl (417 kB)

Downloading tensorboard-2.19.0-py3-none-any.whl (5.5 MB)

Downloading tensorflow_estimator-2.7.0-py2.py3-none-any.whl (463 kB)

Downloading tensorflow_io_gcs_filesystem-0.37.1-cp39-cp39-macosx_10_14_x86_64.whl (2.5 MB)

Downloading termcolor-3.0.1-py3-none-any.whl (7.2 kB)
Downloading markdown-3.8-py3-none-any.whl (106 kB)

Downloading tensorboard_data_server-0.7.2-py3-none-macosx_10_9_x86_64.whl (4.8 MB)

Downloading werkzeug-3.1.3-py3-none-any.whl (224 kB)

Downloading importlib_metadata-8.6.1-py3-none-any.whl (26 kB)
Downloading MarkupSafe-3.0.2-cp39-cp39-macosx_10_9_universal2.whl (14 kB)
Using cached zipp-3.21.0-py3-none-any.whl (9.6 kB)
Installing collected packages: tensorflow-estimator, libclang, keras, flatbuffers, zipp, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, protobuf, opt-einsum, MarkupSafe, keras-preprocessing, h5py, grpcio, google-pasta, gast, astunparse, absl-py, werkzeug, importlib-metadata, markdown, tensorboard, tensorflow
Successfully installed MarkupSafe-3.0.2 absl-py-2.2.2 astunparse-1.6.3 flatbuffers-2.0.7 gast-0.4.0 google-pasta-0.2.0 grpcio-1.71.0 h5py-3.13.0 importlib-metadata-8.6.1 keras-2.7.0 keras-preprocessing-1.1.2 libclang-18.1.1 markdown-3.8 opt-einsum-3.4.0 protobuf-6.30.2 tensorboard-2.19.0 tensorboard-data-server-0.7.2 tensorflow-2.7.0 tensorflow-estimator-2.7.0 tensorflow-io-gcs-filesystem-0.37.1 termcolor-3.0.1 werkzeug-3.1.3 zipp-3.21.0
>>> import tensorflow as tf
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 41, in <module>
    from tensorflow.python.eager import context
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/python/eager/context.py", line 33, in <module>
    from tensorflow.core.framework import function_pb2
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 621, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

You will will have to work one by one until you can find the right combination of packages that will work with tensorflow 2.7.0, which is a fairly old version…

12

try to install protobuf 3.20.0