Loading a deep learning model to a scripted module

Your approach sounds reasonable to me. You may see compatibility issues between Slicer’s python and some generally available python packages. There’s more info in this thread.