Scene Views feature have had lots of instability issues over the years. Now we have a good plan how to fix all that, but we’ll only do this rework after Slicer5 is released.
Until then, I would recommend to use Sequences to store state of selected nodes - such as the camera, maybe volume rendering properties. See a short tutorial here: