All choices that the user makes on the GUI are saved in the fiducial registration wizard parameter node that you select at the top (by default, its name is “FiducialRegistrationWizard”). You can also change these selections form Python by modifying the node, for example, you can switch to similarity mode and change “Place ‘To’” transform like this:
w=getNode('FiducialRegistrationWizard')
w.SetRegistrationModeToSimilarity()
w.SetProbeTransformToNodeId(somTransform.GetID())