Why does the color of the label map change when I save the label map?

Segmentation is stored in a 4D nrrd file with a couple of custom fields to store additional metadata - see detailed description here.

You can use any nrrd reader that is capable to read 4D files. If your software cannot read such files then you can export to 3D labelmap (nrrd file) + color table (text file) and read those.