3-component RGB images appear correctly in slice views and the slices can be displayed in 3D view, too.
4-component RGBA images are required for color volume rendering but RGBA volumes, but slice views of such volumes cannot be displayed in 3D view - see this issue. If you want this to be fixed then add a vote for it. If an issue gets many quotes (or needed for a funded project) then we can allocate developers to fix it. If you need it sooner and you are comfortable with C++ and the VTK library then then you can give it a go and fix it (we can help if you decide to try it). You can also submit a feature request for volume rendering to support RGB volumes (without the need for manually extending it to RGBA).