I'm pretty sure problems other than color blindness could cause differences in color perception. For instance, my grandmother commented on improvements in color perception after she had a cataract removal operation (this is easy to understand as the removed crystalline lens did exhibit some yellowing). I would guess other issues might be possible. I imagine if people had different densities of normal cones (not color blind) in their retinas would have to exhibit different levels of color discrimination (though I have absolutely no idea how significant a factor this is in real life). Also, I would guess that one would expect people with deficits in their central vision (macular degeneration and that sort of a thing) to lose some or most of their ability to discriminate colors simply because of the fact that it is in the central area of the retina where the highest concentrations of cones (and thus color perception) are found (in contrast, the periphery is more rod rich and thus better for perception in low light situations).
Or there could be no problem at all (as DinoBoy mentions, ligthing is a non-trivial issue).
Then you also have to take into account that there are several different kinds of color blindness and that the sort which will be diagnosable by one test won't be diagnosable by a test designed for the other (see the normal and tritan test views vs. the protan and deutan test views at
http://webexhibits.org/causesofcolor/2.html the "hidden" number is easily visible for both the person with normal color vision and the person with the tritan color vision defect but not to people with the other color vision defects).
There's an interesting web site which claims to be able to correct images to enhance contrast for the viewer with a given color vision defect at
http://www.vischeck.com/daltonize/ (it links to the webexhibits section on color blindness linked above).
It is, I believe (IIRC), even possible (though very, very rare) to see more colors than the rest of us (but then, I'd imagine that you'd be wondering why the customers can't tell different colors apart and not the other way around).