In the steel industry, the detection of tiny defects including its 3D characteristics on steel surfaces is very important from the point of view of quality control. A multi-spectral photometric stereo method is an attractive scheme because the shape of the defect can be obtained based on the images which are acquired at the same time by using a multi-channel camera. Moreover, the calculation time required for this scheme can be greatly reduced for real-time application with the aid of a GPU (Graphic Processing Unit). Although a more reliable shape reconstruction of defects can be possible when the numbers of available images are increased, it is not an easy task to construct a camera system which has more than 3 channels in the visible light range. In this paper, a new 6-channel camera system, which can distinguish the vertical/horizontal linearly polarized lights of RGB light sources, was developed by adopting two 3-CCD cameras and two polarized lenses based on the fact that the polarized light is preserved on the steel surface. The photometric stereo scheme with 6 images was accelerated by using a GPU, and the performance of the proposed system was validated through experiments.