The limited number of roads and the increasing number of vehicles demand the automatic regulation of overspeed vehicles, illegal vehicles, and overloaded vehicles and the automatic charge calculation depending on the type of the vehicle. To meet such requirements, it is important to remove the shadow of the vehicle as processing and recognizing an image captured by a camera. The shadow of the vehicle is likely to cause misclassification of the vehicle type due to diverse errors and mistakes occurring when detecting geometrical properties of the vehicle. In case that shadows of two different vehicles are overlapped, not only the type of the vehicles may be misclassified but also it is difficult to accurately identify the type of the vehicles. In this paper, we propose a robust algorithm to remove the shadow of a vehicle by calculating the luminance, the chrominance, the gradient density of the cast shadow from information acquired using the image subtraction of the background, and to recognize the substantial vehicle figure. Even when it is hard to detect and split a target vehicle from its shadow as shadows of vehicles are attached to each other, our robust algorithm can detect the vehicle figure only. We implemented our system with a general camera and conducted experiments on various vehicles on general roads to find out our vehicle shade removal algorithm is efficient when detecting and recognizing vehicles.