The performance of edge detection often relies on its ability to correctly determine the dissimilarities of connected pixels. For grayscale images, the dissimilarity of two pixels is estimated by a scalar difference of their intensities and for color images, this is done by using the vector difference (color distance) of the three-color components. The Euclidean distance in the RGB color space typically measures a color distance. However, the RGB space is not suitable for edge detection since its color components do not coincide with the information human perception uses to separate objects from backgrounds. In this paper, we propose a novel method for color edge detection by taking advantage of the HSV color space and the Mahalanobis distance. The HSV space models colors in a manner similar to human perception. The Mahalanobis distance independently considers the hue, saturation, and lightness and gives them different degrees of contribution for the measurement of color distances. Therefore, our method is robust against the change of lightness as compared to previous approaches. Furthermore, we will introduce a noise-resistant technique for determining image gradients. Various experiments on simulated and real-world images show that our approach outperforms several existing methods, especially when the images vary in lightness or are corrupted by noise.