This paper proposes a novel approach to shadow detection from a single natural image regardless of orientation and type of light sources. This approach is based on the assumption that shadow brightness changes linearly, and the axiom that a region cast shadow on is darker than that not having shadow under the same environment. Firstly, candidates for shadow are extracted by preprocessing. Then, they are quantized to replace the similar values with a representative value because of the more quantization steps of a pixel brightness, the higher linear independency among the neighboring pixels. Finally, shadows are detected according to linear independency of shadow brightness based on the assumption. The experimental results showed the proposed approach can robustly detect umbra as well as self-shadow and penumbra cast on a single-colored background.