In this paper, we propose a method to segment the bottle label in images taken by mobile phones using multi-gradient approaches. In order to segment the label region of interest-object, the saliency map method and Hough Transformation method are first applied to the original images to obtain the candidate region. The saliency map is used to detect the most salient area based on three kinds of features (color, orientation and illumination features). The Hough Transformation is a technique to isolated features of a particular shape within an image. Therefore, we utilize it to find the left and right border of the bottle. Next, we segment the label based on the gradient information obtained from the structure tensor method and edge method. The experimental results have shown that the proposed method is able to accurately segment the labels as the first step of product label recognition system.