In telerobot systems, visual servoing of a task object for a slave arm with an eye-in-hand has drawn an interesting attention. As such a task ingenerally conducted in an unstructured environment, it is very difficult to define the inverse feature Jacobian matrix. To overcome this difficulty, this paper proposes an auto-tuning fuzzy rule-based visual servo algorithm. In this algorithm, a visual servo controller composed of fuzzy rules, receives feature errors as inputs and generates the change of have position as outputs. The fuzzy rules are tuned by using steepest gradient method of the cost function, which is defined as a quadratic function of feature errors. Since the fuzzy rules are tuned automatically, this method can be applied to the visual servoing of a slave arm in real time. The effctiveness of the proposed algorithm is verified through a series of simulations and experiments. The results show that through the learning procedure, the slave arm and track object in real time with reasonable accuracy.