We propose a navigation algorithm using image-based visual servoing utilizing a fixed camera. We define the mobile robot navigation problem as an unconstrained optimization problem to minimize the image error between the goal position and the position of a mobile robot. The residual function which is the image error between the position of a mobile robot and the goal position is generally large for this navigation problem. So, this navigation problem can be considered as the nonlinear least squares problem for the large residual case. For large residual, we propose a method to find the second-order term using the secant approximation method. The performance was evaluated using the simulation.