This paper presents the use of the inertial measurement unit information and the infrared sector information for getting the position of an object. Travel distance is usually calculated from the double integration of the accelerometer output with respect to time; however, the accumulated errors due to the drift are inevitable. The orientation change of the accelerometer also causes error because the gravity is added to the measured acceleration. Unless three axis orientations are completely identified, the accelerometer alone does not provide correct acceleration for estimating the travel distance. We propose a way of minimizing the error due to the change of the orientation. In order to reduce the accumulated error, the infrared sector information is fused with the inertial measurement unit information. Infrared sector information has highly deterministic characteristics, different from RFID. By putting several infrared emitters on the ceiling, the floor is divided into many different sectors and each sector is set to have a unique identification. Infrared light based sector information tells the sector the object is in, but the size of the uncertainty is too large if only the sector information is used. This paper presents an algorithm which combines both the inertial measurement unit information and the sector information so that the size of the uncertainty becomes smaller. It also introduces a framework which can be used with other types of the artificial landmarks. The characteristics of the developed infrared light based sector and the proposed algorithm are verified from the experiments.