For successful assembly of flexible parts, informations about their deformation as well as possible misalignments between the holes and their mating parts are essential. Such informations can be acquired from visual sensors. For robotic assembly, the corrective assembly motion to compensate for such misalignments has to be determined from the measured informations. However, this may not be simply derived from the measured misalignment alone because the part deformation progressively occurs during misalignment compensation. Based on the analysis of flexible parts assembly process, this paper presents a neural net-based inference system that can infer the complex relationship between the corrective motion and the measured information of parts deformation and misalignments. And it verifies the performance of the implemented inference system. The results show that the proposed neural net-based misalignment compensation algorithm Is effective in compensating for the lateral misalignment, and that it can be extended to the assembly tasks under more general conditions.