As propagation of mobile smart devices is widespread, it is an observable trend that the cases of utilizing them are increasing in the school programs, and it is also anticipated that they will be very important part of the educational equipment in near future. For this reason the department of education and science technology has announced a medium and long term project on the education with smart device, which is undergoing the preparation stage, and the various academic and industrial institutes have actively produced the related research results and the application prototypes. In this paper we propose a framework on design and implementation of a visual context recognition system for educational purpose usable in the school program by utilizing a module for recognition of the texts embedded in the image captured by video camera from mobile smart device. The system proposed in this paper is consisted of the four modules, such as, image acquisition, image processing, information extraction, and knowledge representation, which are explained in details with the practical examples.