For HRI (Human-Robot Interaction) in daily life, robots need to recognize non-rigid objects such as clothes and blankets. However, the recognition of non-rigid objects is challenging because of the variation of the shapes according to the places and laying manners. In this paper, the cognition of non-rigid object based on a cognitive system is presented. The characteristics of non-rigid objects are analysed in the view of HRI and referred to design a framework for the cognition of them. We adopt a linguistic cognitive system for describing all of the events happened to robots. When an event related to the non-rigid objects is occurred, the cognitive system describes the event into a sentential form and stores it at a sentential memory, and depicts the objects with a spatial model for being used as references. The cognitive system parses each sentence syntactically and semantically, in which the nouns meaning objects are connected to their models. For answering the questions of humans, sentences are retrieved by searching temporal information in the sentential memory and by spatial reasoning in a schematic imagery. Experiments show the feasibility of the cognitive system for cognizing non-rigid objects in HRI.