It is well-known that a neural network with cyclic connections generates plural limit cycles, thus, being used as a memory system for storing large number of dynamic information. In this paper, a continuous-time cyclic connection neural network was built so that each neuron is connected only to its nearest neurons with binary synaptic weights of ${pm}1$. The type and the number of limit cycles generated by such network has also been demonstrated through simulation. In particular, the effect of chaos signal for transition between limit cycles has been tested. Furthermore, it is evaluated whether the chaotic noise is more effective than random noise in the process of the dynamical neural networks.