In this study, damage detection method using harmony search method and frequency response is proposed. In order to verify this method, the following approaches are implemented. Firstly, damage detection method using harmony search was developed. To detect damage, objective functions that minimize difference with natural frequency and modal strain energy from undamaged and damaged model are used. Secondly, efficiency of developed damage detection method was verified by damage detection of beam structure. And results of harmony search and micro genetic algorithm are compared and evaluated. Thirdly, numerical model was implemented for harbor caisson structure and damage scenario was determined. Lastly, damage detection was performed by proposed method and utility of proposed method is verified.