Heat recovery ventilators (HRV) are developed in order to satisfy both energy conservation and the improvement of indoor air quality as an alternative for current natural ventilation systems and local mechanical ventilation systems in kitchens and bathrooms. However, the performance of HRV system and the consequent effect on heating and cooling energy saving have not been sufficiently validated quantitatively in case of the application of HRVs in real residences. In this study, field measurement and computer simulation were conducted in both summer and winter period to assess the performance and validate energy conservation effect of HRVs. Under the Korea weather condition, average total heat recovery efficiency was $27\%$ in summer and $46\%$ in winter. According to the field measurement, HRV system can save the energy by $10\%$ in summer and 15$\%$ in winter. Furthermore, according to the simulation assessment, HRV system can save the energy by $17\%$ in summer and $17\%$ in winter.