Architecture and music have reflected many changes occurring in a greater cultural realm. Specially, the concept of time and space formed from relativistic physics becomes the basis of cultural awareness, and a method to understand the world. Therefore, this study aims to suggest the possibility to integrate architecture and music, based on the close relationship between architecture and music of the 20th century focusing on time and space. Resultantly, in this study, interrelationship between architecture and music is elucidated by proving both the spatiality of music and the temporality of architecture, and such possibility to integrate the two realms helps understand the reality.