In this paper, an efficient hardware architecture is proposed for $32{ imes}32$ inverse transform HEVC decoder. HEVC is a new image compression standard to deal with much larger image sizes compared with conventional image codecs, such as 4k, 8k images. To process huge image data effectively, it adopts various new block structures. Theses blocks consists of $4{ imes}4$, $8{ imes}8$, $16{ imes}16$, and $32{ imes}32$ block. This paper suggests an effective structures to process $32{ imes}32$ inverse transform. This structure of inverse transform adopts the decomposed $16{ imes}16$ matrixes of $32{ imes}32$ matrix, and simplified the operations by implementing multiplying with shifters and adders. Additionally the operations frequency is downed by using multicycle paths. Also this structure can be easily adopted to a multi-size transform or a forward transform block in HEVC codec.