This paper presents a top-down approach to implement an MPEG-2 audio encoder in VLSI. As the algorithm of an MPEG-2 audio encoder is heavy-weighted and heterogeneous(to be mixture of several strategies), the encoder design process is undertaken carefully from the algorithmic level to the architectural level. Firstly, the encoding algorithm is analyzed and divided into sub-algorithms, called tasks, and the tasks are partitioned in the way of reusing the same designs. Secondly, the partitioned tasks are scheduled and synthesized to make the most efficient use of time and space. In the end, a real-time 5 channel MPEG-2 audio encoder is designed which is a heterogeneous multiprocessor system; two hardwired logic blocks and one specialized DSP processor.