Y$_2$O$_3$films were grown on SiO$_2$-covered Si(111), and hydrogen-terminated Si(111), and hydrogen-terminated Si(111) substrates at 50$0^{circ}C$ by ultrahigh vacuum ionized cluster beam deposition (UHV-ICB). The microstructures and growth behavior of these films have been investigated by transmission electron diffraction (TED) and high-resolution transmission electron microscopy(HREM). The TED results show that the $Y_2$O$_3$grown on the SiO$_2$-Si has the epitaxial relationship of (11-1)Y$_2$O$_3$∥(111)Si and [-110]Y$_2$O$_3$∥[-110]Si. The film on the H-Si substrate contains YSulcorner and amorphous YSiulcornerOulcorner layers at the interface, having the orientation relationship each other. For the YSiulcorner and the Si substrate, the relationship is (0001)YSiulcorner∥(111)Si and [1-210]YSiulcorner∥∥[-110]Si. For the $Y_2$O$_3$and the YSiulcorner ; the relationship is as follows: (11-1)Y$_2$O$_3$∥(0001)YSiulcorner and [-110]Y$_2$O$_3$∥[1-210]YSiulcorner(111)Y$_2$O$_3$∥(0001)YSiulcorner and [-110]Y$_2$O$_3$∥[1-210]YSiulcorner. Explanation is given to describe the formation mechanisms of the interfacial phases of SiOulcorner, YSiulcornerOulcorner and YSiulcorner. It is shown that the crystallinity of the $Y_2$O$_3$film on the SiO$_2$-Si(111) is better than that of $Y_2$O$_3$on H-Si(111).