The vulnerability of privacy in the Identity Management System (IMS) is the most pressing concern of ordinary users. Uncertainty about privacy keeps many users away from utilization of IMS. Therefore, this paper proposes an access-control oriented privacy model for IMS. The proposed model protects privacy using access control techniques with privacy policies in a single circle of trust. We address characteristics of the components of for the proposed model and describe access control procedures. After that, we show the architecture of privacy enforcement and XML-based schema for privacy policies.