The number of smart phone users is rapidly growing due to recent increase in wireless Internet usage, development of a wide variety of applications, and activation of M2M (Machine to machine) services. Although the smart phone offers benefits of mobility and convenience, it also has serious security problems. To utilize M2M services in the smart phone, a flexible integrated authentication and access control facility is an essential requirement. To solve these problems, we propose a context-aware single sign-on and access control system that uses context-awareness, integrated authentication, access control, and an OSGi service platform in the smart phone environment. In addition, we recommend Fuzzy Logic and MAUT (Multi-Attribute Utility Theory) in handling diverse contexts properly as well as in determining the appropriate security level. We also propose a security system whose properties are flexible and convenient through a typical scenario in the smart phone environment. The proposed context-aware security system can provide a flexible, secure and seamless security service by adopting diverse contexts in the smart phone environment.