This paper presents the micro-mechanical modeling for predicting concrete behavior under compressive loading. The model is able to represent the heterogeneities in the microstructure up to three phases, i.e., aggregate particles, matrix and interfaces. The smeared crack concept based on non-linear fracture mechanics is implemented in order to formulate the constitutive relation for each component. The splitting tensile strength is considered as a fracture criterion for cracking in micro-level. The finite element method is employed to simulate the model based on plane stress condition by using quadratic triangular elements. The validation of the model is verified by comparing with the experimental results. The influence of tensile strength from both aggregate and matrix phases on the concrete compressive strength is demonstrated. In addition, a guideline on selecting appropriate tensile strength for each phase to obtain specified concrete compressive strength is also presented.