Image classification is an important problem in computer vision. However, it is a very challenging problem due to the variability, ambiguity and scale change that exists in images. In this paper, we propose a method of a hypergraph based modeling can consider the higher-order relationships of semantic attributes of a scene image and apply it to a scene image classification. In order to generate the hypergraph optimized for specific scene category, we propose a novel search method based on a probabilistic subspace method and also propose a method to aggregate the expression values of the member semantic attributes that belongs to the searched subsets based on a linear transformation method via likelihood based estimation. To verify the superiority of the proposed method, we showed that the discrimination power of the feature vector generated by the proposed method is better than existing methods through experiments. And also, in a scene classification experiment, the proposed method shows a competitive classification performance compared with the conventional methods.