In this paper, a new learning methodology for kernel methods that results in a sparse representation of kernel space from the training patterns for classification problems is suggested. Among the traditional algorithms of linear discriminant function, this paper shows that the relaxation procedure can obtain the maximum margin separating hyperplane of linearly separable pattern classification problem as SVM(Support Vector Machine) classifier does. The original relaxation method gives only the necessary condition of SV patterns. We suggest the sufficient condition to identify the SV patterns in the learning epoches. For sequential learning of kernel methods, extended SVM and kernel discriminant function are defined. Systematic derivation of learning algorithm is introduced. Experiment results show the new methods have the higher or equivalent performance compared to the conventional approach.