This paper is investigated conception and methodology of data selection, cleaning, integration, transformation, reduction, selection and application of data mining techniques, and model evaluation during procedure of the knowledge discovery in database (KDD) based on Mathematics. Statistical role and methodology in KDD is studied as branch of Mathematics. Also, we investigate the history, mathematical background, important modeling techniques using statistics and information, practical applied field and entire examples of data mining. Also we study the differences between data mining and statistics.