MicroRNAs (miRNAs) are endogenous small RNAs regulating plant development and stress responses. In addition, phosphorus (P) is an important macronutrient for plant growth and development. More than two hundred miRNAs have been identified in Glycine Max and a few of miRNAs have been shown to respond to P deficiency, however, whether there are other miRNAs involved in P deficiency response is largely unknown. In this study, we used high-throughput small RNA sequencing and whole-genome-wide mining to identify the potential miRNAs in response to P deficiency. After sequencing, we deduced 183 known, 99 conserved and 126 novel miRNAs in Glycine Max. Among them, in response to P deficiency, the expressions of 27 known, 16 conserved and 12 novel miRNAs showed significant changes in roots, whereas the expressions of 34 known, 14 conserved and 7 novel miRNAs were significantly different in shoots. Furthermore, we validated the predicated novel miRNAs and found that three miRNAs in roots and five miRNAs in shoots responded to P deficiency. Some miRNAs were P-induced whereas some were P-suppressed. Together these results indicated that the miRNAs identified might play important roles in regulating P signaling pathway.