Demand is increasing for mountain-cultivated Panax ginseng (MCG) because its quality is considered superior to that of field-cultivated ginseng (FCG). However, MCG grows very slowly, and the factors that might affect this are unknown. In addition, little information is available about the physiological characteristics of its roots. Here, we investigated local soil environments and compared the histological and chemical properties of MCG and FCG roots. Average diameters, lengths, and fresh weights were much smaller in the former. Photosynthesis rates and root cambial activity also were reduced in the MCG tissues. Our analysis of soil from the mountain site revealed an extremely low phosphorus content, although those samples were richer in total nitrogen and organic matter than were the field soils. MCG roots also contained higher amounts of ginsenosides, and total accumulations increased with age. Moreover, ginsenoside Rh2, a red ginseng-specific compound, accumulated in the MCG roots but not in those from FCG plants. Interestingly, numerous calcium oxalate crystals were found in MCG roots, particularly in their rhizomes (i.e., short stems). Therefore, we can conclude from these results that low levels of the essential mineral phosphorus in mountain soils are a critical factor that retards the growth of mountain ginseng. Likewise, the high accumulation of calcium oxalate crystals in MCG roots might be an adaptation mechanism for survival in such a harsh local environment.