The upconversion (UC) luminescence of $Li^+/Er^{3+}/Yb^{3+}$ co-doped $CaWO_4$ phosphors and effects of $Yb^{3+}$ concentration are investigated in detail. Single crystallized $CaWO_4$ : $Li^+/Er^{3+}/Yb^{3+}$ phosphor can be obtained, co-doped up to 35.0/5.0/30.0 mol% ($Li^+/Er^{3+}/Yb^{3+}$) by solid-state reaction. Under 980 nm excitation, $CaWO_4$ : $Li^+/Er^{3+}/Yb^{3+}$ phosphor exhibited strong green UC emissions visible to the naked eye at 530 and 550 nm induced by the intra 4f transitions of $Er^{3+}$ ($^4H_{11/2}$, $^4S_{3/2}{ ightarrow}^4I_{15/2}$). The optimum doping concentrations of $Yb^{3+}$ that would result in the highest UC luminescence were determined, and a possible UC mechanism that depends on the pumping power is discussed in detail.