The effect of crystal orientation and microstructure on the mechanical properties of $TaN_x$ was investigated. $TaN_x$ films were grown on $SiO_2$ substrates by ultrahigh vacuum unbalanced magnetron sputter deposition in mixed $Ar/N_2$ discharges at 20 mTorr (2.67 Pa) and at $350^{circ}C$. Unlike the Ti-N system, in which TiN is the terminal phase, a large number of N-rich phases in the Ta-N system could lead to layers which had nano-sized lamella structure of coherent cubic and hexagonal phases, with a correct choice of nitrogen fraction in the sputtering mixture and ion irradiation energy during growth. The preferred orientations and the micro-structure of $TaN_x$ layers were controlled by varing incident ion energy $E_i;(=30eV~50eV)$ and nitrogen fractions $f_{N2};(=0.1~0.15)$. $TaN_x$ layers were grown on (0002)-Ti underlayer as a crystallographic template in order to relieve the stress on the films. The structure of the $TaN_x$ film transformed from Bl-NaCl $delta-TaN_x$ to lamellar structured Bl-NaCl $delta-TaN_x$ + hexagonal $varepsilon-TaN_x$ or Bl-NaCl $delta-TaN_x$ + hexagonal $gamma-TaN_x$ with increasing the ion energy at the same nitrogen fraction $f_{N2}$. The hardness of the films also increased by the structural change. At the nitrogen fraction of $0.1~0.125$, the structure of the $TaN_x$ films was changed from $delta-TaN_x;+;varepsilon-TaN_x;to;delta-TaN_x;+;gamma-TaN_x$ with increasing the ion energy. However, at the nitrogen fraction of 0.15 the film structure did not change from $delta-TaN_x;+;varepsilon-TaN_x$ over the whole range of the applied ion energy. The hardness increased significantly from 21.1 GPa to 45.5 GPa with increasing the ion energy.