This paper presents a modified TEM horn that improves radiation characteristics at a low frequency region. The proposed antenna consists of an asymmetric TEM (ATEM) horn and a loop structure with an elliptical shape. The bandwidth and gain at low frequency region can be enhanced by using the ATEM horn configuration and adding a loop structure with an elliptical shape to the ATEM horn. The bandwidth of the proposed antenna is from 2.14 to over 20 GHz, whereas that of the conventional TEM horn is from 2.7 to over 20 GHz, where the dimensions of both antennas are the same except for the thickness of the loop structure. The physical and electrical dimensions of the proposed antenna are $60mm{ imes}62.5mm{ imes}64mm$ ($width{ imes}height{ imes}length$) and $0.428{lambda}_L{ imes}0.445{lambda}_L{ imes}0.456{lambda}_L$, where ${lambda}_L$ corresponds to the lowest frequency of the bandwidth. The realized gain of the proposed antenna is improved by 0.802 dB on average at the low frequency region (2 to 8 GHz), where the maximum gain increase is 2.932 dB when compared to a conventional TEM horn.