We investigate how plasma structures in the solar chromosphere and corona can extend to altitudes much above hydrostatic scale heights from the solar surface even under the force of gravity. Using a simple modified form of equation of motion in the vertical direction, we argue that there are two extreme ways of non-hydrostatic support: dynamical support and magnetic support. If the vertical acceleration is downward and its magnitude is a significant fraction of gravitational acceleration, non-hydrostatic support is dynamical in nature. Otherwise non-hydrostatic support is static, and magnetic support by horizontal magnetic fields is the only other possibility. We describe what kind of observations are needed in the clarification of the nature of non-hydrostatic support. Observations available so far seem to indicate that spicules in the quiet regions and dynamic fibrils in active regions are dynamically supported whereas the general chromosphere as well as prorninences is magnetically supported. Moreover, it appears that magnetic support is required for plasma in some coronal loops as well. We suspect that the identification of a coronal loop with a simple magnetic flux tube might be wrong in this regard.