We analyzed the factors driving succession and the structure, and dynamics of vegetation on sand and gravel bars in order to clarify the differences in vegetation succession in rivers with different river bed substrates. Woody plant communities (dominated by Salix), perennial herb communities (dominated by Miscanthus), and annual plant communities (dominated by Persicaria) appeared in that order from upstream to downstream on the sandbar. The results of DCA ordination based on vegetation data reflected a successional trend. This result suggests that sandbars grow in a downstream direction. Various vegetation types different in successional stage, such as grassland, young stands of Korean red pine (Pinus densiflora), two-layered stands of young and mature pines, and mature pine stands also occurred on gravel bars, but the vegetation in earlier successional stage was established upstream, which is the opposite to the direction found on sandbars. Those results demonstrate that the dynamics of the bed load itself could be a factor affecting vegetation succession in rivers. In fact, sands suspended by running water were transported downstream over the vegetated area of sand bar and thereby created new areas of sandbar on the downstream end of the sandbar. Meanwhile, gravel, which is heavy and thereby is shifted by strong water currents, accumulated on the upstream end of the vegetated area, and thus created new areas of gravel bar in that direction. These results showed that allogenic processes drive vegetation succession on sand and gravel bars in streams and rivers.