An educational and psychological test tends to measure more than a single ability because examinees’
construct structure is mostly multidimensional. Developing a test, researchers are to make enough
sub-factor specific items and implement the exploratory factor analysis based on pilot test data for the
purpose of identifying the construct validity. When some items appear to be loaded on plural factors with
loading values more than 0.3 or 0.4, it is difficult for a researcher to find clear solution of whether such
items be rejected, used for all the related factors, or used for a specific factor. The main purpose of this
study is to find a way on how to deal with those items. For this, a simulation study was administered to
explore influences of using those items on test reliability and validity. Various simulation factors were
considered such as the amounts of factor loadings for the items loaded on plural factors, the amounts of
factor loadings for the items loaded on a single factor, the degrees of correlation between factors, and the
ratios of the former kind items. The results showed that the reliability and validity of subtests could be
improved or deteriorated by including the former items to produce subscores according to the situation of
each simulation condition. Discussions of the findings were provided, followed by the description of
necessity administering extended follow-up simulation studies.