Background: In 2006 Emergency Medical Services Index (EMSI), which summarizes the performance of regional emergency medical services system, was developed. This study assesses the performance of the EMSI to help determine whether EMSI can be used as evaluation tool. Methods: To build a composite score of the EMSI from predefined 24 indicators, 3 normalized values were calculated for each indicator, the normalized values of each indicator were weighted using 4 weighting methods, and the weighted values were aggregated into the final composite score using 2 aggregation schemes. The performance of EMSI was evaluated using 3 criteria: discrimination, construct validity, and sensitivity. Discrimination was the proportion of regions that did not include the overall median rank in the 5th to 95th percentiles rank interval, which was calculated from Monte Carlo simulation. Construct validity was a correlation among the alternative EMSIs. Sensitivity of EMSIs was evaluated by total shift of quartile membership and changes of 5th to 95th percentile intervals. Results: The total discrimination performance of the EMSI was 50.0%. Correlation coefficients between EMSIs using standardized values and those using rescaled values ranged from 0.621 to 0.997. Variation of the quartile membership of regions ranged from 0.0% to 75.0%. The total change in the 5th to 95th percentile intervals ranged from -19 to +17 places. Conclusion: The results suggested that the EMSI could be used as a tool for evaluating quality of regional EMS system and for identifying the areas for quality improvement.