A self-optimizing iterative classification method based on image segments which classifies high-resolution remote sensing images by acquiring training samples through semi-supervised fuzzy Cmeans and designing the self-optimizing iterative classifier based on support vector machine was proposed. Image segments could be obtained by fractal net evolution approach and a few labeled samples were selected; based on labeled samples, image segments were clustered by semi-supervised fuzzy C-means clustering method and then training samples could be obtained by intensity filtration from clustering results; the self-optimizing iterative support vector machine was designed to carry on classification iteratively until the classification requirements were met and during the classification process, training samples were updated and optimized to improve the performance of the classifier by statistical analyses of the two adjacent classifications. QuickBird and WorldView images of Wuhan City were classified by the method proposed by this paper and the overall accuracy achieved 94.67% and 92%. In comparison with the overall accuracy of the classification with training samples selected by manual work, the regular support vector machine classification method and the least squares support vector machine classification method, the accuracy of the suggested method is obviously higher and the classification effect is better.