The training of support vector machine is a difficult issue in classifying large-scale data set. Incremental learning is one of the solutions to the difficulty. After new samples were added to training set, the possible changes of support vector set, were analyzed and a removing algorithm based on density for incremental support vector machine learning was presented. It discarded useless samples, kept the testing accuracy and reduced the training time. Experiments show the validity of this algorithm.