Based on the GBS(graph-based segmentation) algorithm and the hierarchical clustering algorithm, a method to solve the under-segmentation of GBS algorithm was constructed. Meanwhile, the way of multi-threaded parallel processing of data was used to effectively improve the processing speed of the traditional hierarchical clustering algorithm. In the RGB color space, the GBS algorithm was used to obtain the initial segmentation result of each pixel in the image. The pixel value in each type of region was extracted and the hierarchical clustering was carried out to obtain the category label of pixel value in each type of region. According to the category label obtained by hierarchical clustering and the preset category range, the initial segmentation result of each pixel was modified. A new segmentation graph was generated according to the region merging criterion. Experimental results show that compare with the K-means-SLIC algorithm and the GBS algorithm, this method solves the phenomenon of under-segmentation, and produces a semantic segmentation graph with high segmentation accuracy.