Multi-sensor image matching is meritorious both in vision mechanism studies and vision engineering. Image segmenting is an effective method for abstracting image properties, but due to the features of images with different sources, the image segmenting method cannot guarantee agreeable image properties from different sources. With a view to this problem, a method is proposed by segmenting the multi-sensor images before the matching strategy is used. Two matching strategy, the segmentation similarity and the minimal edges, can detect the right match. Matching experiments between optic and infrared images demonstrate its effectiveness.