In order to reduce the impact of the “similarity drift” problem, a matching strategy based on “neighborhood propagation” was proposed, which maps the intra-modal neighbours of the query items onto the target space, and takes their nearest neighbours in the target space as cross-modal neighbours of the query term. Experiments on real data sets prove that the similarity drifting problem exists widely, and the proposed matching strategy can effectively reduce its impact and improve the accuracy of matching.