The FRSHV(friend relationship strength hierarchy vote), a hierarchical model, was proposed to measure the friend relationship strength by user′s daily moving track, semantic positions and the corresponding semantic labels. The daily track similarity was measured by dynamic time warping model using the spatial distance between friends, and the results were then weighed by the entropy of track series. The similarities of friend′s behavior patterns were inferred by latent Dirichlet allocation topic model, respectively using semantic positions and the corresponding semantic labels. Finally, these three similarity results were voted for the ultimate relationship strength. The FRSHV was evaluated by using an open dataset and the results proved the validity of the model in inferring friend′s relationship strength.