Visual tracking in a clutter background remains to be a challenging task by far. The particle filter based tracking algorithm proposed in this paper fuses color and texture information to build a robust measurement function. During the measurement step, the color information and texture information were represented by color histograms and gradient orientation vector respectively. Bhattacharyya coefficient and Euclidean distance were used to set up an effective connection between the estimated model parameters and the image likelihoods. Moreover, to overcome the problem of appearance changes, partial occlusions and significant clutter, an adaptive model update method was adopted. Experimental results show that the proposed method is robust and effective.