The image guidance using image sequences analysis improves the shoot accuracy of missiles greatly. In this paper, we present a new method of automatic object recognition and tracking based on deformable template and genetic annealing algorithm. After the constructing object template and defining energy function according to their features, we use genetic annealing algorithm to search for the optimal solution. Experiment results show the method can be applied to detect and track the object efficiently and accurately.