The traditional object tracking algorithms require manually annotated tracking area, and suffer from the problem of drift. To address these difficulties, the problem of person tracking was focused on, and a new detection-based tracking algorithm was proposed. To reduce failure in tracking, multiple detectors to locate multiple body parts were employed, and then their detection results were mapped to a common body area. To adapt for the quickly moving objects, the KLT tracker and agglomerative clustering for linking the detection windows to form person body trajectories was employed. The experimental results reveal that using multiple detectors improves the tracking performance significantly, and the KLT tracker is adaptable for quickly moving objects. This algorithm is nearly real-time. 