To mitigate the degradation of tracking accuracy induced by numerous false association ghost points, a dual-level ghost point elimination and target tracking algorithm integrating angular measurement with target motion characteristics was introduced. The proposed algorithm employed a cooperative localization strategy that prioritizes association before estimation. By establishing a mapping relationship between angular measurement noise and localization error, a field-of-view grid map and an energy accumulation matrix were constructed. Through a detailed analysis of the spatial geometric distribution characteristics of real targets and false association ghost points within the field of view, a novel elimination criterion based on Hough transform theory was developed, facilitating the primary coarse elimination of ghost points. Additionally, by examining the distribution characteristics of target localization ambiguity regions and motion features, a predictive tracking gate was constructed using motion parameter identification, enabling the secondary fine elimination of ghost points at the kinematic level. Experimental results demonstrate that the proposed algorithm significantly enhances target tracking accuracy, achieving a ghost point elimination rate of 91.82%, thereby effectively addressing the false association problem in multi-target tracking.