Currently, horizontal direction drilling (HDD) has been considered as the mainstream in the trenchless technology used for municipal construction, in which the geo-radar and electromagnetic-based underground pipe detectors are widely used to find the distribution of pipes near the construction route so as to avoid destroying the latent buried pipes. Limited by detection principles, many objects cannot be detected effectively. In this paper, a novel solution for detecting underground pipes is proposed. By introducing tiny ultrasonic detecting system installed on the head of HDD, an online surveillance system for underground pipes was implemented with drilling and detecting implemented simultaneously. As a result, the system gave warning to the operators of the drilling when it detected pipes under the construction route. The experimental results show that construction risks of HDD were reduced considerately and the system proved to be effective.