Backward wave oscillator with over-mode Ku-band high power microwave (HPM) generally outputs mixed TM_0n modes, it has high energy conversion efficiency, but it can′t ensure the purity of output modes. However, it is necessary for high radiation efficiency that the microwave input to the radiation terminal is certain single and pure. So it is hard to put these HPM sources into application. In order to solve this problem, a design method for compact hybrid modes converter of purifying TM_0n mixed modes was proposed. With this method, the TM₀₁,TM₀₂ and TM₀₃ mixed modes can be converted into pure circular waveguide TM₀₁ mode efficiently on the condition of high power capacity and wide wave band. This method also reduces the modes purification difficulty in the design of the HPM sources.