Original omnidirectional images usually need to be unwrapped to cylindrical panoramic images according to human visual habits because of concentric annular distortion. According to the characteristics of catadioptric omnidirectional imaging, an interpolation method for panoramic unwrapping based on bidirectional mapping was proposed. The coordinate mapping relationships were obtained by tracking optical path. Firstly, each pixel in the target cylindrical panoramic image was projected backward onto the original omnidirectional image, and the neighboring pixels for interpolation were located. Then, all the neighboring pixels were projected forward onto the cylindrical panoramic image, and the color value of the target pixel was interpolated by using inverse distance weighted method or 3D-plane interpolation method due to the scattered distribution of the neighboring pixels in cylindrical panoramic images. The proposed method avoids the adverse effect on computing the interpolation weight of neighboring pixels caused by the nonlinear distortion, and the neighboring pixels for interpolation are also accurately located. The experimental results indicate that the proposed method is effective for enhancing the precision of interpolation and the image quality of unwrapped panoramic images.