The camera calibration from vanishing points is easily distracted by noise in the image, leading to inaccurate results which are often inadmissible for camera calibration. To overcome the limitation, an iterative optimization approach, which makes full use of geometric constraints of vanishing points and ellipse in the image, was presented for selfcalibration from single image. According to the polepolar relationship and the orthogonality represented by it, a set of orthogonal conjugate vanishing point pairs were calculated through using the ellipse curve and the coplanar vanishing line. A nonlinear model of the principle distance and principle point was established on the basis of these vanishing point pairs. Choosing the minimum variance of principle distances as optimization criterion and setting multiple points as the initial values of the principle point, the principle distance and principle point were iteratively optimized and their optimal results were obtained. Simulated results and real data show that the approach can effectively realize camera selfcalibration from a single image. Compared with the camera calibration method using vanishing points, the approach achieves more satisfactory calibration results.