The process of calibration and 3D reconstruction of the scene are always required in view synthesis from image sequences. To avoid those complicated processes, the reference images are first classified into primary reference image and subordinate reference image according to the distance between the reference viewpoints and the novel viewpoint. Then the global optimization problem of view synthesis is transformed from the depth field to disparity field, using a process of nonlinear rectification and multi-epipolar technology. Finally, the novel view is synthesized from the image sequence without matching and calibration. Experimental results show that the method is effective and has potential in the future.