Local linear prediction is often applied to predict nonlinear time series, which uses the ordinary least square(LS) method to estimate the parameters in the approximated linear models. If there exists noise in the process, the computational stability of the method is rather poor. This paper presents an improved method that uses the orthogonal least square (OLS) algorithm to estimate both the structure and the parameters in the linear models from linearizing locally the whole nonlinear space. The proposed method can solve the ill-posed numerical problem to some extent and increase the stability of prediction algorithm.