Simulating the extreme boundaries of the entry vehicle using trajectory optimization methods is an effective simulation method before the range test. The Difference-of-Convex (DC) programming method was employed to study the extreme performance of the testing entry vehicle in terms of peak heat flux. The DC decomposition method was utilized to handle constraints such as heat flux, dynamic pressure, and nomal load, and this method was extended to Max-Max type cost functions, such as peak heat flux, peak dynamic pressure, and peak normal load. The Big-M method was adopted to transform the primal problem into a mixed-integer nonlinear programming sub-problem, combining concave-convex decomposition with penalty function technique to address the oscillation and non-convergence issues for the cost function during the iteration process. An improved successive DC programming algorithm based on the DC relaxation model was proposed. Numerical experiments show that the DC relaxation model-based approach has higher approximation accuracy than traditional direct linearization methods, and the proposed algorithm demonstrates high numerical stability, robustness, and optimality of the cost function.