Aiming at the issues of low utilization of interaction data or the need for additional task data in traditional agent exploration work, an online-learnable exploration latent variable that characterizes the current task features to assist the policy network in behavioral decision-making was innovatively introduced. There was no need for additional multi-task data or additional environmental interaction steps in the current task. The exploring latent variable was updated in the learnable environment model, and the environment model underwent supervised updates based on the intelligent agent and real environment interaction data. The exploration in advance in the simulated environment model was assisted by the exploring latent variable, and thus the performance of agents in the real environment was improved. The performance in typical continuous control tasks was raised by about 30% in the experiments, which was of guiding significance for the single-task exploration and meta reinforcement learning research.