Mobile ad hoc network is a communication network formed by mobile nodes with non-infrastructure, which has highly dynamic characteristics. Conventional routing protocols cannot adapt to the frequent topology changes brought by node mobility, and the flooding routing also causes the network performance degradation due to the excessive routing overhead. A stepwise routing algorithm based on reinforcement learning was proposed for adaptive routing in mobile ad hoc networks. This algorithm aims at total round trip time minimization and uses the reinforcement learning algorithm to select the next hop. After selecting the set of nodes that meet the requirements of the target, it combines the confidence parameters to select the route. When the link becomes unreliable, packets are broadcasted to filtered neighbor nodes to improve the reliability and reduce the routing overhead. The main property indication of the proposed algorithm, such as throughput and routing overhead, were analyzed theoretically. The simulation results show that, compared with the reinforcement learning based smart robust routing, the proposed routing algorithm reduces the overhead and maintains a competitive throughput.