Abstract: An efficient and powerful design method for calculating optimal Proportional-Integral-Derivative (PID) controllers for AVR systems is proposed. The method is an improved version of the Discrete Action Reinforcement Learning Automata (DARLA) while discrete probability functions (DPF) of the design variables are not considered independent. The results of the proposed method called Extended Discrete Action Reinforcement Learning Automata (EDARLA) are compared to the results obtained by the well known Ziegler-Nichols (ZN), conventional DARLA and Genetic Algorithms (GA) and conventional CARLA approaches. The extensive simulation results prove superiority of the proposed design method in terms of optimality, efficiency, computation burden and being less sensitive to the ranges considered for the design variables that is the search space. Besides being successful in providing globally optimal results, due to high efficiency and lower computation time, the proposed approach can be considered an interesting candidate for designing and tuning optimal adaptive PID controllers for many practical systems.