Reinforcement Learning based State of Charge Estimation of a Lithium Ion Battery

An accurate data driven, state of charge (SoC) estimation strategy incorporating a Panasonic 18650 lithium-ion battery's (LiB's) primary parameters i.e. voltage, current and temperature is presented in this paper. The mechanism is based on reinforcement learning (RL) along with an ensemble least square boosting (LSB) algorithm and a terminal three point interpolation stage. A twin-delayed deep deterministic neural network-based policy gradient agent (TD3NN) is deployed during the RL training phase, along with a versatile reward function that not only focuses on the accuracy but also the smoothness of the estimated curve. The reward function is designed based on an absolute negative error term and additional bonuses to ensure smoothness and a descending SoC discharge pattern. The LSB algorithm incorporates the number of tree splits and learning cycles to improve the overall accuracy of the proposed approach. The data consists of 55 discharge cycles; a fivefold validation is performed by dividing the data into training and test at 80:20 ratio five times to ensure unbiased results. The numerical values of the performance metrics are reported and analyzed. The minimum root mean square error (RMSE) is reported to be 1.003% and 0.8704% for a single term absolute negative reward function and a multi-term reward function with added bonuses, respectively. These results are not only satisfactory in comparison with the existing literature but also validate the efficacy of the multi-term reward function approach.