A Bottom-up Method for Rule Construction in Neural-Symbolic Reinforcement Learning

Deep learning has been increasingly successful in the last few years, and it obtained a plethora of impressive results in various contexts. However, the drawbacks and the limitations of deep learning and related approaches have recently become indisputable. Actually, the results of deep learning algorithms are hardly interpretable, and they struggle to generalize to unseen situations. In order to overcome these problems, neural-symbolic methods have been recently proposed as a viable approach in various contexts because neural-symbolic methods combine symbolic with sub-symbolic machine learning methods to gain the advantages of both while avoiding that inherent problems. This dissertation focuses on the proposal of a new neural-symbolic method for neural-symbolic reinforcement learning by discussing its design, the developed implementation, and some experimental results. The first part of this dissertation overviews the ordinary reinforcement learning concepts, and it surveys the major deep reinforcement learning and relational reinforcement learning approaches available in the literature. The second part of the dissertation focuses on a structured comparison among some of the most representative neural-symbolic approaches for inductive logic programming, which can be considered as good candidates for neural-symbolic reinforcement learning. The last part of the dissertation presents the proposed method as an evolution of an existing algorithm proposed by Jiang & Luo and called Neural Logic Reinforcement Learning. The original algorithm provides for the generation of rules using a top-down approach, while the algorithm proposed in this dissertation uses a bottom-up approach that generates rules starting from the states of the environment to obtain general rules to be used for training. The proposed algorithm is presented and compared with the original algorithm to empirically assess the validity of the approach. The proposed method is able to effectively learn many tasks and it successfully generalize to slightly different versions of the training tasks. However, despite requiring less information from the user, it performs worse than NLRL. This dissertation is concluded with a discussion on some limitations of the proposed algorithm and with an overview of future research directions.