Neurosymbolic Reinforcement Learning and Planning: A Survey
The area of Neurosymbolic Artificial Intelligence (Neurosymbolic AI) is rapidly developing and has become a popular research topic, encompassing sub-fields such as Neurosymbolic Deep Learning (Neurosymbolic DL) and Neurosymbolic Reinforcement Learning (Neurosymbolic RL). Compared to traditional learning methods, Neurosymbolic AI offers significant advantages by simplifying complexity and providing transparency and e…
Licence
OPEN
CC-BY-4.0
Authors
K. Acharya, W. Raza, C. M. J. M. Dourado, A. Velasquez, H. Song
L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” Journal of artificial intelligence research, vol. 4, pp. 237–285, 1996.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016.
R.-Z. Liu, W. Wang, Y. Shen, Z. Li, Y. Yu, and T. Lu, “An introduction of mini-alphastar,” arXiv preprint arXiv:2104.06890, 2021.
T. R. Besold, A. d. Garcez, S. Bader, H. Bowman, P. Domingos, P. Hitzler, K.-U. Kühnberger, L. C. Lamb, D. Lowd, P. M. V. Lima et al., “Neural-symbolic learning and reasoning: A survey and interpretation,” arXiv preprint arXiv:1711.03902, 2017.
W. Wang and Y. Yang, “Towards data-and knowledge-driven artificial intelligence: A survey on neuro-symbolic computing,” arXiv preprint arXiv:2210.15889, 2022.
D. Yu, B. Yang, D. Liu, and H. Wang, “A survey on neural-symbolic systems,” arXiv preprint arXiv:2111.08164, 2021.
D. Bouneffouf and C. C. Aggarwal, “Survey on applications of neurosymbolic artificial intelligence,” arXiv preprint arXiv:2209.12618, 2022.
K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26–38, 2017.
Y. Li, “Deep reinforcement learning: An overview,” arXiv preprint arXiv:1701.07274, 2017.
Y. Zeng, R. Cai, F. Sun, L. Huang, and Z. Hao, “A survey on causal reinforcement learning,” arXiv preprint arXiv:2302.05209, 2023.
J. Garcıa and F. Fernández, “A comprehensive survey on safe reinforcement learning,” Journal of Machine Learning Research, vol. 16, no. 1, pp. 1437–1480, 2015.
A. Uprety and D. B. Rawat, “Reinforcement learning for iot security: A comprehensive survey,” IEEE Internet of Things Journal, vol. 8, no. 11, pp. 8693–8706, 2020.
S. Padakandla, “A survey of reinforcement learning algorithms for dynamically varying environments,” ACM Computing Surveys (CSUR), vol. 54, no. 6, pp. 1–25, 2021.
L. Buşoniu, R. Babuška, and B. De Schutter, “Multi-agent reinforcement learning: An overview,” Innovations in multi-agent systems and applications-1, pp. 183–221, 2010.
L. Canese, G. C. Cardarilli, L. Di Nunzio, R. Fazzolari, D. Giardino, M. Re, and S. Spanò, “Multi-agent reinforcement learning: A review of challenges and applications,” Applied Sciences, vol. 11, no. 11, p. 4948, 2021.
S. Gronauer and K. Dieopold, “Multi-agent deep reinforcement learning: a survey,” Artificial Intelligence Review, vol. 55, pp. 1–49, 02 2022.
V. Uc-Cetina, N. Navarro-Guerrero, A. Martin-Gonzalez, C. Weber, and S. Wermter, “Survey on reinforcement learning for language processing,” Artificial Intelligence Review, vol. 56, no. 2, pp. 1543–1575, 2023.
J. Luketina, N. Nardelli, G. Farquhar, J. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, and T. Rocktäschel, “A survey of reinforcement learning informed by natural language,” arXiv preprint arXiv:1906.03926, 2019.
Y. Qian, J. Wu, R. Wang, F. Zhu, and W. Zhang, “Survey on reinforcement learning applications in communication networks,” Journal of Communications and Information Networks, vol. 4, no. 2, pp. 30–39, 2019.
J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” The International Journal of Robotics Research, vol. 32, no. 11, pp. 1238–1274, 2013.
C. Yu, J. Liu, S. Nemati, and G. Yin, “Reinforcement learning in healthcare: A survey,” ACM Comput. Surv., vol. 55, no. 1, nov 2021. [Online]. Available: https://doi.org/10.1145/3477600
A. Haydari and Y. Yılmaz, “Deep reinforcement learning for intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 11–32, 2022.
W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” The bulletin of mathematical biophysics, vol. 5, pp. 115–133, 1943.
H. Wang and B. Raj, “On the origin of deep learning,” arXiv preprint arXiv:1702.07800, 2017.
N. Metropolis and S. Ulam, “The monte carlo method,” Journal of the American statistical association, vol. 44, no. 247, pp. 335–341, 1949.
R. Bellman and R. Kalaba, “Dynamic programming and statistical communication theory,” Proceedings of the National Academy of Sciences, vol. 43, no. 8, pp. 749–751, 1957.
R. Bellman, “A markovian decision process,” Journal of mathematics and mechanics, pp. 679–684, 1957.
F. Rosenblatt, The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory, 1957.
A. L. Samuel, “Some studies in machine learning using the game of checkers,” IBM Journal of research and development, vol. 3, no. 3, pp. 210–229, 1959.
A. G. Ivakhnenko, “Polynomial theory of complex systems,” IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-1, no. 4, pp. 364–378, 1971.
K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological cybernetics, vol. 36, no. 4, pp. 193–202, 1980.
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural computation, vol. 1, no. 4, pp. 541–551, 1989.
A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE transactions on systems, man, and cybernetics, no. 5, pp. 834–846, 1983.
C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, pp. 279–292, 1992.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” nature, vol. 323, no. 6088, pp. 533–536, 1986.
G. Tesauro, “Practical issues in temporal difference learning,” Advances in neural information processing systems, vol. 4, 1991.
R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Reinforcement learning, pp. 5–32, 1992.
C. J. C. H. Watkins, “Learning from delayed rewards,” 1989.
L.-J. Lin, Reinforcement learning for robots using neural networks. Carnegie Mellon University, 1992.
E. Lindholm, J. Nickolls, S. Oberman, and J. Montrym, “Nvidia tesla: A unified graphics and computing architecture,” IEEE micro, vol. 28, no. 2, pp. 39–55, 2008.
M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents,” Journal of Artificial Intelligence Research, vol. 47, pp. 253–279, 2013.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning. PMLR, 2015, pp. 1889–1897.
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016.
Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in International conference on machine learning. PMLR, 2016, pp. 1995–2003.
T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” arXiv preprint arXiv:1511.05952, 2015.
V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning. PMLR, 2016, pp. 1928–1937.
J. Ho and S. Ermon, “Generative adversarial imitation learning,” Advances in neural information processing systems, vol. 29, 2016.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540, 2016.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning. PMLR, 2017, pp. 1126–1135.
M. G. Bellemare, W. Dabney, and R. Munos, “A distributional perspective on reinforcement learning,” in International conference on machine learning. PMLR, 2017, pp. 449–458.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” in International conference on machine learning. PMLR, 2017, pp. 2778–2787.
M. Hessel, J. Modayil, H. Van Hasselt, T. Schaul, G. Ostrovski, W. Dabney, D. Horgan, B. Piot, M. Azar, and D. Silver, “Rainbow: Combining improvements in deep reinforcement learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018.
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al., “Mastering the game of go without human knowledge,” nature, vol. 550, no. 7676, pp. 354–359, 2017.
T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning. PMLR, 2018, pp. 1861–1870.
D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke et al., “Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation,” arXiv preprint arXiv:1806.10293, 2018.
L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning et al., “Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures,” in International conference on machine learning. PMLR, 2018, pp. 1407–1416.
D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel et al., “Mastering chess and shogi by self-play with a general reinforcement learning algorithm,” arXiv preprint arXiv:1712.01815, 2017.
A. Team, AlphaStar: Mastering the real-time strategy game StarCraft II. DeepMind blog, 2019.
M. Jaderberg, W. M. Czarnecki, I. Dunning, L. Marris, G. Lever, A. G. Castaneda, C. Beattie, N. C. Rabinowitz, A. S. Morcos, A. Ruderman et al., “Human-level performance in 3d multiplayer games with population-based reinforcement learning,” Science, vol. 364, no. 6443, pp. 859–865, 2019.
A. Fawzi, M. Balog, A. Huang, T. Hubert, B. Romera-Paredes, M. Barekatain, A. Novikov, F. J. R Ruiz, J. Schrittwieser, G. Swirszcz et al., “Discovering faster matrix multiplication algorithms with reinforcement learning,” Nature, vol. 610, no. 7930, pp. 47–53, 2022.
N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano, “Learning to summarize with human feedback,” Advances in Neural Information Processing Systems, vol. 33, pp. 3008–3021, 2020.
J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu, “The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision,” arXiv preprint arXiv:1904.12584, 2019.
D. Kahneman, Thinking, fast and slow. macmillan, 2011.
G. Booch, F. Fabiano, L. Horesh, K. Kate, J. Lenchner, N. Linck, A. Loreggia, K. Murgesan, N. Mattei, F. Rossi et al., “Thinking fast and slow in ai,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 17, 2021, pp. 15 042–15 046.
A. d. Garcez and L. C. Lamb, “Neurosymbolic ai: The 3 rd wave,” Artificial Intelligence Review, pp. 1–20, 2023.
L. C. Lamb, A. Garcez, M. Gori, M. Prates, P. Avelar, and M. Vardi, “Graph neural networks meet neural-symbolic computing: A survey and perspective,” arXiv preprint arXiv:2003.00330, 2020.
M. K. Sarker, L. Zhou, A. Eberhart, and P. Hitzler, “Neuro-symbolic artificial intelligence,” AI Communications, vol. 34, no. 3, pp. 197–209, 2021.
S. Bader and P. Hitzler, “Dimensions of neural-symbolic integration-a structured survey,” arXiv preprint cs/0511042, 2005.
H. Kautz, “The third ai summer: Aaai robert s. engelmore memorial lecture,” AI Magazine, vol. 43, no. 1, pp. 105–125, 2022.
J. Salvador, J. Oliveira, and M. Breternitz, “Reinforcement learning: A literature review (september 2020),” arXiv, no. December, pp. 1–36, 2020.
S. Balhara, N. Gupta, A. Alkhayyat, I. Bharti, R. Q. Malik, S. N. Mahmood, and F. Abedi, “A survey on deep reinforcement learning architectures, applications and emerging trends,” IET Communications, 2022.
A. Gosavi, “Reinforcement learning: A tutorial survey and recent advances,” INFORMS Journal on Computing, vol. 21, no. 2, pp. 178–192, 2009.
H.-n. Wang, N. Liu, Y.-y. Zhang, D.-w. Feng, F. Huang, D.-s. Li, and Y.-m. Zhang, “Deep reinforcement learning: a survey,” Frontiers of Information Technology & Electronic Engineering, vol. 21, no. 12, pp. 1726–1744, Dec 2020. [Online]. Available: https://doi.org/10.1631/FITEE.1900533
S. Milani, N. Topin, M. Veloso, and F. Fang, “A survey of explainable reinforcement learning,” arXiv preprint arXiv:2202.08434, 2022.
A. Aubret, L. Matignon, and S. Hassas, “A survey on intrinsic motivation in reinforcement learning,” arXiv preprint arXiv:1908.06976, 2019.
P. Yadav, A. Mishra, J. Lee, and S. Kim, “A survey on deep reinforcement learning-based approaches for adaptation and generalization,” arXiv preprint arXiv:2202.08444, 2022.
M. L. Puterman, “Chapter 8 markov decision processes,” 1990.
Z. T. Qin, H. Zhu, and J. Ye, “Reinforcement learning for ridesharing: An extended survey,” Transportation Research Part C: Emerging Technologies, vol. 144, p. 103852, 2022.
J. Zhu, F. Wu, and J. Zhao, “An overview of the action space for deep reinforcement learning,” in 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence, 2021, pp. 1–10.
R. Paulus, C. Xiong, and R. Socher, “A deep reinforced model for abstractive summarization,” arXiv preprint arXiv:1705.04304, 2017.
M. Garnelo, K. Arulkumaran, and M. Shanahan, “Towards deep symbolic reinforcement learning,” arXiv preprint arXiv:1609.05518, 2016.
A. d. Garcez, A. R. R. Dutra, and E. Alonso, “Towards symbolic reinforcement learning with common sense,” arXiv preprint arXiv:1804.08597, 2018.
Z. Ma, Y. Zhuang, P. Weng, H. H. Zhuo, D. Li, W. Liu, and J. Hao, “Learning symbolic rules for interpretable deep reinforcement learning,” arXiv preprint arXiv:2103.08228, 2021.
M. Landajuela, B. K. Petersen, S. Kim, C. P. Santiago, R. Glatt, N. Mundhenk, J. F. Pettit, and D. Faissol, “Discovering symbolic policies with deep reinforcement learning,” in International Conference on Machine Learning. PMLR, 2021, pp. 5979–5989.
L. Mitchener, D. Tuckey, M. Crosby, and A. Russo, “Detect, understand, act: A neuro-symbolic hierarchical reinforcement learning framework,” Machine Learning, vol. 111, no. 4, pp. 1523–1549, 2022.
M. Jin, Z. Ma, K. Jin, H. H. Zhuo, C. Chen, and C. Yu, “Creativity of ai: Automatic symbolic option discovery for facilitating deep reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 6, 2022, pp. 7042–7050.
D. Kimura, M. Ono, S. Chaudhury, R. Kohita, A. Wachi, D. J. Agravante, M. Tatsubori, A. Munawar, and A. Gray, “Neuro-symbolic reinforcement learning with first-order logic,” arXiv preprint arXiv:2110.10963, 2021.
W. Xiong, T. Hoang, and W. Y. Wang, “Deeppath: A reinforcement learning method for knowledge graph reasoning,” arXiv preprint arXiv:1707.06690, 2017.
R. Das, S. Dhuliawala, M. Zaheer, L. Vilnis, I. Durugkar, A. Krishnamurthy, A. Smola, and A. McCallum, “Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning,” arXiv preprint arXiv:1711.05851, 2017.
O. Bastani, Y. Pu, and A. Solar-Lezama, “Verifiable reinforcement learning via policy extraction,” Advances in neural information processing systems, vol. 31, 2018.
G. Anderson, A. Verma, I. Dillig, and S. Chaudhuri, “Neurosymbolic reinforcement learning with formally verified exploration,” Advances in neural information processing systems, vol. 33, pp. 6172–6183, 2020.
A. Velasquez, B. Bissey, L. Barak, A. Beckus, I. Alkhouri, D. Melcer, and G. Atia, “Dynamic automaton-guided reward shaping for monte carlo tree search,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 13, 2021, pp. 12 015–12 023.
A. Velasquez, B. Bissey, L. Barak, D. Melcer, A. Beckus, I. Alkhouri, and G. Atia, “Multi-agent tree search with dynamic reward shaping,” in Proceedings of the International Conference on Automated Planning and Scheduling, vol. 32, 2022, pp. 652–661.
M. Kazemi, M. Perez, F. Somenzi, S. Soudjani, A. Trivedi, and A. Velasquez, “Translating omega-regular specifications to average objectives for model-free reinforcement learning,” in Proceedings of the 21st international conference on autonomous agents and multiagent systems, 2022, pp. 732–741.
A. Verma, H. Le, Y. Yue, and S. Chaudhuri, “Imitation-projected programmatic reinforcement learning,” Advances in Neural Information Processing Systems, vol. 32, 2019.
A. Verma, V. Murali, R. Singh, P. Kohli, and S. Chaudhuri, “Programmatically interpretable reinforcement learning,” in International Conference on Machine Learning. PMLR, 2018, pp. 5045–5054.
J. P. Inala, O. Bastani, Z. Tavares, and A. Solar-Lezama, “Synthesizing programmatic policies that inductively generalize,” in 8th International Conference on Learning Representations, 2020.
D. Trivedi, J. Zhang, S.-H. Sun, and J. J. Lim, “Learning to synthesize programs as interpretable and generalizable policies,” Advances in neural information processing systems, vol. 34, pp. 25 146–25 163, 2021.
M. Hasanbeig, N. Y. Jeppu, A. Abate, T. Melham, and D. Kroening, “Deepsynth: Automata synthesis for automatic task segmentation in deep reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, 2021, pp. 7647–7656.
A. Silva and M. Gombolay, “Encoding human domain knowledge to warm start reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 6, 2021, pp. 5042–5050.
D. Lyu, F. Yang, B. Liu, and S. Gustafson, “Sdrl: interpretable and data-efficient deep reinforcement learning leveraging symbolic planning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 2970–2977.
S. Zhu, I. Ng, and Z. Chen, “Causal discovery with reinforcement learning,” arXiv preprint arXiv:1906.04477, 2019.
M. P. Deisenroth, G. Neumann, J. Peters et al., “A survey on policy search for robotics,” Foundations and Trends® in Robotics, vol. 2, no. 1–2, pp. 1–142, 2013.
C. Belta, A. Bicchi, M. Egerstedt, E. Frazzoli, E. Klavins, and G. J. Pappas, “Symbolic planning and control of robot motion [grand challenges of robotics],” IEEE Robotics & Automation Magazine, vol. 14, no. 1, pp. 61–70, 2007.
J. Sun, H. Sun, T. Han, and B. Zhou, “Neuro-symbolic program search for autonomous driving decision module design,” in Conference on Robot Learning. PMLR, 2021, pp. 21–30.
T. Silver, A. Athalye, J. B. Tenenbaum, T. Lozano-Perez, and L. P. Kaelbling, “Learning neuro-symbolic skills for bilevel planning,” arXiv preprint arXiv:2206.10680, 2022.
O. Ahmed, F. Träuble, A. Goyal, A. Neitz, Y. Bengio, B. Schölkopf, M. Wüthrich, and S. Bauer, “Causalworld: A robotic manipulation benchmark for causal structure and transfer learning,” arXiv preprint arXiv:2010.04296, 2020.
D. Zha, J. Xie, W. Ma, S. Zhang, X. Lian, X. Hu, and J. Liu, “Douzero: Mastering doudizhu with self-play deep reinforcement learning,” in International Conference on Machine Learning. PMLR, 2021, pp. 12 333–12 344.
LiuQiao, LiYang, DuanHong, LiuYao, and QinZhiguang, “Knowledge graph construction techniques,” Journal of Computer Research and Development, vol. 53, p. 582, 2016.
J. L. Martinez-Rodriguez, I. Lopez-Arevalo, and A. B. Rios-Alvarado, “Openie-based approach for knowledge graph construction from text,” Expert Systems with Applications, vol. 113, pp. 339–355, 2018.
S. Ji, S. Pan, E. Cambria, P. Marttinen, and S. Y. Philip, “A survey on knowledge graphs: Representation, acquisition, and applications,” IEEE transactions on neural networks and learning systems, vol. 33, no. 2, pp. 494–514, 2021.
A. Karimi and P. S. Duggirala, “Formalizing traffic rules for uncontrolled intersections,” in 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS). IEEE, 2020, pp. 41–50.
J. Barnat, L. Brim, M. Češka, and P. Ročkai, “Divine: Parallel distributed model checker,” in 2010 ninth international workshop on parallel and distributed methods in verification, and second international workshop on high performance computational systems biology. IEEE, 2010, pp. 4–7.
C. Barrett, A. Stump, C. Tinelli et al., “The smt-lib standard: Version 2.0,” in Proceedings of the 8th international workshop on satisfiability modulo theories (Edinburgh, UK), vol. 13, 2010, p. 14.
S. Chaudhuri, K. Ellis, O. Polozov, R. Singh, A. Solar-Lezama, Y. Yue et al., “Neurosymbolic programming,” Foundations and Trends® in Programming Languages, vol. 7, no. 3, pp. 158–243, 2021.
S. Harnad, “The symbol grounding problem,” Physica D: Nonlinear Phenomena, vol. 42, no. 1-3, pp. 335–346, 1990.
R. J. Mooney, “Learning to connect language and perception.” in AAAI. Chicago, 2008, pp. 1598–1601.
Kamal Acharya (Graduate Student Member, IEEE) received his Engineering degree in Electronics and Communication Engineering from Tribhuvan University, Kathmandu, Nepal in 2011 and Masters degree in Information System Engineering from Purbanchal University, Kathmandu, Nepal in 2019. Currently, he is pursuing PhD. in Information Systems from University of Maryland, Baltimore County (UMBC), Baltimore, MD. He has been involved in teaching profession for about 7 years in the various universities of Nepal, Tribhuvan University and Purbanchal Univesity were among few of them. He is mainly associated with the courses like programming(C,C++,Python), Computer Networks and Computer Architecture. He is working as Graduate Research Assistant in UMBC. He is also serving as an reviewer for IEEE Transactions on Artificial Intelligence (TAI) and IEEE Transactions on Intelligent Transportation Systems. His preferred areas of research are Natural Language Processing(NLP), Deep Learning and Reinforcemnt Learning.
Waleed Raza received a B.E. degree in Electronic Engineering from the Department of Electronic Engineering, Dawood University of Engineering and Technology Karachi, Pakistan in 2017. He received an M.S. degree in underwater acoustic communication engineering from the College of underwater acoustic engineering, Harbin Engineering University, Harbin China where he researched OFDM communication for underwater technology. He holds the editorial board member for Engineering, Technology, and Applied Science Research (ETASR) (2021-present), He is an active reviewer of a few journals including IEEE Sensor Journal, IEEE Access, and International Journal of Electronics and Communications (2018-2022), he has recently joined the IEEE as a student member. Currently, he is pursuing a Ph.D. degree at Embry Riddle Aeronautical University in Electrical and Computer Engineering. His research area of interest includes underwater acoustic OFDM communication, underwater acoustic target detection, artificial intelligence, machine learning for communication engineering, and autonomous unmanned systems such as UAVs and their characteristics.
Carlos M. J. M. Dourado Jr received the Ph.D. degree in Informatics from the University of Fortaleza, Ceara, Brazil in February 2019 and MSc in Teleinformatics Engineering from the PPGETI/UFC (UFC, 2008). He completed a BSe in Electronics Engineering at the University of Fortaleza (Unifor, 2004). He is an associate professor and researcher at the Department of Telematics (DTEL)/Graduate Program in Computer Engineering (PPGCC) at the Federal Institutte of Ceara (IFCE), Brazil and PostDoctoral Research Associate in Embry-Riddle Aeronautical University. His main research areas include Internet of Things and Artificial Intelligence.
Alvaro Velasquez is a program manager in the Innovation Information Office (I2O) of the Defense Advanced Research Projects Agency (DARPA), where he currently leads the Assured Neuro-Symbolic Learning and Reasoning (ANSR) program. Before that, Alvaro oversaw the machine intelligence portfolio of investments for the Information Directorate of the Air Force Research Laboratory (AFRL). Alvaro received his PhD in Computer Science from the University of Central Florida and is a recipient of the National Science Foundation Graduate Research Fellowship Program (NSF GRFP) award, the University of Central Florida 30 Under 30 award, and best paper and patent awards from AFRL. He has co-authored 60 papers and two patents and serves as Associate Editor of the IEEE Transactions on Artificial Intelligence and his research has been funded by the Air Force Office of Scientific Research.
Houbing Song (M’12–SM’14-F’23) received the Ph.D. degree in electrical engineering from the University of Virginia, Charlottesville, VA, in August 2012. He is currently a Tenured Associate Professor, the Director of NSF Center for Aviation Big Data Analytics (Planning), the Associate Director for Leadership of the DOT Transportation Cybersecurity Center for Advanced Research and Education (Tier 1 Center), and the Director of the Security and Optimization for Networked Globe Laboratory (SONG Lab, www.SONGLab.us), University of Maryland, Baltimore County (UMBC), Baltimore, MD. Prior to joining UMBC, he was a Tenured Associate Professor of Electrical Engineering and Computer Science at Embry-Riddle Aeronautical University, Daytona Beach, FL. He serves as an Associate Editor for IEEE Transactions on Artificial Intelligence (TAI) (2023-present), IEEE Internet of Things Journal (2020-present), IEEE Transactions on Intelligent Transportation Systems (2021-present), and IEEE Journal on Miniaturization for Air and Space Systems (J-MASS) (2020-present). He was an Associate Technical Editor for IEEE Communications Magazine (2017-2020). He is the editor of eight books, the author of more than 100 articles and the inventor of 2 patents. His research interests include cyber-physical systems/internet of things, cybersecurity and privacy, and AI/machine learning/big data analytics. His research has been sponsored by federal agencies (including National Science Foundation, US Department of Transportation, and Federal Aviation Administration, among others) and industry. His research has been featured by popular news media outlets, including IEEE GlobalSpec’s Engineering360, Association for Uncrewed Vehicle Systems International (AUVSI), Security Magazine, CXOTech Magazine, Fox News, U.S. News & World Report, The Washington Times, and New Atlas. Dr. Song is an IEEE Fellow and an ACM Distinguished Member. He has been an ACM Distinguished Speaker (2020-present), an IEEE Vehicular Technology Society (VTS) Distinguished Lecturer (2023-present) and an IEEE Systems Council Distinguished Lecturer (2023-present). Dr. Song has been a Highly Cited Researcher identified by Clarivate™ (2021, 2022). Dr. Song received Research.com Rising Star of Science Award in 2022, 2021 Harry Rowe Mimno Award bestowed by IEEE Aerospace and Electronic Systems Society, and 10+ Best Paper Awards from major international conferences, including IEEE CPSCom-2019, IEEE ICII 2019, IEEE/AIAA ICNS 2019, IEEE CBDCom 2020, WASA 2020, AIAA/ IEEE DASC 2021, IEEE GLOBECOM 2021 and IEEE INFOCOM 2022.
Metadata record
One description, two standard projections
Built from what the sources declared and what the gates observed.
Nothing absent has been filled in here.
1 value read out of the text by the enrichment rules and 33 links to or from other resources — a lab's files, the pages it links, the works it cites stand under their elements, marked inferred, and are kept apart in the exports.
DCMI Metadata Terms. Dublin Core has no element that separates the original file from the text extracted out of it, and none for LOM's educational characterisation. Both survive here as provenance statements and in the record itself, not in the projection.
the standard ↗
The groups below are this library's, for reading. DCMI Terms itself has no categories; each term keeps its standard name.
The area of Neurosymbolic Artificial Intelligence (Neurosymbolic AI) is rapidly developing and has become a popular research topic, encompassing sub-fields such as Neurosymbolic Deep Learning (Neurosymbolic DL) and Neurosymbolic Reinforcement Learning (Neurosymbolic RL). Compared to traditional learning methods, Neurosymbolic AI offers significant advantages by simplifying complexity and providing transparency and explainability. Reinforcement Learning(RL), a long-standing Artificial Intelligence(AI) concept that mimics human behavior using rewards and punishment, is a fundamental component of Neurosymbolic RL, a recent integration of the two fields that has yielded promising results. The aim of this paper is to contribute to the emerging field of Neurosymbolic RL by conducting a literature survey. Our evaluation focuses on the three components that constitute Neurosymbolic RL: neural, symbolic, and RL. We categorize works based on the role played by the neural and symbolic parts in RL, into three taxonomies:Learning for Reasoning, Reasoning for Learning and Learning-Reasoning. These categories are further divided into sub-categories based on their applications. Furthermore, we analyze the RL components of each research work, including the state space, action space, policy module, and RL algorithm. Additionally, we identify research opportunities and challenges in various applications within this dynamic field.
The abstract, and the depositor's additional notes after it when the source has a field for them.
Where the work was published, in the source's own words: a journal with its volume and pages, a conference, an imprint.
Relations and custody
1/9
Is part ofdcterms:isPartOf
none found — the source did not declare it
The repository, book or record this resource was found inside.
Has partdcterms:hasPart
none found — the source did not declare it
What this resource is made of, when the source lists its parts.
Is version ofdcterms:isVersionOf
none found — the source did not declare it
Has versiondcterms:hasVersion
none found — the source did not declare it
Referencesdcterms:references
inferred
read its reference list, line 354citation
arXiv:2104.06890
arXiv:1711.03902
arXiv:2210.15889
arXiv:2111.08164
arXiv:2209.12618
arXiv:1701.07274
arXiv:2302.05209
arXiv:1906.03926
doi:10.1145/3477600
arXiv:1702.07800
arXiv:1312.5602
arXiv:1509.02971
and 21 more, every one of them in the export
Works this one cites, when the source declares them as relations. What its text links to and its reference list cites is inferred, and stands under it apart.
Is referenced bydcterms:isReferencedBy
none found — the source did not declare it
What links to or cites this one, when a source declares it; inferred from the texts held otherwise, and set apart.
Requiresdcterms:requires
none found — the source did not declare it
What this resource needs in order to be used, when a source declares it. The files a lab works on are inferred, and stand under it apart.
Is required bydcterms:isRequiredBy
none found — the source did not declare it
What needs this resource, when a source declares it. The lab a component belongs to is inferred, and stands under it apart.
Provenancedcterms:provenance
this engine
source
conversion
latexml-html
Retrieved from arXiv on 2026-10-09 in response to the search string “(all:"artificial intelligence" OR all:"machine learning" OR all:"generative AI" OR all:"deep learning" OR all:"reinforcement learning" OR all:"large language model") AND (all:"AI concepts" OR all:"types of AI" OR all:"AI fundamentals" OR all:"recognizing AI" OR all:"recognising AI" OR all:"general versus narrow AI" OR all:"narrow AI" OR all:"general AI" OR all:"machine intelligence" OR all:"AI strengths and weaknesses" OR all:"traditional software" OR all:"rule-based systems" OR all:"introduction to AI" OR all:"introduction to artificial intelligence" OR all:"artificial intelligence introduction" OR all:"AI primer" OR all:"foundations of artificial intelligence" OR all:"overview of AI" OR all:"understanding AI" OR all:"history of AI" OR all:"AI essentials" OR all:"AI terminology" OR all:"metaphors for AI" OR all:"AI fundamental concepts" OR all:"AI key concepts" OR all:"philosophy of AI" OR all:"critical AI literacy")”. arXiv served the resource and is not asserted to be its publisher or author.
Text extracted from latexml-html to Markdown by arxiv-html; the original is retained unchanged beside it.
Where it was collected from, what was converted, and what container it came out of — the custody statements that would otherwise be mistaken for authorship.
IEEE 1484.12.1 Learning Object Metadata. LOM has no element for an SPDX identifier or a licence URI, so both are written into 6.3 Rights.Description. Flattening this record into simple Dublin Core would lose more again, which is why the two projections exist side by side rather than one being generated from the other.
the standard ↗
The area of Neurosymbolic Artificial Intelligence (Neurosymbolic AI) is rapidly developing and has become a popular research topic, encompassing sub-fields such as Neurosymbolic Deep Learning (Neurosymbolic DL) and Neurosymbolic Reinforcement Learning (Neurosymbolic RL). Compared to traditional learning methods, Neurosymbolic AI offers significant advantages by simplifying complexity and providing transparency and explainability. Reinforcement Learning(RL), a long-standing Artificial Intelligence(AI) concept that mimics human behavior using rewards and punishment, is a fundamental component of Neurosymbolic RL, a recent integration of the two fields that has yielded promising results. The aim of this paper is to contribute to the emerging field of Neurosymbolic RL by conducting a literature survey. Our evaluation focuses on the three components that constitute Neurosymbolic RL: neural, symbolic, and RL. We categorize works based on the role played by the neural and symbolic parts in RL, into three taxonomies:Learning for Reasoning, Reasoning for Learning and Learning-Reasoning. These categories are further divided into sub-categories based on their applications. Furthermore, we analyze the RL components of each research work, including the state space, action space, policy module, and RL algorithm. Additionally, we identify research opportunities and challenges in various applications within this dynamic field.
The abstract, and the depositor's additional notes after it as a second LangString when the source has a field for them.
Role, entity and date per declared contribution. A role outside LOM's vocabulary is reported in the entry's description instead.
3 Meta-metadata
4/4
Identifier3.1
this engine
resource_id
URI: tag:aim-pro.eu,2026:oer/f9cf3be0033f/record
The identifier of this metadata record — the resource's own, with /record after it, because the record is a description of the resource and not the resource.
Contribute3.2
this engine
source
AIM-PRO WP3 OER harvester (arXiv) — creator
Who generated this record and when — a statement about the record, not about the resource.
Metadata schema3.3
LOMv1.0
aimpro-oer-profile/1
LOMv1.0, and the profile this was built against.
Language3.4
language gate
read the declared field
en
4 Technical
3/7
Format4.1
conversion
latexml-html
text/markdown
One value per form held: the original as the source published it, and the Markdown this engine extracted.
Size4.2
conversion
latexml-html
94694
Bytes. The original's, because the resource is the file and not our conversion of it.
Location4.3
this engine
resource_id
https://arxiv.org/abs/2309.01038
Where the source serves it.
Requirement4.4
not collected — this library does not fill it
Software or hardware needed to use it. No source declares it.
Installation remarks4.5
not collected — this library does not fill it
No source declares it.
Other platform requirements4.6
not collected — this library does not fill it
No source declares it.
Duration4.7
not collected — this library does not fill it
Playing time, for audio and video. The corpus holds neither.
5 Educational
1/11
Interactivity type5.1
not collected — this library does not fill it
Active, expositive or mixed. A judgement about how the resource is used; no source declares it.
Yes unless the licence reserves nothing — attribution is a restriction. The conditions after the dash are the licence gate's reading; the export carries LOM's bare term.
The SPDX id, the licence URI and the conditions. LOM has no element for any of the three, so this is where they survive.
7 Relation
0/2
Kind7.1
none found — the source did not declare it
Resource7.2
inferred
read its reference list, line 354citation
references: arXiv:2104.06890
references: arXiv:1711.03902
references: arXiv:2210.15889
references: arXiv:2111.08164
references: arXiv:2209.12618
references: arXiv:1701.07274
references: arXiv:2302.05209
references: arXiv:1906.03926
references: doi:10.1145/3477600
references: arXiv:1702.07800
references: arXiv:1312.5602
references: arXiv:1509.02971
and 21 more, every one of them in the export
The container a file was found inside, and the Markdown extracted from the original. What a lab requires, and the lab a component belongs to, are inferred and stand apart.
8 Annotation
0/3
Entity8.1
not collected — this library does not fill it
Comments on the resource's educational use, by whoever made them. The platform's review grades competencies, which are classification (9), and writes no comment here.
Date8.2
not collected — this library does not fill it
Description8.3
not collected — this library does not fill it
9 Classification
0/4
Purpose9.1
not collected — this library does not fill it
Empty in the record: no source declares a competency. The alignment reads the resource for them and stands beside the record, never in it, and a taxon path derived from the search string that found it would be a claim about the query.
Taxon path9.2
not collected — this library does not fill it
Where the competency framework goes. Empty in the record for the reason above.
Description9.3
not collected — this library does not fill it
Keyword9.4
not collected — this library does not fill it
What could not be established3
Where the source's metadata could not be carried
over as it was — missing, contradictory, with no matching term in the
standard, restructured, or taken from the repository — and what was done
instead. Without these notes, an empty element would look like something
the harvester missed.
Status
Field
Why
Not available
rights_holder
no rights holder is named at source; the licence is recorded without one rather than attributed to the platform that served it
Not available
publisher
the source named no publisher of the work; where it was collected from is recorded as collection provenance instead, which is a different claim
Not available
educational
the source declared no educational metadata — no resource type, audience, context, difficulty or learning time. Nothing here estimates them
