OER·harvester

← Back to the library
arXiv HTML resource

The Duality of Generative AI and Reinforcement Learning in Robotics: A Review

Recently, generative AI and reinforcement learning (RL) have been redefining what is possible for AI agents that take information flows as input and produce intelligent behavior. As a result, we are seeing similar advancements in embodied AI and robotics for control policy generation. Our review paper examines the integration of generative AI models with RL to advance robotics. Our primary focus is on the duality be…

Licence
OPEN CC-BY-4.0
Authors
Angelo Moroncelli, Vishal Soni, Marco Forgione, Dario Piga, Blerina S…
Published
2024-10-21 · arXiv
Language
en
Length
18858 words
Type
narrative text

Cites 98 works

inferred
Open original ↗

10 Conclusion

The integration of generative AI models and RL marks a transformative advancement in robotics, enhancing reasoning and adaptability in complex tasks [36, 151, 101, 103, 163, 94, 71, 66]. This synergy combines the extensive knowledge and in-context learning of foundation models, the generative capabilities on multi-modal input fusion of generative AI and the high learning capacity in decision-making of RL in robots. Our review critically examined this integration, with a focus on duality aspects between generative AI and RL for robot action generation; extracting policies from text, video and states information. We first studied how Generative Tools for RL consist in a solid portion of research papers where LLMs and VLMs play a major role, and how the union of diffusion models and RL is a very exciting trend in this direction, that deserves further investigation. Moreover, the exploration of how RL can harness foundation world models to enhance model-based RL training further highlights the potential of this integration. Finally, we categorized preliminary work on the other side of our duality investigation—RL for Generative Policies—and we focused on combining generative AI with RL in training, fine-tuning, and distilling generative policies. While the opportunities for such a research field are vast, we highlighted remaining challenges, notably in grounding abstract representations of foundation models to physical world concepts—an area needing significant exploration. We concluded that RL-based fine-tuning of robotic generative policies, along with the exploration of failure modes of black-box policies, remains underexplored. While, the scalability and computational demands of these systems pose substantial challenges, especially in resource-constrained environments, limiting the possibility of online training. Addressing these challenges is critical for building fully autonomous robots, controlled by generative AI models able to create policies from multi-modal inputs. Based on our findings, we shared promising research directions for the coming years.