Improving reasoning capabilities in Large Language Models: techniques and experiences

Abstract

In this talk, I will present recent advancements and effective methodologies aimed at enhancing the reasoning capabilities of Large Language Models (LLMs). Specifically, our innovative approaches leveraging reinforcement learning (RL) will be discussed, highlighting emerging techniques that significantly improve the mathematical reasoning proficiency of AI systems.

Additionally, I will share insights and experiences from our research journey, emphasizing both common challenges and unexpected breakthroughs. The discussion will conclude with my perspectives on promising future directions, particularly the development and integration of intelligent agents and reinforcement learning frameworks to specialize in the reasoning abilities of LLMs.

Speaker’s profile

Allan Jie is a Research Scientist at ByteDance Research, focusing on improving the mathematical reasoning in Large Language Models (LLMs). His work explores innovative reinforcement learning methodologies improve the problem-solving capabilities of LLMs.