Reinforcement Learning with Decomposed Subtasks

Recent research explores enhancing Reinforcement Learning for language model agents. One approach introduces decomposing trajectory rewards into subtasks to prevent lossy updates in Group Relative Policy Optimization (GRPO), especially during complex tasks with sparse feedback. Simultaneously, Reinforcement Learning with Verifiable Rewards (RLVR) is being tested on small models. Specifically, Qwen3.5-0.8B was trained using GRPO and a Wikipedia-search tool for open-domain question answering, demonstrating that the reason-over-search recipe can function without distillation from larger teacher models.