LoongSage Documentation#
LoongSage is a post-training reinforcement learning framework for large language models: Ray as the scheduling foundation, Megatron-Core as the training backend, SGLang as the inference backend. It aims at training ultra-large models with a minimalist architecture, zero-intrusion access for any agent, and system-level train/inference consistency guarantees.
Highlights#
Zero-intrusion agents — an agent only talks to the framework’s inference service through a standard chat-completion API; multi-turn context concatenation, loss masking, trajectory collection and sandbox lifecycles are all handled by AgentFlow.
Consistency by construction — Router Replay, FP32 output layer, TITO, importance-sampling correction and other alignment techniques keep the train/inference probability bias at the 1e-4 magnitude, see Train-Inference Consistency.
Fully asynchronous training — sampling and training are decoupled, advance in parallel and overlap in time, with configurable staleness and sliding windows, see Fully Async Mode.
Highly extensible plugin system — 12 extension points covering agents, reward models, sandboxes, advantage estimation, policy loss, asynchronous scheduling and more; add an implementation and reference it from yaml to hot-plug it in, starting with Custom Agent Development Guide.
Multi-paradigm on-policy distillation — native support for PG-style, GKD-style and hybrid distillation, with built-in TopK, full-vocabulary and JSD KL strategies, as well as multi-teacher distillation, see On-Policy Distillation.
Where to start#
Getting a first training run going? → Quick Start
How do I launch, and which run modes exist? → Run Guide
How do I plug in my own agent, reward function or sandbox? → Custom Agent Development Guide, Custom Reward Function Development Guide, Custom Sandbox Development Guide
What does a given yaml field do? → Config Reference
Getting Started
Training
Extension Guides
Reference