29e164a69d
[Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine ( #181 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-16 17:26:49 +08:00
e13db01f67
[lite] [refactor] Add GSM8k GRPO example. ( #179 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-16 13:10:26 +08:00
7be4ab0d18
PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
...
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
2025-07-09 16:51:26 +08:00
15dfbe837c
PullRequest: 332 [lite] Support FSDP engines
...
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
2025-07-09 16:24:25 +08:00
8771778995
PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
...
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
2025-07-09 14:18:55 +08:00
cf0db6a43c
format
2025-07-07 17:02:32 +08:00
e251abbbe8
add remote sglang engine
2025-07-07 17:01:35 +08:00
Wei Fu