988b08561d
.
2025-07-22 15:09:53 +08:00
4e9715cb41
.
2025-07-22 13:48:44 +08:00
c661f8fe24
.
2025-07-22 13:14:55 +08:00
9f822a9275
finish code walkthrough doc
2025-07-22 13:10:49 +08:00
92495df9e3
add quickstart
2025-07-21 18:48:10 +08:00
4804b05637
[Refactor] Rename files in arealite before release. ( #190 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
* rename
* rename
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-21 12:57:50 +08:00
18f8a056b6
[Feature] [lite] Merge from internal dev repo ( #189 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher
Merge branch mzy/lite/launcher of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/370
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* .
* .
* .
* fix
* PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results.
Merge branch fw/lite-boba of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/392
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* .
* fix
* .
* qwen2 grpo works
* fix
* fix
* async works
* fix
* slurm launcher not tested
* fix arg parse
* .
* sglang server wrapper
* .
* .
* slurm run
* ready for boba
* debug
* 32k run
* .
* .
* fix
* .
* .
* .
* .
* .
* fix
* .
* fix
* .
* .
* .
* .
* fix
* .
* .
* .
* .
* .
* .
* .
* refactor train engine
* refactor train engine
* .
* fix update weight error
* .
* .
* match train
* format
* .
* fix
* seems to work
* .
* .
* .
* .
* format
* format
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-21 12:52:43 +08:00
f68a4f677d
Implement fsdp distributed update ( #183 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* added remote nccl weight update
feat: implement update_weights_from_distributed in fsdp_engine.py
unfinishd test, raise PR first
coroutine for each server
chore: change uploads weights behavior, change test order
fix small bug
fixed test
* fix rebase
* add test.sh
* updated, test stil fails
* .
* .
* .
* fix: full_tensor() should happen in all rank (#187 )
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
Co-authored-by: 博惟 <bowei.fw@antgroup.com>
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
Co-authored-by: ChangyiYang <112288487+ChangyiYang@users.noreply.github.com>
2025-07-21 12:42:00 +08:00
29e164a69d
[Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine ( #181 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
* PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub
Merge branch fw/lite-train-engine of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/368
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation
Merge branch fw/lite-fix0716 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/371
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-16 17:26:49 +08:00
0283cfa124
change doc ( #180 )
2025-07-16 17:24:30 +08:00
e13db01f67
[lite] [refactor] Add GSM8k GRPO example. ( #179 )
...
* PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine
Merge branch mzy/add-gradient-ckpt of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/353
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add gradient checkpointing
* PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/354
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration
Merge branch fw/lite-fix1 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* .
* fix
* .
* PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities
Merge branch fw/lite-fix2 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* .
* fix destroy process group
* fix ci
* PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset
Merge branch fw/lite-fix3 of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/358
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* .
* .
* .
* fix loss mask
* fix
* .
---------
Co-authored-by: 晓雷 <meizhiyu.mzy@antgroup.com>
2025-07-16 13:10:26 +08:00
4490b117e4
[Feature] Add pre-commit ( #178 )
...
* fix ci
* .
* .
* .
2025-07-16 12:50:07 +08:00
517353c065
fix ci ( #175 )
2025-07-16 12:44:38 +08:00
ef4215d6f1
[Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. ( #161 )
...
* refactor hf engine
* format file
* revert file format
* Squashed commit of the following:
commit 8d4b8dc90f#164 )
commit 3bf9c85e40#163 )
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112 )
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116 )
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139 )
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150 )
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
commit d48bf007cf42c717bb9dbd4ahttps://github.com/inclusionAI/AReaL into lite
commit 42c717b6e4c38cffca203c7chttps://github.com/inclusionAI/AReaL into lite
commit c38cffc023https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
commit b9dbd4a2c1#159 )
commit 17ea7fe94d#156 )
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
* add autotp for hf
* refactor test
* fix bugs
* fix issues
* format files
* Squashed commit of the following:
commit 9ed043f6ab#174 )
commit 8cc9b1feb5#170 )
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
---------
Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
2025-07-16 12:44:10 +08:00
9ed043f6ab
format ( #174 )
2025-07-15 10:24:48 +08:00
8cc9b1feb5
added LocalSGlangEngine and test ( #170 )
...
* added LocalSGLangEngine
* upload test file
* add build args
* fix sgl_local generate
* improved sgl local robustness
* test
* test updated
* added fallback when sgl engine isn't initialized
* finish test local engine
* added LocalSGlangEngine and test
* format and fix
format and fix, raise when generate missing field
format
* change cli_args.py
* add comment header
format
---------
Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>
2025-07-15 10:22:00 +08:00
c5f023571e
[Fix] Fix CI running condition for lite. ( #172 )
...
* .
* fix
* .
* .
* fix
* fix
* fix
2025-07-12 14:47:56 +08:00
8d4b8dc90f
[Doc] Add an instruction about how to run the SFT example. ( #164 )
2025-07-10 13:14:10 +08:00
3bf9c85e40
[Fix] Merge previous contributions from fw/refactor to lite ( #163 )
...
* initial proposal
* add arealite
* .
* change api
* .
* remove LOG_ROOT
* remove MODEL_SAVE_PATH
* remove PARAM_REALLOC_PATH, DATASET_CACHE
* prepare for testing
* prepare for testing
* ready for run
* local run
* tests mainly pass
* format
* .
* amend cluster.py
* .
* .
* client test pass
* pass rollout test
* remove unused imports
* add arealite readme
* change api
* .
* .
* .
* .
* .
* .
* .
* .
* format
* .
* implement iteraptable generation (#112 )
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
* .
* fix
* .
* .
* .
* pass controller generate batch test
* .
* refactor rollout controller into worker and controller
* .
* .
* .
* change to async rollout
* pass rollout controller test
* pass test
* .
* update readme
* .
* sft debug
* .
* add lisence
* remove unused files
* remove unsed args in ppo
* add hf engine wrapper (#116 )
* add hf engine
* fix issues
* fix ppo bugs and add test
* add hf client interface and modify cli args
* fix bugs
* fix issues
* Merge fw/refactor
* Finish hf wrapper test
* add test
---------
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
* format
* format
* .
* refine hf engine
* .
* fix
* add fsdp engine and sft tests
* .
* .
* .
* pass ppo unittest
* pass ppo and rollout controller tests
* clear unused imports
* rename ppo to grpo
* change reward function organization
* reorganize code
* add dataset api
* .
* .
* .
* format
* chmod fix
* .
* rename workflow to collector
* refactor llm_client location
* .
* .
* fix llm server api
* refactor config structure
* .
* fix tests
* .
* .
* .
* Fix unresolved issue in SFTTrainer PR (#139 )
* .
* .
* efficient loading
* format
* .
* .
* .
* .
* .
* .
* Add CI for testing AReaLite (#150 )
* ci: add test-arealite
* ci: add checkout before running test-arealite
* ci: add USERNAME
* ci: add test script
* ci: add GitHub mirror
* ci: fix typo
* ci: clone one commit
* ci: fix condition
* ci: set command timeout to 60m
* ci: enable pip cache
* ci: optimize container lifecycle
* ci: split into many stages
* ci(test-arealite): fix typo
* ci: fix wrong env
* ci: fix pytest
* ci: uninstall transformer-engine
* ci: uninstall transformer-engine
* ci: fix model paths
* ci: show stdout/stderr
* ci: fix not clean up
* ci: backup sglang
* ci: remove tmp repo dir when run
* ci: fix docker run exit 1 condition
* ci(test-arealite): limit the concurrency and extend command timeout
* .
* merge fw/refactor
* revert some changes
* fix
---------
Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
Co-authored-by: Jayon02 <qiujiangc@outlook.com>
Co-authored-by: root <meizhiyu.mzy>
Co-authored-by: Zijian Zhang <futrime@outlook.com>
2025-07-10 12:56:24 +08:00
d48bf007cf
Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite
2025-07-10 12:53:30 +08:00
42c717b6e4
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
2025-07-10 11:15:01 +08:00
c38cffc023
PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods
...
Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/340
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
* refactor trainer
* add close
* rm mb_spec
* fix
2025-07-10 11:10:10 +08:00
a203c7c750
ci: fix env sha
2025-07-09 22:31:46 +08:00
a70cd28f8f
ci: fix on demand condition
2025-07-09 21:19:15 +08:00
8e201ef3e2
ci: build images on demand
2025-07-09 21:16:23 +08:00
ee6f5a89f3
chore: empty commit
2025-07-09 18:52:38 +08:00
7be4ab0d18
PullRequest: 339 [Fix] Fix some minor issues to pass all tests.
...
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* support fsdp engine and sglang remote engine
* minor fix
* .
2025-07-09 16:51:26 +08:00
15dfbe837c
PullRequest: 332 [lite] Support FSDP engines
...
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
2025-07-09 16:24:25 +08:00
7a438c0650
PullRequest: 336 add wrapper
...
Merge branch lite-util-wrapper of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/336
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* add wrapper
2025-07-09 15:44:27 +08:00
8771778995
PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases.
...
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/331
Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>
* .
* add test for sglang remote engine
* fix
2025-07-09 14:18:55 +08:00
b9dbd4a2c1
Update to persistent wechat QR code. ( #159 )
2025-07-09 10:50:19 +08:00
57b9b945ab
add readme
2025-07-07 17:46:46 +08:00
cf0db6a43c
format
2025-07-07 17:02:32 +08:00
7ab6755379
Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite
2025-07-07 17:02:21 +08:00
e251abbbe8
add remote sglang engine
2025-07-07 17:01:35 +08:00
3b2f43a295
checkout prev impl
2025-07-07 15:53:34 +08:00
95c315e0b8
checkout previous implementations
2025-07-07 15:52:44 +08:00
3a0f1e558c
checkout previous impl
2025-07-07 15:51:32 +08:00
17ea7fe94d
fix math reward verifier ( #156 )
...
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
* PullRequest: 329 bugfix: math verifier blocks the async training
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/329
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: math verifier block the async training
* format
---------
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
Co-authored-by: garrett4wade <fuwth17@gmail.com>
2025-07-07 15:49:13 +08:00
6710d5f275
Merge branch 'lite' of https://code.alipay.com/inclusionAI/AReaL into lite
2025-07-07 15:35:02 +08:00
b6e19dbf60
format
2025-07-07 14:02:59 +08:00
28c9479981
add tests template
2025-07-07 13:51:23 +08:00
1006be8a01
add directory structure
2025-07-07 13:49:54 +08:00
1dfe91c470
add api
2025-07-07 13:47:41 +08:00
5b7c83b5d9
update qrcode ( #153 )
2025-07-07 12:28:21 +08:00
0ff8c59435
[Fix] Merge error fixes. ( #152 )
...
* PullRequest: 293 fix get_param_realloc_path
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
* PullRequest: 297 bugfix: reward is always -5
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
* PullRequest: 321 fix checkpoint save dir
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
* PullRequest: 328 [Doc] update installation
Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/328
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* [Doc] update installation
---------
Co-authored-by: 温差 <xushusheng.xss@antgroup.com>
Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
2025-07-07 10:30:27 +08:00
ed6bcd66a7
PullRequest: 321 fix checkpoint save dir
...
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/321
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix checkpoint save dir
2025-07-03 10:03:33 +08:00
e0aee03109
Fix several syntax warning related to invalid escape sequence ( #144 )
...
by using raw strings or properly escaping the backslashes.
```log
AReaL/realhf/impl/dataset/math_parser.py:292: SyntaxWarning: invalid escape sequence '\%'
string = string.replace("\%", "")
AReaL/realhf/impl/dataset/math_parser.py:402: SyntaxWarning: invalid escape sequence '\d'
pattern = "-?\d*\.?\d+"
AReaL/realhf/impl/model/parallelism/tensor_parallel/modules.py:1125: SyntaxWarning: invalid escape sequence '\s'
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
2025-07-01 10:12:39 +08:00
623f7c7407
PullRequest: 297 bugfix: reward is always -5
...
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/297
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* bugfix: reward is always -5
2025-06-27 13:06:37 +08:00
a5cabddcea
PullRequest: 293 fix get_param_realloc_path
...
Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
https://code.alipay.com/inclusionAI/AReaL/pull_requests/293
Reviewed-by: 博惟 <bowei.fw@antgroup.com>
* fix get_param_realloc_path
2025-06-26 21:26:56 +08:00
晓雷