louis_lifu
/
AReaL
mirror of https://github.com/inclusionAI/AReaL
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
222 Commits 12 Branches 4 Tags 21 MiB
Commit Graph

8 Commits

Author SHA1 Message Date
Jayon02 ef4215d6f1
[Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161) 
* refactor hf engine

* format file

* revert file format

* Squashed commit of the following:

commit 8d4b8dc90f
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date:   Thu Jul 10 13:14:10 2025 +0800

    [Doc] Add an instruction about how to run the SFT example. (#164)

commit 3bf9c85e40
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date:   Thu Jul 10 12:56:24 2025 +0800

    [Fix] Merge previous contributions from fw/refactor to lite (#163)

    * initial proposal

    * add arealite

    * .

    * change api

    * .

    * remove LOG_ROOT

    * remove MODEL_SAVE_PATH

    * remove PARAM_REALLOC_PATH, DATASET_CACHE

    * prepare for testing

    * prepare for testing

    * ready for run

    * local run

    * tests mainly pass

    * format

    * .

    * amend cluster.py

    * .

    * .

    * client test pass

    * pass rollout test

    * remove unused imports

    * add arealite readme

    * change api

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * .

    * format

    * .

    * implement iteraptable generation (#112)

    Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>

    * .

    * fix

    * .

    * .

    * .

    * pass controller generate batch test

    * .

    * refactor rollout controller into worker and controller

    * .

    * .

    * .

    * change to async rollout

    * pass rollout controller test

    * pass test

    * .

    * update readme

    * .

    * sft debug

    * .

    * add lisence

    * remove unused files

    * remove unsed args in ppo

    * add hf engine wrapper  (#116)

    * add hf engine

    * fix issues

    * fix ppo bugs and add test

    * add hf client interface and modify cli args

    * fix bugs

    * fix issues

    * Merge fw/refactor

    * Finish hf wrapper test

    * add test

    ---------

    Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>

    * format

    * format

    * .

    * refine hf engine

    * .

    * fix

    * add fsdp engine and sft tests

    * .

    * .

    * .

    * pass ppo unittest

    * pass ppo and rollout controller tests

    * clear unused imports

    * rename ppo to grpo

    * change reward function organization

    * reorganize code

    * add dataset api

    * .

    * .

    * .

    * format

    * chmod fix

    * .

    * rename workflow to collector

    * refactor llm_client location

    * .

    * .

    * fix llm server api

    * refactor config structure

    * .

    * fix tests

    * .

    * .

    * .

    * Fix unresolved issue in SFTTrainer PR (#139)

    * .

    * .

    * efficient loading

    * format

    * .

    * .

    * .

    * .

    * .

    * .

    * Add CI for testing AReaLite (#150)

    * ci: add test-arealite

    * ci: add checkout before running test-arealite

    * ci: add USERNAME

    * ci: add test script

    * ci: add GitHub mirror

    * ci: fix typo

    * ci: clone one commit

    * ci: fix condition

    * ci: set command timeout to 60m

    * ci: enable pip cache

    * ci: optimize container lifecycle

    * ci: split into many stages

    * ci(test-arealite): fix typo

    * ci: fix wrong env

    * ci: fix pytest

    * ci: uninstall transformer-engine

    * ci: uninstall transformer-engine

    * ci: fix model paths

    * ci: show stdout/stderr

    * ci: fix not clean up

    * ci: backup sglang

    * ci: remove tmp repo dir when run

    * ci: fix docker run exit 1 condition

    * ci(test-arealite): limit the concurrency and extend command timeout

    * .

    * merge fw/refactor

    * revert some changes

    * fix

    ---------

    Co-authored-by: meizhiyu.mzy <meizhiyu.mzy@antgroup.com>
    Co-authored-by: Chayenne <zhaochen20@outlook.com>
    Co-authored-by: zhaochenyang <zhaochenyang20@gmail.com>
    Co-authored-by: Jayon02 <qiujiangc@outlook.com>
    Co-authored-by: root <meizhiyu.mzy>
    Co-authored-by: Zijian Zhang <futrime@outlook.com>

commit d48bf007cf
Merge: 42c717b b9dbd4a
Author: 博惟 <bowei.fw@antgroup.com>
Date:   Thu Jul 10 12:53:30 2025 +0800

    Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite

commit 42c717b6e4
Merge: c38cffc a203c7c
Author: 博惟 <bowei.fw@antgroup.com>
Date:   Thu Jul 10 11:15:01 2025 +0800

    Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite

commit c38cffc023
Author: 博惟 <bowei.fw@antgroup.com>
Date:   Thu Jul 10 11:10:10 2025 +0800

    PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods

    Merge branch fw/lite-dev of git@code.alipay.com:inclusionAI/AReaL.git into lite
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/340

    Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>

    * support fsdp engine and sglang remote engine
    * minor fix
    * .
    * refactor trainer
    * add close
    * rm mb_spec
    * fix

commit b9dbd4a2c1
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date:   Wed Jul 9 10:50:19 2025 +0800

    Update to persistent wechat QR code. (#159)

commit 17ea7fe94d
Author: xssstory <33601810+xssstory@users.noreply.github.com>
Date:   Mon Jul 7 15:49:13 2025 +0800

    fix math reward verifier (#156)

    * PullRequest: 293 fix get_param_realloc_path

    Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/293

    Reviewed-by: 博惟 <bowei.fw@antgroup.com>

    * fix get_param_realloc_path

    * PullRequest: 297 bugfix: reward is always -5

    Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/297

    Reviewed-by: 博惟 <bowei.fw@antgroup.com>

    * bugfix: reward is always -5

    * PullRequest: 321 fix checkpoint save dir

    Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/321

    Reviewed-by: 博惟 <bowei.fw@antgroup.com>

    * fix checkpoint save dir

    * PullRequest: 328 [Doc] update installation

    Merge branch sxj/doc of git@code.alipay.com:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/328

    Reviewed-by: 博惟 <bowei.fw@antgroup.com>

    * [Doc] update installation

    * PullRequest: 329 bugfix: math verifier blocks the async training

    Merge branch xss/debug of git@code.alipay.com:inclusionAI/AReaL.git into gh
    https://code.alipay.com/inclusionAI/AReaL/pull_requests/329

    Reviewed-by: 博惟 <bowei.fw@antgroup.com>

    * bugfix: math verifier block the async training

    * format

    ---------

    Co-authored-by: 冰临 <shenxujie.sxj@antgroup.com>
    Co-authored-by: garrett4wade <fuwth17@gmail.com>

* add autotp for hf

* refactor test

* fix bugs

* fix issues

* format files

* Squashed commit of the following:

commit 9ed043f6ab
Author: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
Date:   Tue Jul 15 10:24:48 2025 +0800

    format (#174)

commit 8cc9b1feb5
Author: Night <32424487+PrinsYin@users.noreply.github.com>
Date:   Mon Jul 14 19:22:00 2025 -0700

    added LocalSGlangEngine and test (#170)

    * added LocalSGLangEngine

    * upload test file

    * add build args

    * fix sgl_local generate

    * improved sgl local robustness

    * test

    * test updated

    * added fallback when sgl engine isn't initialized

    * finish test local engine

    * added LocalSGlangEngine and test

    * format and fix

    format and fix, raise when generate missing field

    format

    * change cli_args.py

    * add comment header

    format

    ---------

    Co-authored-by: ChangyiYang <changyiyang2023@gmail.com>

---------

Co-authored-by: Jayon02 <12012211@mail..sustech.edu.cn>
Co-authored-by: Wei Fu <36355462+garrett4wade@users.noreply.github.com>
 2025-07-16 12:44:10 +08:00
博惟 7be4ab0d18 PullRequest: 339 [Fix] Fix some minor issues to pass all tests. 
Merge branch fw/lite of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/339

Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>


* support fsdp engine and sglang remote engine
* minor fix
* .
 2025-07-09 16:51:26 +08:00
博惟 15dfbe837c PullRequest: 332 [lite] Support FSDP engines 
Merge branch mzy/lite/engines of git@code.alipay.com:inclusionAI/AReaL.git into lite
https://code.alipay.com/inclusionAI/AReaL/pull_requests/332

Reviewed-by: 晓雷 <meizhiyu.mzy@antgroup.com>


* fsdp2 engine
* fix utils
* add fsdp engine test
* .
* fsdp engine test passed
* unsqueeze immediately before model inputs and after model outputts
* add optimizer save/load, add position id calculation for input
* .
* format
* not to squeeze
* add train and eval api
* .
* .
* improve fsdp engine data preprocessing
* format
* PullRequest: 337 [lite] Add SFT trainer example.
* trainer log
* minor changes
* add update weights from disk
* fix type annotation
 2025-07-09 16:24:25 +08:00
garrett4wade cf0db6a43c format 2025-07-07 17:02:32 +08:00
garrett4wade 7ab6755379 Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite 2025-07-07 17:02:21 +08:00
博惟 e251abbbe8 add remote sglang engine 2025-07-07 17:01:35 +08:00
garrett4wade b6e19dbf60 format 2025-07-07 14:02:59 +08:00
博惟 1dfe91c470 add api 2025-07-07 13:47:41 +08:00