From ad6e5bd3fa5e9126912ea7a28149471b53b8e444 Mon Sep 17 00:00:00 2001 From: Wei Fu <36355462+garrett4wade@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:10:32 +0800 Subject: [PATCH] [Doc] Add verl benchmark scripts (#71) * update benchmark script * . * add benchmark docs * . --- .../README.md | 8 +++++++- .../areal_config.yaml | 0 .../build_cmd.py | 0 .../scaling_trend_vs_verl.png | Bin 4 files changed, 7 insertions(+), 1 deletion(-) rename benchmark/{verl_v0_3_0_post1_312a8cb => verl_v0_3_0_post1_76084d3}/README.md (90%) rename benchmark/{verl_v0_3_0_post1_312a8cb => verl_v0_3_0_post1_76084d3}/areal_config.yaml (100%) rename benchmark/{verl_v0_3_0_post1_312a8cb => verl_v0_3_0_post1_76084d3}/build_cmd.py (100%) rename benchmark/{verl_v0_3_0_post1_312a8cb => verl_v0_3_0_post1_76084d3}/scaling_trend_vs_verl.png (100%) diff --git a/benchmark/verl_v0_3_0_post1_312a8cb/README.md b/benchmark/verl_v0_3_0_post1_76084d3/README.md similarity index 90% rename from benchmark/verl_v0_3_0_post1_312a8cb/README.md rename to benchmark/verl_v0_3_0_post1_76084d3/README.md index e4a9242..db94486 100644 --- a/benchmark/verl_v0_3_0_post1_312a8cb/README.md +++ b/benchmark/verl_v0_3_0_post1_76084d3/README.md @@ -8,10 +8,16 @@ We compare against the latest release of verl (v0.3.0.post1) as of May 7, 2025. ## How to Reproduce +### verl + +We provide code and instructions [in this repo](https://github.com/garrett4wade/verl-benchmark/blob/main/readme_benchmark.md). + +### AReaL + Run `build_cmd.py` to generate the CLI command to run AReaL: ```bash -python3 benchmark/verl_v0_3_1_312a8cb/build_cmd.py --model-size 1 --ctx 32768 --n-nodes 4 +python3 benchmark/verl_v0_3_1_76084d3/build_cmd.py --model-size 1 --ctx 32768 --n-nodes 4 ``` The above command generates the command to run AReaL with `DeepSeek-R1-Distill-Qwen-1.5B` using 32k context length (31k generation length) on 4 nodes (32 GPUs). You can choose `model_size` from [1, 7, 32] and `n_nodes` from [4, 8, 16, 32, 64]. diff --git a/benchmark/verl_v0_3_0_post1_312a8cb/areal_config.yaml b/benchmark/verl_v0_3_0_post1_76084d3/areal_config.yaml similarity index 100% rename from benchmark/verl_v0_3_0_post1_312a8cb/areal_config.yaml rename to benchmark/verl_v0_3_0_post1_76084d3/areal_config.yaml diff --git a/benchmark/verl_v0_3_0_post1_312a8cb/build_cmd.py b/benchmark/verl_v0_3_0_post1_76084d3/build_cmd.py similarity index 100% rename from benchmark/verl_v0_3_0_post1_312a8cb/build_cmd.py rename to benchmark/verl_v0_3_0_post1_76084d3/build_cmd.py diff --git a/benchmark/verl_v0_3_0_post1_312a8cb/scaling_trend_vs_verl.png b/benchmark/verl_v0_3_0_post1_76084d3/scaling_trend_vs_verl.png similarity index 100% rename from benchmark/verl_v0_3_0_post1_312a8cb/scaling_trend_vs_verl.png rename to benchmark/verl_v0_3_0_post1_76084d3/scaling_trend_vs_verl.png