mirror of https://github.com/inclusionAI/AReaL
add figure to explain AReaLite components
This commit is contained in:
parent
a5e55acb26
commit
3a97f06be1
10
README.md
10
README.md
|
@ -27,8 +27,8 @@ like how you enjoy real-world milk tea (cheers).
|
||||||
code. This allows users to build their own **agentic** and **RLVR** training workflows
|
code. This allows users to build their own **agentic** and **RLVR** training workflows
|
||||||
with minimal effort.
|
with minimal effort.
|
||||||
- 🔥 **Asynchronous RL**: With algorithm-system co-design, AReaL supports fully
|
- 🔥 **Asynchronous RL**: With algorithm-system co-design, AReaL supports fully
|
||||||
asynchronous RL for **the fastest training speed**! Experimental support for multi-turn
|
asynchronous RL for **the fastest training speed**! Experimental support for
|
||||||
agentic RL is also provided.
|
multi-turn agentic RL is also provided.
|
||||||
- 🛠️ **Open & Reproducible**: We continuously release _all code, datasets, and training
|
- 🛠️ **Open & Reproducible**: We continuously release _all code, datasets, and training
|
||||||
recipes_ for RL training of LLMs.
|
recipes_ for RL training of LLMs.
|
||||||
- 🚀 **Scalability**: AReaL can seamlessly adapt to different computational resource
|
- 🚀 **Scalability**: AReaL can seamlessly adapt to different computational resource
|
||||||
|
@ -69,7 +69,7 @@ New highlights in AReaLite:
|
||||||
- Instead of the *system-centric* architecture in old AReaL, AReaLite follows an
|
- Instead of the *system-centric* architecture in old AReaL, AReaLite follows an
|
||||||
**AI-centric** API design that aims to provide the following key features:
|
**AI-centric** API design that aims to provide the following key features:
|
||||||
|
|
||||||
- **Light-weight** & **easy to write** algorithm and training workflow customization.
|
- **Light-weight** & **easy-to-write** algorithm and training workflow customization.
|
||||||
- **Easy to scale up** without knowing system and infrastructure details.
|
- **Easy to scale up** without knowing system and infrastructure details.
|
||||||
- **Adaptable and plugable:** Smooth to integrate with other modern AI applications.
|
- **Adaptable and plugable:** Smooth to integrate with other modern AI applications.
|
||||||
|
|
||||||
|
@ -106,8 +106,8 @@ Our training scripts will automatically download the dataset (openai/gsm8k) and
|
||||||
python3 -m arealite.launcher.local examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml
|
python3 -m arealite.launcher.local examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
On a Ray cluster with 2 nodes & 8 GPUs each node, runs (Remember to change paths in the YAML
|
On a Ray cluster with 2 nodes & 8 GPUs each node, runs (Remember to change paths in the
|
||||||
file to your own shared storage):
|
YAML file to your own shared storage):
|
||||||
|
|
||||||
```
|
```
|
||||||
python3 -m arealite.launcher.ray examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml \
|
python3 -m arealite.launcher.ray examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml \
|
||||||
|
|
|
@ -12,9 +12,9 @@ understand, and develop with effectively. The primary issue stems from its
|
||||||
*system-centric* rather than *AI-centric* architecture and API design. An *AI-centric*
|
*system-centric* rather than *AI-centric* architecture and API design. An *AI-centric*
|
||||||
design aims to provide three key features:
|
design aims to provide three key features:
|
||||||
|
|
||||||
- **Light-weight & focused customization:** Users can implement their algorithms and
|
- **Light-weight & easy-to-write customization:** Users can implement their algorithms
|
||||||
training workflows with minimal and concentrated code, often in just a few files or
|
and training workflows with minimal and concentrated code, often in just a few files
|
||||||
even a single file.
|
or even a single file.
|
||||||
- **Easy to scale up:** Experiments can be scaled up seamlessly without requiring
|
- **Easy to scale up:** Experiments can be scaled up seamlessly without requiring
|
||||||
knowledge of underlying system or infrastructure details.
|
knowledge of underlying system or infrastructure details.
|
||||||
- **Adaptable and plugable:** Users is free to integrate the system with code or APIs
|
- **Adaptable and plugable:** Users is free to integrate the system with code or APIs
|
||||||
|
@ -80,6 +80,16 @@ arealite/
|
||||||
|
|
||||||
### Component Overview
|
### Component Overview
|
||||||
|
|
||||||
|
The AReaLite codebase is structured into four distinct layers: the API layer, backend
|
||||||
|
layer, customization layer, and entry point layer. As illustrated in the figure below,
|
||||||
|
workflow and algorithm customization logic resides in separate layers above the backend.
|
||||||
|
We prioritize keeping the entry point and customization layers clean and intuitive,
|
||||||
|
isolating them from the complex backend implementation. With AReaLite, users can define
|
||||||
|
their custom training workflows and algorithms entirely within a single entry point
|
||||||
|
file.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
#### 1. API Layer (`api/`)
|
#### 1. API Layer (`api/`)
|
||||||
|
|
||||||
The API layer establishes clean contracts between components through abstract interfaces
|
The API layer establishes clean contracts between components through abstract interfaces
|
||||||
|
|
Binary file not shown.
After Width: | Height: | Size: 17 KiB |
Loading…
Reference in New Issue