diff --git a/README.md b/README.md index 3c73302..cb7bbb7 100644 --- a/README.md +++ b/README.md @@ -27,8 +27,8 @@ like how you enjoy real-world milk tea (cheers). code. This allows users to build their own **agentic** and **RLVR** training workflows with minimal effort. - 🔥 **Asynchronous RL**: With algorithm-system co-design, AReaL supports fully - asynchronous RL for **the fastest training speed**! Experimental support for multi-turn - agentic RL is also provided. + asynchronous RL for **the fastest training speed**! Experimental support for + multi-turn agentic RL is also provided. - 🛠️ **Open & Reproducible**: We continuously release _all code, datasets, and training recipes_ for RL training of LLMs. - 🚀 **Scalability**: AReaL can seamlessly adapt to different computational resource @@ -69,7 +69,7 @@ New highlights in AReaLite: - Instead of the *system-centric* architecture in old AReaL, AReaLite follows an **AI-centric** API design that aims to provide the following key features: - - **Light-weight** & **easy to write** algorithm and training workflow customization. + - **Light-weight** & **easy-to-write** algorithm and training workflow customization. - **Easy to scale up** without knowing system and infrastructure details. - **Adaptable and plugable:** Smooth to integrate with other modern AI applications. @@ -106,8 +106,8 @@ Our training scripts will automatically download the dataset (openai/gsm8k) and python3 -m arealite.launcher.local examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml ``` -On a Ray cluster with 2 nodes & 8 GPUs each node, runs (Remember to change paths in the YAML -file to your own shared storage): +On a Ray cluster with 2 nodes & 8 GPUs each node, runs (Remember to change paths in the +YAML file to your own shared storage): ``` python3 -m arealite.launcher.ray examples/arealite/gsm8k_grpo.py --config examples/arealite/configs/gsm8k_grpo.yaml \ diff --git a/arealite/README.md b/arealite/README.md index d7b7696..028d651 100644 --- a/arealite/README.md +++ b/arealite/README.md @@ -12,9 +12,9 @@ understand, and develop with effectively. The primary issue stems from its *system-centric* rather than *AI-centric* architecture and API design. An *AI-centric* design aims to provide three key features: -- **Light-weight & focused customization:** Users can implement their algorithms and - training workflows with minimal and concentrated code, often in just a few files or - even a single file. +- **Light-weight & easy-to-write customization:** Users can implement their algorithms + and training workflows with minimal and concentrated code, often in just a few files + or even a single file. - **Easy to scale up:** Experiments can be scaled up seamlessly without requiring knowledge of underlying system or infrastructure details. - **Adaptable and plugable:** Users is free to integrate the system with code or APIs @@ -80,6 +80,16 @@ arealite/ ### Component Overview +The AReaLite codebase is structured into four distinct layers: the API layer, backend +layer, customization layer, and entry point layer. As illustrated in the figure below, +workflow and algorithm customization logic resides in separate layers above the backend. +We prioritize keeping the entry point and customization layers clean and intuitive, +isolating them from the complex backend implementation. With AReaLite, users can define +their custom training workflows and algorithms entirely within a single entry point +file. + +![arealite-layers](../assets/arealite_layers.png) + #### 1. API Layer (`api/`) The API layer establishes clean contracts between components through abstract interfaces diff --git a/assets/arealite_layers.png b/assets/arealite_layers.png new file mode 100644 index 0000000..f57fdab Binary files /dev/null and b/assets/arealite_layers.png differ