lzhengning
/
JittorMirror
mirror of https://github.com/Jittor/Jittor
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

improve Linux compatibility

This commit is contained in:
lzhengning 2021-06-10 17:33:57 +08:00
parent 582bdfe52b
commit 42da3afd36
5 changed files with 103 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
README.cn.md
View File

@ -89,7 +89,7 @@ for i,(x,y) in enumerate(get_data(n)):
Jittor框架对环境要求如下:
* 操作系统: **Linux**(e.g. Ubuntu/CentOS/Arch) 或 **Windows Subsystem of Linux（WSL）**
* 操作系统: **Linux**(e.g. Ubuntu/CentOS/Arch), **macOS**（x86_64）或 **Windows Subsystem of Linux（WSL）**
* Python：版本 >= 3.7
* C++编译器 （需要下列至少一个）
- g++ （>=5.4.0）
@ -100,7 +100,9 @@ Jittor框架对环境要求如下:
如果您不希望手动配置环境，我们推荐使用 Docker 进行安装。
除此之外，您还可以使用 pip 安装和手动安装。
注意：目前Jittor通过WSL的方式在Windows操作系统上运行，WSL的安装方法请参考[微软官网](https://docs.microsoft.com/en-us/windows/wsl/install-win10)，WSL版本目前尚不支持CUDA。
注意1：目前Jittor通过WSL的方式在Windows操作系统上运行，WSL的安装方法请参考[微软官网](https://docs.microsoft.com/en-us/windows/wsl/install-win10)，WSL版本目前尚不支持CUDA。
注意2：macOS 用户需要安装额外依赖，请参考 [macOS 安装](#macOS-安装)。
Jittor 提供了三种安装方法：docker，pip和手动安装：
@ -112,6 +114,7 @@ Jittor 提供了三种安装方法：docker，pip和手动安装：
## Docker 安装
我们提供了Docker安装方式，免去您配置环境，Docker安装方法如下：
@ -145,6 +148,27 @@ python3.7 -m jittor.test.test_example
如果测试运行通过,恭喜你已经安装完成.
jittor会自动在路径中寻找合适的编译器, 如果您希望手动指定编译器, 请使用环境变量 `cc_path` 和 `nvcc_path`(可选).
## macOS 安装
macOS 请使用 [homebrew](https://brew.sh) 安装额外的依赖 (python>=3.7, onednn)。
```bash
brew install python@3.7 onednn libomp
```
之后您可以通过 pip 安装 jittor，并测试是否可以成功运行。
```bash
python3.7 -m pip install jittor
python3.7 -m jittor.test.test_example
```
目前在macOS中，jittor 只支持 CPU 计算。
## 手动安装

28
README.md
View File

@ -92,9 +92,10 @@ We provide some jupyter notebooks to help you quick start with Jittor.
Jittor environment requirements:
* System: **Linux**(e.g. Ubuntu/CentOS/Arch) (or **Windows** Subsystem of Linux)
* System: **Linux**(e.g. Ubuntu/CentOS/Arch), **macOS** (x86_64), or **Windows Subsystem of Linux (WSL)**
* Python version >= 3.7
* CPU compiler (require at least one of the following)
* g++ (>=5.4.0)
@ -105,7 +106,9 @@ Jittor environment requirements:
Note: Currently Jittor runs on the Windows operating system through WSL. For the installation method of WSL, please refer to [Microsoft official website](https://docs.microsoft.com/en-us/windows/wsl/install-win10). WSL does not yet support CUDA.
Note#1: Currently Jittor runs on the Windows operating system through WSL. For the installation method of WSL, please refer to [Microsoft official website](https://docs.microsoft.com/en-us/windows/wsl/install-win10). WSL does not yet support CUDA.
Note#2: macOS users have to install additional dependencies, see [macOS install](#macOS-install).
Jittor offers three ways to install: docker, pip, or manual.
@ -139,6 +142,27 @@ python3.7 -m jittor.test.test_example
```
## macOS install
Please first install additional dependencies with [homebrew](https://brew.sh).
```bash
brew install python@3.7 onednn libomp
```
Then you can install jittor through pip and run the example.
```bash
python3.7 -m pip install jittor
python3.7 -m jittor.test.test_example
```
Currently jittor only supports CPU in macOS.
## manual install
We will show how to install Jittor in Ubuntu 16.04 step by step, Other Linux distributions may have similar commands.

37
README.src.md
View File

@ -113,7 +113,7 @@ We provide some jupyter notebooks to help you quick start with Jittor.
Jittor框架对环境要求如下:
* 操作系统: **Linux**(e.g. Ubuntu/CentOS/Arch) 或 **Windows Subsystem of Linux（WSL）**
* 操作系统: **Linux**(e.g. Ubuntu/CentOS/Arch), **macOS**（x86_64）或 **Windows Subsystem of Linux（WSL）**
* Python：版本 >= 3.7
* C++编译器 （需要下列至少一个）
- g++ （>=5.4.0）
@ -124,13 +124,15 @@ Jittor框架对环境要求如下:
如果您不希望手动配置环境，我们推荐使用 Docker 进行安装。
除此之外，您还可以使用 pip 安装和手动安装。
注意：目前Jittor通过WSL的方式在Windows操作系统上运行，WSL的安装方法请参考[微软官网](https://docs.microsoft.com/en-us/windows/wsl/install-win10)，WSL版本目前尚不支持CUDA。
注意1：目前Jittor通过WSL的方式在Windows操作系统上运行，WSL的安装方法请参考[微软官网](https://docs.microsoft.com/en-us/windows/wsl/install-win10)，WSL版本目前尚不支持CUDA。
注意2：macOS 用户需要安装额外依赖，请参考 [macOS 安装](#macOS-安装)。
Jittor 提供了三种安装方法：docker，pip和手动安装：
Jittor environment requirements:
* System: **Linux**(e.g. Ubuntu/CentOS/Arch) (or **Windows** Subsystem of Linux)
* System: **Linux**(e.g. Ubuntu/CentOS/Arch), **macOS** (x86_64), or **Windows Subsystem of Linux (WSL)**
* Python version >= 3.7
* CPU compiler (require at least one of the following)
* g++ (>=5.4.0)
@ -141,7 +143,9 @@ Jittor environment requirements:
Note: Currently Jittor runs on the Windows operating system through WSL. For the installation method of WSL, please refer to [Microsoft official website](https://docs.microsoft.com/en-us/windows/wsl/install-win10). WSL does not yet support CUDA.
Note#1: Currently Jittor runs on the Windows operating system through WSL. For the installation method of WSL, please refer to [Microsoft official website](https://docs.microsoft.com/en-us/windows/wsl/install-win10). WSL does not yet support CUDA.
Note#2: macOS users have to install additional dependencies, see [macOS install](#macOS-install).
Jittor offers three ways to install: docker, pip, or manual.
@ -183,6 +187,31 @@ python3.7 -m jittor.test.test_example
如果测试运行通过,恭喜你已经安装完成.
jittor会自动在路径中寻找合适的编译器, 如果您希望手动指定编译器, 请使用环境变量 `cc_path` 和 `nvcc_path`(可选).
## macOS 安装
## macOS install
macOS 请使用 [homebrew](https://brew.sh) 安装额外的依赖 (python>=3.7, onednn)。
Please first install additional dependencies with [homebrew](https://brew.sh).
```bash
brew install python@3.7 onednn libomp
```
之后您可以通过 pip 安装 jittor，并测试是否可以成功运行。
Then you can install jittor through pip and run the example.
```bash
python3.7 -m pip install jittor
python3.7 -m jittor.test.test_example
```
目前在macOS中，jittor 只支持 CPU 计算。
Currently jittor only supports CPU in macOS.
## 手动安装
## manual install

10
python/jittor/compiler.py
View File

@ -1026,10 +1026,12 @@ LOG.vv("compile order:", files)
# manual Link omp using flags(os.RTLD_NOW | os.RTLD_GLOBAL)
# if cc_type=="icc":
# os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
# libname = {"clang":"omp", "icc":"iomp5", "g++":"gomp"}[cc_type]
# libname = ctypes.util.find_library(libname)
# assert libname is not None, "openmp library not found"
# ctypes.CDLL(libname, os.RTLD_NOW | os.RTLD_GLOBAL)
if platform.system() == 'Linux':
libname = {"clang":"omp", "icc":"iomp5", "g++":"gomp"}[cc_type]
libname = ctypes.util.find_library(libname)
assert libname is not None, "openmp library not found"
ctypes.CDLL(libname, os.RTLD_NOW | os.RTLD_GLOBAL)
# get os release
if platform.system() == 'Linux':

17
python/jittor_utils/__init__.py
View File

@ -4,7 +4,7 @@
# This file is subject to the terms and conditions defined in
# file 'LICENSE.txt', which is part of this source code package.
# ***************************************************************
from multiprocessing import Pool
from multiprocessing import Pool, Value
import multiprocessing as mp
import subprocess as sp
import os
@ -169,10 +169,17 @@ def run_cmds(cmds, cache_path, jittor_path, msg="run_cmds"):
bk = mp.current_process()._config.get('daemon')
mp.current_process()._config['daemon'] = False
if pool_size == 0:
mem_bytes = os.sysconf('SC_PAGE_SIZE') * os.sysconf('SC_PHYS_PAGES')
mem_gib = mem_bytes/(1024.**3)
pool_size = min(16,max(int(mem_gib // 3), 1))
LOG.i(f"Total mem: {mem_gib:.2f}GB, using {pool_size} procs for compiling.")
try:
mem_bytes = os.sysconf('SC_PAGE_SIZE') * os.sysconf('SC_PHYS_PAGES')
mem_gib = mem_bytes/(1024.**3)
pool_size = min(16,max(int(mem_gib // 3), 1))
LOG.i(f"Total mem: {mem_gib:.2f}GB, using {pool_size} procs for compiling.")
except ValueError:
# On macOS, python with version lower than 3.9 do not support SC_PHYS_PAGES.
# Use hard coded pool size instead.
pool_size = 4
LOG.i(f"using {pool_size} procs for compiling.")
p = Pool(pool_size, initializer=pool_initializer)
p.__enter__()
import atexit