forked from nudt_dsp/netrans
netrans commandline tools
This commit is contained in:
xujiao
parent
e56b1e19da
commit
50714682d5
|
|
@ -0,0 +1,250 @@
|
|||
# 命令行工具 netrans_cli 使用介绍
|
||||
netrans_cli 是 netrans 进行模型转换的命令行工具，使用 ntrans_cli 完成模型转换的步骤如下：
|
||||
1. 导入模型
|
||||
2. 生成并修改前处理配置文件 *_inputmeta.yml
|
||||
3. 量化模型
|
||||
4. 导出模型
|
||||
|
||||
|
||||
## 安装
|
||||
|
||||
**<font color="#dd0000">注意：</font>准备好的工程目录应符合netrans 要求，且和脚本在用一目录下**
|
||||
|
||||
比如导入一个TensorFlow的lenet模型，那么目录格式应如下：
|
||||
```bash
|
||||
.
|
||||
├── export.sh
|
||||
├── gen_inputmeta.sh
|
||||
├── import_model.sh
|
||||
├── infer.sh
|
||||
├── lenet
|
||||
│ ├── 0.jpg
|
||||
│ ├── dataset.txt
|
||||
│ ├── inputs_outputs.txt
|
||||
│ └── lenet.pb
|
||||
└── quantize.sh
|
||||
```
|
||||
|
||||
## netrans_cli 脚本介绍
|
||||
|
||||
|脚本|功能|使用|
|
||||
|:---|---|---|
|
||||
|import_model.sh| 模型导入功能，将模型转换成 pnna 支持的格式| sh import_model.sh model_name|
|
||||
|gen_inputmeta.sh| 预处理模版生成功能，生成预处理模版，根据模型进行对于的修改| sh gen_inputmeta.sh model_name|
|
||||
| quantize.sh| 量化功能， 对模型进行量化生成量化参数文件| sh quantize.sh model_name quantize_data_type|
|
||||
|infer.sh| 推理功能，对模型进行推理|sh infer.sh model_name quantize_data_type<br/>注意：该功能仅用于测试，和开发板上推理结果不完全一致|
|
||||
|export.sh|导出功能，将量化好的模型导出成 pnna 上可以运行的runtime| sh export.sh model_name quantize_data_type|
|
||||
|img2dat.sh|图片数据转换成量化后的二进制文件。需要转换的图片名存放在dataset.txt中。需要实先进行 inputmeta 文件生成 和 模型量化。 | sh img2dat.sh model_name quantize_data_type|
|
||||
|
||||
## 使用 netrans_cli 完成模型转换
|
||||
### import.sh 导入模型
|
||||
|
||||
使用import.sh脚本进行导入
|
||||
- 用法： ./import.sh 以模型文件名命名的模型数据文件夹，例如：
|
||||
```shell
|
||||
$ ./import.sh lenet
|
||||
... ...
|
||||
I End importing tensorflow...
|
||||
I Dump net to lenet.json
|
||||
I Save net to lenet.data
|
||||
I ----------------Error(0),Warning(0)----------------
|
||||
SUCCESS
|
||||
```
|
||||
"lenet"是文件夹名，也作为模型名和权重文件名。导入会打印相关日志信息，成功后会打印SUCESS。导入后lenet文件夹应该有"lenet.json"和"lenet.data"文件:
|
||||
```shell
|
||||
$ ls -lrt lenet
|
||||
total 3396
|
||||
-rwxr-xr-x 1 hope hope 1727201 Nov 5 2018 lenet.pb
|
||||
-rw-r--r-- 1 hope hope 553 Nov 5 2018 0.jpg
|
||||
-rwxr--r-- 1 hope hope 6 Apr 21 17:04 dataset.txt
|
||||
-rw-rw-r-- 1 hope hope 69 Jun 7 09:19 inputs_outputs.txt
|
||||
-rw-r--r-- 1 hope hope 5553 Jun 7 09:21 lenet.json
|
||||
-rw-r--r-- 1 hope hope 1725178 Jun 7 09:21 lenet.data
|
||||
```
|
||||
|
||||
### gen_inputmeta 生成Inputmeta文件
|
||||
在netrans_cli目录下使用inputmeta_gen.sh脚本生成inputmeta文件
|
||||
- ./gen_inputmeta.sh 以模型文件名命名的模型数据文件夹，例如：
|
||||
```shell
|
||||
$ ./gen_inputmeta.sh lenet
|
||||
... ...
|
||||
I Namespace(generate='inputmeta', input_meta_output=None, model='lenet.json', separated_database=True, which='generate')
|
||||
I Load model in lenet.json
|
||||
I Generate input meta lenet_inputmeta.yml
|
||||
I ----------------Error(0),Warning(0)----------------
|
||||
```
|
||||
|
||||
```shell
|
||||
$ ls -lrt lenet
|
||||
total 3400
|
||||
-rwxr-xr-x 1 hope hope 1727201 Nov 5 2018 lenet.pb
|
||||
-rw-r--r-- 1 hope hope 553 Nov 5 2018 0.jpg
|
||||
-rwxr--r-- 1 hope hope 6 Apr 21 17:04 dataset.txt
|
||||
-rw-rw-r-- 1 hope hope 69 Jun 7 09:19 inputs_outputs.txt
|
||||
-rw-r--r-- 1 hope hope 5553 Jun 7 09:21 lenet.json
|
||||
-rw-r--r-- 1 hope hope 1725178 Jun 7 09:21 lenet.data
|
||||
-rw-r--r-- 1 hope hope 948 Jun 7 09:35 lenet_inputmeta.yml
|
||||
```
|
||||
可以看到，最终生成的是*.yml文件，该文件用于为Netrans中间模型配置输入层数据集合。<b>Netrans中的量化、推理、导出和图片转dat的操作都需要用到这个文件。因此，此步骤不可跳过。</b>
|
||||
|
||||
可以看到，最终生成的是*.yml文件，该文件用于为Netrans中间模型配置输入层数据集合。<b>Netrans中的量化、推理、导出和图片转dat的操作都需要用到这个文件。因此，此步骤不可跳过。</b>
|
||||
|
||||
Inputmeta.yml文件结构如下：
|
||||
```yaml
|
||||
%YAML 1.2
|
||||
---
|
||||
# !!!This file disallow TABs!!!
|
||||
# "category" allowed values: "image, undefined"
|
||||
# "database" allowed types: "H5FS, SQLITE, TEXT, LMDB, NPY, GENERATOR"
|
||||
# "tensor_name" only support in H5FS database
|
||||
# "preproc_type" allowed types:"IMAGE_RGB, IMAGE_RGB888_PLANAR, IMAGE_RGB888_PLANAR_SEP,
|
||||
IMAGE_I420,
|
||||
# IMAGE_NV12, IMAGE_YUV444, IMAGE_GRAY, IMAGE_BGRA, TENSOR"
|
||||
input_meta:
|
||||
databases:
|
||||
- path: dataset.txt
|
||||
type: TEXT
|
||||
ports:
|
||||
- lid: data_0
|
||||
category: image
|
||||
dtype: float32
|
||||
sparse: false
|
||||
tensor_name:
|
||||
layout: nhwc
|
||||
shape:
|
||||
- 50
|
||||
- 224
|
||||
- 224
|
||||
- 3
|
||||
preprocess:
|
||||
reverse_channel: false
|
||||
mean:
|
||||
- 103.94
|
||||
- 116.78
|
||||
- 123.67
|
||||
scale: 0.017
|
||||
preproc_node_params:
|
||||
preproc_type: IMAGE_RGB
|
||||
add_preproc_node: false
|
||||
preproc_perm:
|
||||
- 0
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
- lid: label_0
|
||||
redirect_to_output: true
|
||||
category: undefined
|
||||
tensor_name:
|
||||
dtype: float32
|
||||
shape:
|
||||
- 1
|
||||
- 1
|
||||
|
||||
```
|
||||
|
||||
上面示例文件的各个参数解释：
|
||||
|
||||
| 参数 | 说明 |
|
||||
| :--- | ---
|
||||
| input_meta | 预处理参数配置申明。 |
|
||||
| databases | 数据配置，包括设置 path、type 和 ports 。|
|
||||
| path | 数据集文件的相对（执行目录）或绝对路径。默认为 dataset.txt, 不建议修改。 |
|
||||
| type | 数据集文件格式，固定为TEXT。 |
|
||||
| ports | 指向网络中的输入或重定向的输入，目前只支持一个输入，如果网络存在多个输入，请与@ccyh联系。 |
|
||||
| lid | 输入层的lid |
|
||||
| category | 输入的类别。将此参数设置为以下值之一：image（图像输入）或 undefined（其他类型的输入）。 |
|
||||
| dtype | 输入张量的数据类型，用于将数据发送到 pnna 网络的输入端口。支持的数据类型包括 float32 和 quantized。 |
|
||||
| sparse | 指定网络张量是否以稀疏格式存在。将此参数设置为以下值之一：true（稀疏格式）或 false（压缩格式）。 |
|
||||
| tensor_name | 留空此参数 |
|
||||
| layout | 输入张量的格式，使用 nchw 用于 Caffe、Darknet、ONNX 和 PyTorch 模型。使用 nhwc 用于 TensorFlow、TensorFlow Lite 和 Keras 模型。 |
|
||||
| shape | 此张量的形状。第一维，shape[0]，表示每批的输入数量，允许在一次推理操作之前将多个输入发送到网络。如果batch维度设置为0，则需要从命令行指定--batch-size。如果 batch维度设置为大于1的值，则直接使用inputmeta.yml中的batch size并忽略命令行中的--batch-size。 |
|
||||
| fitting | 保留字段 |
|
||||
| preprocess | 预处理步骤和顺序。预处理支持下面的四个键，键的顺序代表预处理的顺序。您可以相应地调整顺序。 |
|
||||
| reverse_channel | 指定是否保留通道顺序。将此参数设置为以下值之一：true（保留通道顺序）或 false（不保留通道顺序）。对于 TensorFlow 和 TensorFlow Lite 框架的模型使用 true。 |
|
||||
| mean | 用于每个通道的均值。 |
|
||||
| scale | 张量的缩放值。均值和缩放值用于根据公式 (inputTensor - mean) × scale 归一化输入张量。|
|
||||
| preproc_node_params | 预处理节点参数，在 OVxlib C 项目案例中启用预处理任务 |
|
||||
| add_preproc_node | 用于处理 OVxlib C 项目案例中预处理节点的插入。[true, false] 中的布尔值，表示通过配置以下参数将预处理层添加到导出的应用程序中。此参数仅在 add_preproc_node 参数设置为 true 时有效。|
|
||||
| preproc_type | 预处理节点输入类型。 [IMAGE_RGB, IMAGE_RGB888_PLANAR,IMAGE_YUV420, IMAGE_GRAY, IMAGE_BGRA, TENSOR] 中的字符串值 |
|
||||
| preproc_perm | 预处理节点输入的置换参数。 |
|
||||
| redirect_to_output | 将database张量重定向到图形输出的特殊属性。如果为该属性设置了一个port，网络构建器将自动为该port生成一个输出层，以便后处理文件可以直接处理来自database的张量。 如果使用网络进行分类，则上例中的lid“input_0”表示输入数据集的标签lid。 您可以设置其他名称来表示标签的lid。 请注意，redirect_to_output 必须设置为 true，以便后处理文件可以直接处理来自database的张量。 标签的lid必须与后处理文件中定义的 labels_tensor 的lid相同。 [true, false] 中的布尔值。 指定是否将由张量表示的输入端口的数据直接发送到网络输出。true（直接发送到网络输出）或 false（不直接发送到网络输出）|
|
||||
|
||||
可以根据实际情况对生成的inputmeta文件进行修改。
|
||||
|
||||
### 模型量化
|
||||
如果我们训练好的模型的数据类型是float32的，为了使模型以更高的效率在pnna上运行，我们可以对模型进行量化操作，量化操作可能会带来一定程度的精度损失。<
|
||||
在netrans_cli目录下使用quantize.sh脚本进行量化操作。
|
||||
|
||||
用法：./quantize.sh 以模型文件名命名的模型数据文件夹 量化类型，例如：
|
||||
|
||||
```shell
|
||||
$ ./quantize.sh lenet uint8
|
||||
... ...
|
||||
I End quantization...
|
||||
I Dump net quantize tensor table to lenet_asymmetric_affine.quantize
|
||||
I Save net to lenet.data
|
||||
I ----------------Error(0),Warning(0)----------------
|
||||
SUCCESS
|
||||
|
||||
```
|
||||
|
||||
支持的量化类型有：uint8、int8、int16
|
||||
|
||||
### 模型推理
|
||||
|
||||
在netrans_cli目录下使用infer.sh脚本进行推理
|
||||
|
||||
- 用法：./infer.sh 以模型文件名命名的模型数据文件夹 数据类型，例如：
|
||||
|
||||
```shell
|
||||
$ ./infer.sh lenet uint8
|
||||
... ...
|
||||
I Build lenet complete.
|
||||
I Running 1 iterations
|
||||
I Save tensor ./inf/iter_0_attach_output_out0_0_out0_1_10.tensor
|
||||
I Save tensor ./inf/iter_0_attach_input_x-input_out0_1_out0_1_28_28_1.tensor
|
||||
I Iter(0), top(5), tensor(@attach_output/out0_0:out0) :
|
||||
I 0: 1.0
|
||||
I 9: 0.0
|
||||
I 8: 0.0
|
||||
I 7: 0.0
|
||||
I 6: 0.0
|
||||
I Check const pool...
|
||||
I Queue cancelled.
|
||||
I End infer...
|
||||
I ----------------Error(0),Warning(0)----------------
|
||||
=========== End infer lenet model ===========
|
||||
|
||||
```
|
||||
|
||||
推理支持的数据类型：float、uint8、int8、int16，其中使用uint8、int8、int16推理时需要先进行模型量化。
|
||||
|
||||
### 模型导出
|
||||
|
||||
在netrans_cli目录下使用export.sh脚本进行推理。
|
||||
|
||||
用法：./export.sh 以模型文件名命名的模型数据文件夹 数据类型，例如：
|
||||
|
||||
```shell
|
||||
./export.sh lenet uint8
|
||||
... ...
|
||||
I End exporting ovxlib case...
|
||||
I ----------------Error(0),Warning(0)----------------
|
||||
=======================================================================
|
||||
=========== End Generate lenet ovx C code with type of asymmetric_affine ===========
|
||||
=======================================================================
|
||||
```
|
||||
|
||||
导出支持的数据类型：float、uint8、int8、int16，其中使用uint8、int8、int16导出时需要先进行模型量化。导出的工程会在模型所在的目录下面的wksp目录里。
|
||||
network_binary.nb文件在"asymmetric_affine"文件夹中：
|
||||
|
||||
```shell
|
||||
ls -lrt lenet/wksp/asymmetric_affine/
|
||||
-rw-r--r-- 1 hope hope 694912 Jun 7 09:55 network_binary.nb
|
||||
```
|
||||
|
||||
目前支持将生成的network_binary.nb文件部署到pnna硬件平台。具体部署方法请参阅模型部署相关文档。
|
||||
|
||||
## 使用实例
|
||||
|
||||
请参照examples，examples 提供 [caffe 模型转换示例](./examples/caffe_model.md),[darknet 模型转换示例](./examples/darknet_model.md),[tensorflow 模型转换示例](./examples/tensorflow_model.md),[onnx 模型转换示例](./examples/onnx_model.md)。
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
#!/bin/bash
|
||||
|
||||
if [ -z "$NETRANS_PATH" ]; then
|
||||
echo "Need to set enviroment variable NETRANS_PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
OVXGENERATOR=$NETRANS_PATH/pnnacc
|
||||
|
||||
OVXGENERATOR="$OVXGENERATOR export ovxlib"
|
||||
|
||||
DATASET=dataset.txt
|
||||
|
||||
VERIFT='FLASE'
|
||||
function export_network()
|
||||
{
|
||||
NAME=$1
|
||||
pushd $NAME
|
||||
|
||||
QUANTIZED=$2
|
||||
|
||||
if [ ${QUANTIZED} = 'float' ]; then
|
||||
TYPE=float;
|
||||
quantization_type="none_quantized"
|
||||
generate_path='./wksp/none_quantized'
|
||||
elif [ ${QUANTIZED} = 'uint8' ]; then
|
||||
quantization_type="asymmetric_affine"
|
||||
generate_path='./wksp/asymmetric_affine'
|
||||
TYPE=quantized;
|
||||
elif [ ${QUANTIZED} = 'int8' ]; then
|
||||
quantization_type="dynamic_fixed_point-8"
|
||||
generate_path='./wksp/dynamic_fixed_point-8'
|
||||
TYPE=quantized;
|
||||
elif [ ${QUANTIZED} = 'int16' ]; then
|
||||
quantization_type="dynamic_fixed_point-16"
|
||||
generate_path='./wksp/dynamic_fixed_point-16'
|
||||
TYPE=quantized;
|
||||
else
|
||||
echo "=========== wrong quantization_type ! ( float / uint8 / int8 / int16 )==========="
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo " ======================================================================="
|
||||
echo " =========== Start Generate $NAME ovx C code with type of ${quantization_type} ==========="
|
||||
echo " ======================================================================="
|
||||
|
||||
mkdir -p "${generate_path}"
|
||||
|
||||
# if want to import c code into win IDE , change --target-ide-project command-line param from 'linux64' -> 'win32'
|
||||
if [ ${QUANTIZED} = 'float' ]; then
|
||||
cmd="$OVXGENERATOR \
|
||||
--model ${NAME}.json \
|
||||
--model-data ${NAME}.data \
|
||||
--model-quantize ${NAME}.quantize \
|
||||
--dtype ${TYPE} \
|
||||
--pack-nbg-viplite \
|
||||
--model-quantize ${NAME}_${quantization_type}.quantize \
|
||||
--with-input-meta ${NAME}_inputmeta.yml\
|
||||
--optimize 'VIP8000NANOQI_PLUS_PID0XB1'\
|
||||
#--optimize None\
|
||||
--target-ide-project 'linux64' \
|
||||
--viv-sdk ${NETRANS_PATH}/pnna_sdk \
|
||||
--output-path ${generate_path}/${NAME}_${quantization_type}"
|
||||
else
|
||||
|
||||
if [ -f ${NAME}_${quantization_type}.quantize ]; then
|
||||
echo -e "\033[31m using ${NAME}_${quantization_type}.quantize \033[0m"
|
||||
else
|
||||
echo -e "\033[31m Can not find ${NAME}_${quantization_type}.quantize \033[0m"
|
||||
exit -1;
|
||||
fi
|
||||
|
||||
cmd="$OVXGENERATOR \
|
||||
--model ${NAME}.json \
|
||||
--model-data ${NAME}.data \
|
||||
--model-quantize ${NAME}.quantize \
|
||||
--dtype ${TYPE} \
|
||||
--pack-nbg-viplite \
|
||||
--model-quantize ${NAME}_${quantization_type}.quantize \
|
||||
--with-input-meta ${NAME}_inputmeta.yml\
|
||||
--optimize 'VIP8000NANOQI_PLUS_PID0XB1'\
|
||||
--target-ide-project 'linux64' \
|
||||
--viv-sdk ${NETRANS_PATH}/pnna_sdk \
|
||||
--output-path ${generate_path}/${NAME}_${quantization_type}"
|
||||
fi
|
||||
if [${VERIFY}='TRUE']; then
|
||||
echo $cmd
|
||||
fi
|
||||
eval $cmd
|
||||
|
||||
# copy input file into source code folder
|
||||
# sourcefile="`cat ${DATASET}`"
|
||||
# cpcmd="cp -fr $sourcefile ${generate_path}/"
|
||||
|
||||
# echo $cpcmd
|
||||
# eval $cpcmd
|
||||
|
||||
# temp='wksp/temp'
|
||||
|
||||
# mkcmd="mkdir -p ${temp}"
|
||||
# eval $mkcmd
|
||||
|
||||
# sourcefile="`cat ${DATASET}`"
|
||||
# cpcmd="cp -fr $sourcefile ${temp}/"
|
||||
|
||||
# echo $cpcmd
|
||||
# eval $cpcmd
|
||||
|
||||
cpcmd="cp ${generate_path}_nbg_viplite/network_binary.nb ${generate_path}/"
|
||||
eval $cpcmd
|
||||
|
||||
delcmd="rm -rf ${generate_path}_nbg_viplite"
|
||||
eval $delcmd
|
||||
|
||||
# rm -rf ${generate_path}
|
||||
# mvcmd="mv ${temp} ${generate_path}"
|
||||
|
||||
# eval $mvcmd
|
||||
|
||||
echo " ======================================================================="
|
||||
echo " =========== End Generate $NAME ovx C code with type of ${quantization_type} ==========="
|
||||
echo " ======================================================================="
|
||||
|
||||
popd
|
||||
}
|
||||
|
||||
if [ "$#" -lt 2 ]; then
|
||||
echo "Input a network name and quantized type ( float / uint8 / int8 / int16 )"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [ ! -e "${1%/}" ]; then
|
||||
echo "Directory ${1%/} does not exist !"
|
||||
exit -2
|
||||
fi
|
||||
|
||||
export_network ${1%/} ${2%/}
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
#!/bin/sh
|
||||
|
||||
if [ -z "$NETRANS_PATH" ]; then
|
||||
echo "Need to set enviroment variable NETRANS_PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$#" -ne 1 ]; then
|
||||
echo "Enter a network name !"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [ ! -e "${1%/}" ]; then
|
||||
echo "Directory ${1%/} does not exist !"
|
||||
exit 3
|
||||
fi
|
||||
|
||||
|
||||
netrans=$NETRANS_PATH/pnnacc
|
||||
|
||||
|
||||
|
||||
NAME=${1%/}
|
||||
cd $NAME
|
||||
$netrans generate \
|
||||
inputmeta \
|
||||
--model ${NAME}.json \
|
||||
--separated-database \
|
||||
|
|
@ -0,0 +1,209 @@
|
|||
#!/bin/bash
|
||||
|
||||
if [ -z "$NETRANS_PATH" ]; then
|
||||
echo "Need to set enviroment variable NETRANS_PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
function import_caffe_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTCAFFE=$NETRANS_PATH/pnnacc
|
||||
|
||||
CONVERTCAFFE="$CONVERTCAFFE import caffe"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME Caffe model ==========="
|
||||
if [ -f ${NAME}.caffemodel ]; then
|
||||
cmd="$CONVERTCAFFE \
|
||||
--model ${NAME}.prototxt \
|
||||
--weights ${NAME}.caffemodel \
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data"
|
||||
else
|
||||
echo "=========== fake Caffe model data file==========="
|
||||
cmd="$CONVERTCAFFE \
|
||||
--model ${NAME}.prototxt \
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data"
|
||||
fi
|
||||
}
|
||||
|
||||
function import_tensorflow_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTF=$NETRANS_PATH/pnnacc
|
||||
|
||||
CONVERTF="$CONVERTF import tensorflow"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME Tensorflow model ==========="
|
||||
cmd="$CONVERTF \
|
||||
--model ${NAME}.pb \
|
||||
--output-data ${NAME}.data \
|
||||
--output-model ${NAME}.json \
|
||||
$(cat inputs_outputs.txt)"
|
||||
}
|
||||
|
||||
function import_onnx_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTONNX=$NETRANS_PATH/pnnacc
|
||||
CONVERTONNX="$CONVERTONNX import onnx"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME ONNX model ==========="
|
||||
cmd="$CONVERTONNX \
|
||||
--model ${NAME}.onnx \
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data"
|
||||
}
|
||||
|
||||
function import_tflite_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTTFLITE=$NETRANS_PATH/pnnacc
|
||||
CONVERTTFLITE="$CONVERTTFLITE import tflite"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME TFLite model ==========="
|
||||
cmd="$CONVERTTFLITE \
|
||||
--model ${NAME}.tflite \
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data"
|
||||
}
|
||||
|
||||
function import_darknet_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTDARKNET=$NETRANS_PATH/pnnacc
|
||||
CONVERTDARKNET="$CONVERTDARKNET import darknet"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME darknet model ==========="
|
||||
cmd="$CONVERTDARKNET \
|
||||
--model ${NAME}.cfg \
|
||||
--weight ${NAME}.weights \
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data"
|
||||
}
|
||||
|
||||
function import_pytorch_network()
|
||||
{
|
||||
NAME=$1
|
||||
CONVERTPYTORCH=$NETRANS_PATH/pnnacc
|
||||
CONVERTPYTORCH="$CONVERTPYTORCH import pytorch"
|
||||
|
||||
|
||||
if [ -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m rm ${NAME}.json \033[0m"
|
||||
rm ${NAME}.json
|
||||
fi
|
||||
|
||||
if [ -f ${NAME}.data ]; then
|
||||
echo -e "\033[31m rm ${NAME}.data \033[0m"
|
||||
rm ${NAME}.data
|
||||
fi
|
||||
|
||||
echo "=========== Converting $NAME pytorch model ==========="
|
||||
cmd="$CONVERTPYTORCH \
|
||||
--model ${NAME}.pt \q
|
||||
--output-model ${NAME}.json \
|
||||
--output-data ${NAME}.data \
|
||||
$(cat input_size.txt)"
|
||||
}
|
||||
|
||||
function import_network()
|
||||
{
|
||||
NAME=$1
|
||||
pushd $NAME
|
||||
|
||||
|
||||
if [ -f ${NAME}.prototxt ]; then
|
||||
import_caffe_network ${1%/}
|
||||
elif [ -f ${NAME}.pb ]; then
|
||||
import_tensorflow_network ${1%/}
|
||||
elif [ -f ${NAME}.onnx ]; then
|
||||
import_onnx_network ${1%/}
|
||||
elif [ -f ${NAME}.tflite ]; then
|
||||
import_tflite_network ${1%/}
|
||||
elif [ -f ${NAME}.weights ]; then
|
||||
import_darknet_network ${1%/}
|
||||
elif [ -f ${NAME}.pt ]; then
|
||||
import_pytorch_network ${1%/}
|
||||
else
|
||||
echo "=========== can not find suitable model files ==========="
|
||||
fi
|
||||
|
||||
echo $cmd
|
||||
eval $cmd
|
||||
|
||||
if [ -f ${NAME}.data -a -f ${NAME}.json ]; then
|
||||
echo -e "\033[31m SUCCESS \033[0m"
|
||||
else
|
||||
echo -e "\033[31m ERROR ! \033[0m"
|
||||
fi
|
||||
popd
|
||||
}
|
||||
|
||||
if [ "$#" -ne 1 ]; then
|
||||
echo "Input a network name !"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [ ! -e "${1%/}" ]; then
|
||||
echo "Directory ${1%/} does not exist !"
|
||||
exit -2
|
||||
fi
|
||||
|
||||
import_network ${1%/}
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
#!/bin/bash
|
||||
|
||||
if [ -z "$NETRANS_PATH" ]; then
|
||||
echo "Need to set enviroment variable NETRANS_PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TENSORZONX=$NETRANS_PATH/pnnacc
|
||||
|
||||
TENSORZONX="$TENSORZONX inference"
|
||||
|
||||
DATASET=./dataset.txt
|
||||
|
||||
function inference_network()
|
||||
{
|
||||
NAME=$1
|
||||
pushd $NAME
|
||||
QUANTIZED=$2
|
||||
inf_path='./inf'
|
||||
|
||||
if [ ${QUANTIZED} = 'float' ]; then
|
||||
TYPE=float32;
|
||||
quantization_type="float32"
|
||||
elif [ ${QUANTIZED} = 'uint8' ]; then
|
||||
quantization_type="asymmetric_affine"
|
||||
TYPE=quantized;
|
||||
elif [ ${QUANTIZED} = 'int8' ]; then
|
||||
quantization_type="dynamic_fixed_point-8"
|
||||
TYPE=quantized;
|
||||
elif [ ${QUANTIZED} = 'int16' ]; then
|
||||
quantization_type="dynamic_fixed_point-16"
|
||||
TYPE=quantized;
|
||||
else
|
||||
echo "=========== wrong quantization_type ! ( float / uint8 / int8 / int16 )==========="
|
||||
exit -1
|
||||
fi
|
||||
|
||||
cmd="$TENSORZONX \
|
||||
--dtype ${TYPE} \
|
||||
--batch-size 1 \
|
||||
--model-quantize ${NAME}_${quantization_type}.quantize \
|
||||
--model ${NAME}.json \
|
||||
--model-data ${NAME}.data \
|
||||
--output-dir ${inf_path} \
|
||||
--with-input-meta ${NAME}_inputmeta.yml \
|
||||
--device CPU"
|
||||
|
||||
echo $cmd
|
||||
eval $cmd
|
||||
echo "=========== End inference $NAME model ==========="
|
||||
|
||||
popd
|
||||
}
|
||||
|
||||
if [ "$#" -lt 2 ]; then
|
||||
echo "Input a network name and quantized type ( float / uint8 / int8 / int16 )"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [ ! -e "${1%/}" ]; then
|
||||
echo "Directory ${1%/} does not exist !"
|
||||
exit -2
|
||||
fi
|
||||
|
||||
inference_network ${1%/} ${2%/}
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
#!/bin/bash
|
||||
|
||||
if [ -z "$NETRANS_PATH" ]; then
|
||||
echo "Need to set enviroment variable NETRANS_PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TENSORZONEX=$NETRANS_PATH/pnnacc
|
||||
TENSORZONEX="$TENSORZONEX quantize"
|
||||
|
||||
|
||||
DATASET=./dataset.txt
|
||||
|
||||
function quantize_network()
|
||||
{
|
||||
NAME=$1
|
||||
pushd $NAME
|
||||
|
||||
QUANTIZED=$2
|
||||
|
||||
if [ ${QUANTIZED} = 'float' ]; then
|
||||
echo "=========== do not need quantied==========="
|
||||
exit -1
|
||||
elif [ ${QUANTIZED} = 'uint8' ]; then
|
||||
quantization_type="asymmetric_affine"
|
||||
elif [ ${QUANTIZED} = 'int8' ]; then
|
||||
quantization_type="dynamic_fixed_point-8"
|
||||
elif [ ${QUANTIZED} = 'int16' ]; then
|
||||
quantization_type="dynamic_fixed_point-16"
|
||||
else
|
||||
echo "=========== wrong quantization_type ! ( uint8 / int8 / int16 )==========="
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo " ======================================================================="
|
||||
echo " ==== Start Quantizing $NAME model with type of ${quantization_type} ==="
|
||||
echo " ======================================================================="
|
||||
|
||||
if [ -f ${NAME}_${quantization_type}.quantize ]; then
|
||||
echo -e "\033[31m rm ${NAME}_${quantization_type}.quantize \033[0m"
|
||||
rm ${NAME}_${quantization_type}.quantize
|
||||
fi
|
||||
|
||||
cmd="$TENSORZONEX \
|
||||
--batch-size 1 \
|
||||
--qtype ${QUANTIZED} \
|
||||
--rebuild \
|
||||
--quantizer ${quantization_type%-*} \
|
||||
--model-quantize ${NAME}_${quantization_type}.quantize \
|
||||
--model ${NAME}.json \
|
||||
--model-data ${NAME}.data \
|
||||
--with-input-meta ${NAME}_inputmeta.yml \
|
||||
--device CPU"
|
||||
echo $cmd
|
||||
eval $cmd
|
||||
|
||||
if [ -f ${NAME}_${quantization_type}.quantize ]; then
|
||||
echo -e "\033[31m SUCCESS \033[0m"
|
||||
else
|
||||
echo -e "\033[31m ERROR ! \033[0m"
|
||||
fi
|
||||
|
||||
popd
|
||||
}
|
||||
|
||||
if [ "$#" -lt 2 ]; then
|
||||
echo "Input a network name and quantized type ( uint8 / int8 / int16 )"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [ ! -e "${1%/}" ]; then
|
||||
echo "Directory ${1%/} does not exist !"
|
||||
exit -2
|
||||
fi
|
||||
|
||||
quantize_network ${1%/} ${2%/}
|
||||
Loading…
Reference in New Issue