# xacl_fmk

**Repository Path**: pyxin/xacl_fmk

## Basic Information

- **Project Name**: xacl_fmk
- **Description**: No description available
- **Primary Language**: C++
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 4
- **Created**: 2025-04-25
- **Last Updated**: 2025-04-25

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# <font face="微软雅黑">

# XACL FMK

[![XACL FMK](https://img.shields.io/static/v1?label=release&message=released&color=brightgreen&style=flat&logo=visual-studio-code) ![XACL FMK](https://img.shields.io/static/v1?label=author&message=wang-bain&color=blue&style=flat&logo=gitee) ![XACL FMK](https://img.shields.io/static/v1?label=email&message=wang.bei@outlook.com&color=orange&style=flat&logo=microsoft-outlook)](https://gitee.com/wang-bain/xacl_fmk)

***
XACL FMK工具**支持**的推理场景如下：

* [x] Device合设部署场景下的om模型推理。
* [x] Device拉远部署场景下的om模型推理。
* [x] 指定Device场景推理。
* [x] 循环Loop场景推理。
* [x] 数据Dump和性能采集场景推理。
* [x] 动态分档和动态Shape场景。
***

XACL FMK工具**不支持**的推理场景如下：
* [ ] 动态场景下AIPP网络推理。

***

## 内容列表
- [工具编译](#工具编译)
- [工具使用](#工具使用)
    - [工具帮助](#工具帮助)
    - [普通场景](#普通场景)
    - [指定Device场景](#指定Device场景)
    - [循环Loop场景](#循环Loop场景)
    - [数据Dump和性能采集场景推理](#数据Dump和性能采集场景推理)
    - [动态分档和动态Shape场景](#动态分档和动态Shape场景)
    - [拉远部署场景](#拉远部署场景)

## 工具编译

### 本地编译
1. 确认RUN包是否安装，且安装路径与CMakeLists.txt文件中相同，否则请修改为实际路径：

```Bash
if (NOT DEFINED ENV{RUN_PATH})
    set(RUN_PATH "/usr/local/Ascend")
else ()
    set(RUN_PATH $ENV{RUN_PATH})
endif ()

message(STATUS "RUN_PATH: ${RUN_PATH}")

```

2. 确认CANN包是否安装，且安装路径与CMakeLists.txt文件中相同，否则请修改为实际路径：
* 头文件目录：
  * **dsmi_common_interface.h**头文件存在于${KERNEL_INCLUDE_PATH}/driver目录
  * **ascend_hal.h**头文件存在于${KERNEL_INCLUDE_PATH}/driver目录
  * **acl.h**头文件存在于${ACL_INCLUDE_PATH}/acl目录

* 动态库目录
  * **libdrvdsmi_host.so**库文件存在于${DRIVER_LIB_PATH}目录
  * **libascend_hal.so**库文件存在于${DRIVER_LIB_PATH}目录
  * **libascendcl.so**库文件存在于${ACL_LIB_PATH}目录
```Bash
# KERNEL_INCLUDE_PATH
if (NOT DEFINED ENV{KERNEL_INCLUDE_PATH})
    SET(KERNEL_INCLUDE_PATH "${RUN_PATH}/driver/kernel/inc")
else ()
    SET(KERNEL_INCLUDE_PATH $ENV{KERNEL_INCLUDE_PATH})
endif ()
# ACL_INCLUDE_PATH
if (NOT DEFINED ENV{ACL_INCLUDE_PATH})
    SET(ACL_INCLUDE_PATH "${RUN_PATH}/x86_64-linux/include")
else ()
    SET(ACL_INCLUDE_PATH $ENV{ACL_INCLUDE_PATH})
endif ()
include_directories(${KERNEL_INCLUDE_PATH} ${ACL_INCLUDE_PATH} inc)
message(STATUS "INCLUDE_PATH: ${KERNEL_INCLUDE_PATH} ${ACL_INCLUDE_PATH} inc")

# ACL_LIB_PATH
if (NOT DEFINED ENV{ACL_LIB_PATH})
    SET(ACL_LIB_PATH "${RUN_PATH}/x86_64-linux/lib64")
else ()
    SET(ACL_LIB_PATH $ENV{ACL_LIB_PATH})
endif ()
# DRIVER_LIB_PATH
if (NOT DEFINED ENV{DRIVER_LIB_PATH})
    SET(DRIVER_LIB_PATH "${RUN_PATH}/driver/lib64/driver")
else ()
    SET(DRIVER_LIB_PATH $ENV{DRIVER_LIB_PATH})
endif ()
link_directories(${ACL_LIB_PATH} ${DRIVER_LIB_PATH})
message(STATUS "LIB_PATH: ${ACL_LIB_PATH} ${DRIVER_LIB_PATH}")

```
3. ./xacl_fmk.sh执行脚本编译
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./xacl_fmk.sh 
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RUN_PATH: /usr/local/Ascend
-- INCLUDE_PATH: /usr/local/Ascend/driver/kernel/inc /usr/local/Ascend/x86_64-linux/include inc
-- LIB_PATH: /usr/local/Ascend/x86_64-linux/lib64 /usr/local/Ascend/driver/lib64/driver
-- Configuring done
-- Generating done
-- Build files have been written to: /home/wang-bain/xacl_fmk/build/intermediates/host
Scanning dependencies of target xacl_fmk
[ 33%] Building CXX object CMakeFiles/xacl_fmk.dir/utils.cpp.o
[ 66%] Building CXX object CMakeFiles/xacl_fmk.dir/main.cpp.o
[100%] Linking CXX executable /home/wang-bain/xacl_fmk/out/xacl_fmk
[100%] Built target xacl_fmk

```

## 工具帮助
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk/out# ./xacl_fmk -h
2021-04-22 21:39:51.794 - INFO - [XACL]: Usage: ./xacl_fmk [input parameters]
-m=model                 Required, om model file path. Relative and absolute paths are supported.

-i=inputFiles            Optional, input files or directories. Use commas to separate each inputs.
                         Set inputs to zeros if not specified. Relative and absolute paths are supported.
-o=outputPath            Optional, path of output files. The default value is the current path.
                         Relative and absolute paths are supported.

-v=dynamicShape          Optional, dynamic size of shape.
                         Use semicolon to separate each input, use commas to separate each dim.
                         The default value is NULL, indicating that dynamicShape function is disabled.
                         Enter the actual shape size when dynamicShape function is enabled.
-x=imageRank             Optional, rank size of image. Use commas to separate height and width.
                         The default value is NULL, indicating that imageRank function is disabled.
                         Enter the actual height and width size when imageRank function is enabled.
-y=batchRank             Optional, rank size of batch.
                         The default value is NULL, indicating that batchRank function is disabled.
                         Enter the actual size of the batch when batchRank function is enabled.
-z=dimsRank              Optional, rank size of dims. Use commas to separate each dim.
                         The default value is NULL, indicating that dimsRank function is disabled.
                         Enter the actual size of each dims when dimsRank function is enabled.

-n=nodeId                Optional, ID of the Ascend device used for inference.
                         The default value is 0, indicating that device 0 is used for inference.
-l=loopNum               Optional, the number of inference times.
                         The default value is 1, indicating that inference is performed once.
-d=dumpJson              Optional, configuration file used to save operator data or profiling.
                         The default value is NULL, indicating that the save function is disabled.

-h=help                  Show this help message.

```

## 工具使用
***
1. 编译完成后可执行文件为： ./out/xacl_fmk
2. 执行前请假在环境变量： source ${RUN_PATH}/bin/setenv.bash
3. 请严格区分不同使用场景，静态，分档，动态Shape禁止混用入参，否则会导致程序CoreDump
***

### 普通场景

#### 执行命令，入参为仅包含-m， xacl_fmk自动构造全零输入，输出保存在当前目录
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om

```

#### 执行命令，入参为仅包含-m， -i， -o， -i指定文件输入，输出保存在-o目录下
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om -i xacl_fmk_test.bin -o xacl_fmk_test

```

#### 执行命令，入参为仅包含-m， -i， -o， -i指定目录输入，输出保存在-o目录下，xacl_fmk会根据输入文件大小和模型输入大小判断是否需要合并输入文件
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om -i xacl_fmk_test_input -o xacl_fmk_test

```

### 指定Device场景
#### 执行命令，入参增加-n，不设置时默认为0。
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om -n 0

```

### 循环Loop场景
#### 执行命令，入参增加-l，不设置时默认为1。
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om -l 5

```

### 数据Dump和性能采集场景推理
#### 获取模型json
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# atc --mode=1 --om=xacl_fmk_test.om --json=xacl_fmk_test.json
ATC start working now, please wait for a moment.
ATC run success, welcome to the next use.

```
#### 获取模型名称
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# less ./xacl_fmk_test.json | tail -n 3
  "name": "xacl_fmk_test",
  "version": 2
}

```
#### 创建xacl_fmk_dump.json

采集精度数据
1. dump_list中的model_name请填写上一步获取的模型名称
2. dump_path表示保存精度数据的目录，请务必先创建该目录
3. dump_mode表示保存精度数据的方式，all表示保存算子的输入和输出，output表示进保存算子输出（默认场景），input表示仅保存算子输入
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk/out# vim xacl_fmk_dump.json 
{
    "dump":{ 
        "dump_list": [{"model_name": "xacl_fmk_test"}],
        "dump_path": "/home/wang-bain/xacl_fmk/out",
        "dump_mode": "all"
    }
}

```

采集性能数据
1. switch表示profiling采集开关
2. output表示保存性能数据的目录，请务必先创建该目录
3. aicpu表示是否采集AICPU算子性能
4. aic_metrics表示采集AICORE算子性能数据的类型，详情请参考Profiling工具使用指南
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk/out# vim xacl_fmk_dump.json 
{
    "profiler":{ 
        "switch": "on",
        "output": "/home/wang-bain/xacl_fmk/out",
        "aicpu": "on",
        "aic_metrics": "PipeUtilization"
    }       
}

```
**【注意】** 
1. xacl_fmk_dump.json文件格式务必保证和上述样例一致，同时采集精度和性能数据时，可以合并json，并确保**json格式合法**。
2. xacl_fmk_dump.json文件不建议同时采集精度数据和性能数据，采集精度数据时会拉长推理执行时间，造成性能数据不准确。

#### 执行命令，入参增加-d，不设置时默认为空。
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m ./xacl_fmk_test.om -d xacl_fmk_dump.json

```

### 动态分档和动态Shape场景

#### 动态分档场景适用于动态HWSize，动态Batch和动态Dims三种场景

***

1. 使用-x HeightSize,WidthSize 表示动态HWSize场景下的HW实际大小，以逗号分隔，如： -x "128,128"。
2. 使用-y BatchSize 表示动态Batch场景下的Batch实际大小，如： -y "32"。
3. 使用-z Dim0Size,Dim1Size...,DimNSize 表示动态Dims场景下输入各维度实际大小，以逗号分隔各维度，如： -z "32,3,128,128"。

***

#### 动态Shape场景

***

1. 使用-v Dim0Size,Dim1Size...,DimNSize 表示动态Shape场景下输入的各维度实际大小，以逗号分割各维度，以分号分割各输入，如：-v "32,3,128,128;32,3,128,128"
2. 动态Shape场景由于输入输出未知，无法判断输入输出内存大小，因此必须指定-i输入文件，否则会执行失败

***

#### 以动态Batch分档为例，入参增加-y指定模型执行实际batch

```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m xacl_fmk_test.om  -y 4

```

#### 以过程动态Shape为例
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m xacl_fmk_test.om -i xacl_fmk_test.bin -o xacl_fmk_test

```

#### 以输入动态Shape为例，入参增加-v，设置真实输入Shape，以引号包含，维度间用逗号,分割，多输入以分号;分割
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m xacl_fmk_test.om -i xacl_fmk_test.bin -o xacl_fmk_test -v "800,1199,3"

```

### 拉远部署场景

#### 执行命令
```Bash
root@ubuntu-7131:/home/wang-bain/xacl_fmk# ./out/xacl_fmk -m xacl_fmk_test.om -i xacl_fmk_test.bin -o xacl_fmk_test

```
# </font>