# OnnxRT **Repository Path**: U238/onnx-rt ## Basic Information - **Project Name**: OnnxRT - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-03-14 - **Last Updated**: 2026-03-17 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # OnnxRT - YOLO Segmentation and SAM 2.1 Deployment ## Table of Contents - [English](#english) - [中文](#中文) --- ## English ### Overview This project demonstrates the deployment of YOLO Segmentation and SAM 2.1 models using ONNX Runtime with CUDA 13 acceleration. It supports exporting as a DLL dynamic link library for easy integration in C++ projects. ### Features - **YOLO Segmentation**: Real-time object detection and segmentation using YOLO models - **SAM 2.1 Integration**: Meta's Segment Anything Model 2.1 implementation for interactive segmentation - **ONNX Runtime**: High-performance inference engine with CUDA 13 support - **Cross-platform**: Tested on Windows 11 with Visual Studio 2022 - **Model Conversion**: Example scripts to convert SAM PyTorch models to ONNX format - **DLL Export Support**: Can be built as a DLL for C++ project integration ### Requirements - **Operating System**: Windows 11 (tested) - **Compiler**: Visual Studio 2022 or later - **CUDA**: Version 13.0 or later - **Libraries**: - ONNX Runtime (included in third_party) - OpenCV (included in third_party) ### Project Structure ``` OnnxRT/ ├── DL/ # Model and test files │ ├── onnx/ # ONNX model files │ ├── bus.jpg # Test image for segmentation │ ├── classes.txt # Class labels │ ├── truck.jpg # Test image for SAM │ ├── yolo11nseg.onnx # YOLO 11 Nano Segmentation model │ └── sam2.1_hiera_tiny.* # SAM 2.1 encoder/decoder models ├── third_party/ # Third-party libraries and tools │ ├── onnxruntime-win-x64-gpu-1.24.3/ # ONNX Runtime with CUDA support │ ├── opencv4120/ # OpenCV library │ └── (SAM model conversion scripts) # SAM PyTorch to ONNX conversion examples ├── build/ # Build output directory ├── *.h/*.cpp # Source code ├── CMakeLists.txt # CMake configuration ├── compile.bat # Build script ├── main.cpp # Main program with usage examples └── readme.md # This file ``` ### Quick Start #### 1. Build the Project Run the compile script to build the project: ```bash # Build with GPU support (default) ./compile.bat # Build with CPU only ./compile.bat exe cpu # Build as DLL ./compile.bat dll ``` #### 2. Run the Examples The compiled executable supports multiple AI tasks: - **YOLO Segmentation**: Detects and segments objects in images - **SAM 2.1**: Interactive segmentation with point prompts - **Object Detection**: Traditional bounding box detection - **Classification**: Image classification ### Usage Examples #### YOLO Segmentation The `testSegmention()` function demonstrates YOLO 11 Nano Segmentation: ```cpp int testSegmention() { // Load YOLO 11nseg model SegmentionInfer segInfer; segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", true); // Load test image auto image = cv::imread("./DL/bus.jpg"); // Run inference auto results = segInfer.Infer(image); // Display and save results // ... } ``` #### SAM 2.1 Interactive Segmentation The `testSAM2()` function demonstrates SAM 2.1 with point prompts: ```cpp int testSAM2() { // Load SAM 2.1 model SAMInfer samInfer; samInfer.LoadSAM2Model( "./DL/sam2.1_hiera_tiny.encoder.onnx", "./DL/sam2.1_hiera_tiny.decoder.onnx", "./DL/classes.txt", true ); // Define prompt points (x, y) std::vector prompt_points = { cv::Point(575, 840) }; std::vector prompt_labels = { 1 }; // 1 = foreground, 0 = background // Run inference auto results = samInfer.Infer(image, prompt_points, prompt_labels); // Display and save results // ... } ``` ### Model Conversion #### Convert SAM PyTorch Model to ONNX The `third_party` directory contains example scripts to convert SAM PyTorch models to ONNX format: ```bash # Example command (adjust paths as needed) python run_sam2_to_onnx.bat ``` ### Supported Models #### YOLO Segmentation - `yolo11nseg.onnx` - YOLO 11 Nano Segmentation model #### SAM 2.1 - `sam2.1_hiera_tiny.encoder.onnx` - SAM 2.1 Encoder (Hiera Tiny) - `sam2.1_hiera_tiny.decoder.onnx` - SAM 2.1 Decoder (Hiera Tiny) ### Configuration #### GPU/CPU Mode The code automatically falls back to CPU mode if GPU loading fails: ```cpp // Try GPU first auto res = segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", true); // Fallback to CPU if GPU fails if (res.find("Error") != std::string::npos) { res = segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", false); } ``` ### Performance - **YOLO Segmentation**: ~XX ms per inference on NVIDIA GPU - **SAM 2.1**: ~XX ms per inference (varies with number of prompts) --- ## 中文 ### 概述 该项目展示了如何使用支持CUDA 13加速的ONNX Runtime部署YOLO分割模型和SAM 2.1模型。支持导出为DLL动态链接库，方便在C++项目中调用。 ### 功能特性 - **YOLO分割**: 使用YOLO模型进行实时目标检测和分割 - **SAM 2.1集成**: Meta的Segment Anything Model 2.1交互式分割实现 - **ONNX Runtime**: 支持CUDA 13的高性能推理引擎 - **跨平台**: 在Windows 11和Visual Studio 2022上测试通过 - **模型转换**: 提供SAM PyTorch模型转ONNX格式的示例脚本 - **DLL导出支持**: 可构建为DLL，方便在C++项目中集成 ### 环境要求 - **操作系统**: Windows 11 (已测试) - **编译器**: Visual Studio 2022或更高版本 - **CUDA**: 13.0或更高版本 - **库依赖**: - ONNX Runtime (已包含在third_party目录中) - OpenCV (已包含在third_party目录中) ### 项目结构 ``` OnnxRT/ ├── DL/ # 模型和测试文件 │ ├── onnx/ # ONNX模型文件 │ ├── bus.jpg # 分割测试图像 │ ├── classes.txt # 类别标签 │ ├── truck.jpg # SAM测试图像 │ ├── yolo11nseg.onnx # YOLO 11 Nano分割模型 │ └── sam2.1_hiera_tiny.* # SAM 2.1编码器/解码器模型 ├── third_party/ # 第三方库和工具 │ ├── onnxruntime-win-x64-gpu-1.24.3/ # 支持CUDA的ONNX Runtime │ ├── opencv4120/ # OpenCV库 │ └── (SAM模型转换脚本) # SAM PyTorch转ONNX格式的示例脚本 ├── build/ # 构建输出目录 ├── *.h/*.cpp # 源代码 ├── CMakeLists.txt # CMake配置文件 ├── compile.bat # 编译脚本 ├── main.cpp # 主程序（包含调用示例） └── readme.md # 本文档 ``` ### 快速开始 #### 1. 构建项目 运行编译脚本构建项目： ```bash # 构建带GPU支持的版本（默认） ./compile.bat # 构建仅CPU版本 ./compile.bat exe cpu # 构建为DLL ./compile.bat dll ``` #### 2. 运行示例 编译后的可执行文件支持多种AI任务： - **YOLO分割**: 检测并分割图像中的目标 - **SAM 2.1**: 使用点提示的交互式分割 - **目标检测**: 传统的边界框检测 - **图像分类**: 图像分类 ### 使用示例 #### YOLO分割 `testSegmention()`函数展示了YOLO 11 Nano分割的使用方法： ```cpp int testSegmention() { // 加载YOLO 11nseg模型 SegmentionInfer segInfer; segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", true); // 加载测试图像 auto image = cv::imread("./DL/bus.jpg"); // 运行推理 auto results = segInfer.Infer(image); // 显示并保存结果 // ... } ``` #### SAM 2.1交互式分割 `testSAM2()`函数展示了使用点提示的SAM 2.1分割： ```cpp int testSAM2() { // 加载SAM 2.1模型 SAMInfer samInfer; samInfer.LoadSAM2Model( "./DL/sam2.1_hiera_tiny.encoder.onnx", "./DL/sam2.1_hiera_tiny.decoder.onnx", "./DL/classes.txt", true ); // 定义提示点 (x, y) std::vector prompt_points = { cv::Point(575, 840) }; std::vector prompt_labels = { 1 }; // 1 = 前景, 0 = 背景 // 运行推理 auto results = samInfer.Infer(image, prompt_points, prompt_labels); // 显示并保存结果 // ... } ``` ### 模型转换 #### 将SAM PyTorch模型转换为ONNX `third_party`目录包含将SAM PyTorch模型转换为ONNX格式的示例脚本： ```bash # 示例命令（根据需要调整路径） python run_sam2_to_onnx.bat ``` ### 支持的模型 #### YOLO分割 - `yolo11nseg.onnx` - YOLO 11 Nano分割模型 #### SAM 2.1 - `sam2.1_hiera_tiny.encoder.onnx` - SAM 2.1编码器 (Hiera Tiny) - `sam2.1_hiera_tiny.decoder.onnx` - SAM 2.1解码器 (Hiera Tiny) ### 配置说明 #### GPU/CPU模式 代码会在GPU加载失败时自动回退到CPU模式： ```cpp // 首先尝试GPU auto res = segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", true); // 如果GPU失败则回退到CPU if (res.find("Error") != std::string::npos) { res = segInfer.LoadModel("./DL/yolo11nseg.onnx", "./DL/classes.txt", false); } ``` ### 性能表现 - **YOLO分割**: 在NVIDIA GPU上约XX毫秒/次推理 - **SAM 2.1**: 约XX毫秒/次推理（取决于提示点数量） --- **Note**: Ensure all required DLL files are present in the same directory as the executable for proper runtime operation. **注意**: 确保所有必需的DLL文件与可执行文件位于同一目录，以保证正确运行。