# multi_gpu_test

**Repository Path**: LouisJiangjing/multi_gpu_test

## Basic Information

- **Project Name**: multi_gpu_test
- **Description**: PyTorch多GPU测试仓库
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2022-08-30
- **Last Updated**: 2023-04-24

## Categories & Tags

**Categories**: Uncategorized

**Tags**: cv, 工具

## README

# pytorch多卡训练DDP

## 1. 库安装步骤

1. 安装apex（应该也可用不用安装）  
`git clone https://github.com/NVIDIA/apex`  
`cd apex`  
`pip3 install -v --no-cache-dir ./`  
2. 测试是否安装成功
`python -m torch.distributed.launch --nproc_per_node=4 main.py`

## 2. 运行程序

1. torch.nn.DataParallel，`python main_DP.py` 
2. torch.nn.parallel.DistributedDataParallel，`python -m torch.distributed.launch --nproc_per_node=4 main_DPP.py` （其中nproc_per_node可填GPU数目）

## 3. 运行结果

1. torch.nn.DataParallel：   
    GPU占用：1091MiB；2757MiB；2757MiB；2757MiB；  
    耗时：0:07:01.833007  
2. torch.nn.parallel.DistributedDataParallel：
    GPU占用：961MiB；961MiB；961MiB；961MiB  
    耗时：0:00:57.362566  

## 参考  
[github仓库1](https://github.com/wisesky/chenhub.com/blob/235dae22fe576759064012198ea5e5e976ff34d0/source/_posts/GPU-in-Pytorch-并行和分布式实践.md)   
[github仓库2(主要是这个)](https://github.com/lesliejackson/PyTorch-Distributed-Training/blob/master/main.py)   
[知乎的博客](https://zhuanlan.zhihu.com/p/206467852)