This is the model repository of YNU's deep learning principles and platform course assignments, which mainly use remote
sensing image datasets to achieve **classification**, **colorization** and **super-resolution**.
The **NWPU-RESISC45** dataset, which is publicly available from the Northwestern Polytechnical University, is the main
dataset used in this project. In this dataset, there are a total of 45 categories of remote sensing images, and each
category contains 700 remote sensing images, all of which are 256x256 in size.
At the same time, the project is divided by network architecture, and the separate network architecture folder is also a
relatively complete code for that network, which can be taken down and run directly.
## Get Started
```bash
git clone https://github.com/JackieLin2004/Intelligent-RS.git
cd ./Intelligent-RS/
```
## Requirements
```bash
pip install torch==1.12.0
pip install torchvision==0.13.0
pip install pillow==9.4.0
pip install matplotlib==3.5.1
pip install seaborn==0.13.2
pip install scikit-learn==1.2.1
pip install scikit-image==0.24.0
```
## Environments
You can install the environment by the following method:
```bash
pip install -r requirements.txt
```
Or you can execute it if you want the full conda environment:
```bash
conda env create -f environment.yaml
```
## 0. Preparation
First, you need to put the **NWPU-RESISC45** dataset into the folder in the following format:
```bash
/dataset/NWPU-RESISC45/subfolder
```
Though it is not necessary for the categorization task and the super-resolution reconstruction task.
But it's needed for colorization tasks, and it is recommended to run the code under the following folder first:
```bash
/data_prepare/
```
## 1. Image Classification
This repository utilizes five classical convolutional neural networks for image classification and also experiments with
the Transformer architecture for image classification.
Convolutional Neural Networks include network architectures such as **AlexNet**, **VGGNet**, **GoogLeNet**, **ResNeXt**,
and **DenseNet**, while Transformer architectures include **Swin Transformer**.
### 1.1 Preparing the Dataset
Using AlexNet as an example, if you want to use these networks for classification model training, you first need to
place the appropriate dataset:
```bash
/dataset/NWPU-RESISC45/
```
### 1.2 Run the Training Script
```bash
/AlexNet/train.py
```
### 1.3 Classification Projections
```bash
/AlexNet/predictAll.py
```
### 1.4 Charting Indicators
```bash
/AlexNet/draw_indicators.ipynb
```
### 1.5 Indicator Charts for Various Models
### 1.6 Classification Model Comparison
Networks & Metrics
AlexNet
VGGNet
GoogLeNet
ResNeXt
DenseNet
Swin Transformer
Accuracy
0.864
0.920
0.905
0.938
0.929
0.884
Loss
0.545
0.367
0.417
0.374
0.316
0.456
Precision
0.867
0.922
0.910
0.939
0.931
0.886
Recall
0.864
0.920
0.905
0.938
0.929
0.884
F1 Score
0.864
0.920
0.905
0.938
0.929
0.884
AUC
0.997
0.998
0.998
0.999
0.998
0.997
## 2. Image Super-Resolution
In this section, two network architecture implementations are used in this project. The former is a traditional
convolutional approach and the latter is a generative adversarial network.
For convolutional methods, we used SRResNet.
And for generative adversarial networks, we used SRGAN, ESRGAN. in addition, we made a little improvement to SRGAN,
which is tentatively called ISRGAN here.
### 2.1 Create Data List
Using SRResNet as an example, first you create the data list:
```bash
/SRResNet/create_data_list.py
```
Then the json file will be obtained:
```bash
/SRResNet/data/test_images.json and train_images.json
```
### 2.2 Run the Training Script
```bash
/SRResNet/train.py
```
### 2.3 Evaluation of Test Sets
```bash
/SRResNet/evaluate.ipynb
```
### 2.4 Prediction of a Single Image
```bash
/SRResNet/test.py
```
### 2.5 Comparison of Data on Indicators
SRResNet
SRGAN
ISRGAN
PSNR
34.524
30.628
30.169
SSIM
0.935
0.891
0.863
Time
0.008
0.008
0.011
For SRResNet, the loss varies as shown in Fig:
For SRGAN, the loss varies as shown in Fig:
For ISRGAN, the loss varies as shown in Fig:
Obviously, as can be seen from the above graph of the loss curve of GAN, the loss of adversarial neural network is not
stable, especially the generative loss and discriminative loss are doing fierce confrontation between each other. We do
not directly judge the effect by the loss.
### 2.6 Effective Demonstration
For ISRGAN, we replaced its original ordinary convolution module with a residual module, which is visually better than
the original one, although there is no improvement in the metrics.
However, as can be seen from the loss plot, the overall trend of ISRGAN is smoother and better.
## 3. Image Colorization
In this section, for the image colorization task, we again proceeded with two ideas. One is CNN architecture to extract
color features and the other is GAN architecture to generate images.
The Colorization1 and Colorization2 folders then correspond to the implementation of the CNN architecture and the GAN
architecture, respectively.
### 3.1 Prepared Work
Using Colorization1 as an example, first you should do some prepared work:
```bash
/Colorization1/prework.ipynb
```
Then the target file will be obtained:
```bash
/dataset/colorization_/
```
In the case of the Colorization2 model there is no need for preprocessing, as this is already done in Preparation.
### 3.2 Run the Training Script
```bash
/Colorization1/train.py
```
### 3.3 Evaluation of Test Sets
```bash
/Colorization1/evaluate.ipynb
```
### 3.4 Prediction of a Single Image
```bash
/Colorization1/test.py
```
### 3.5 Comparison of Data on Indicators
Colorization1
Colorization2
PSNR
30.733
30.980
SSIM
0.865
0.977
Time
0.0042
0.0265
For Colorization1, the loss varies as shown in Fig:
For Colorization2, the loss varies as shown in Fig:
### 3.6 Effective Demonstration
## To be continue...
