# AugTS

**Repository Path**: MyAnonymousSubmission/aug-ts

## Basic Information

- **Project Name**: AugTS
- **Description**: evaluation of AugTS
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2022-08-04
- **Last Updated**: 2023-02-17

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Learnable Augmentation based Post-hoc Confidence Calibration for Corrupted Shift

## Dependency

- torch==1.9.0
- scikit-learn==0.24.2
- torchvision==0.10.0
- opencv-python==4.5.3.56
- torch_two_sample==0.1

## Datasets

You can set the dataset root directory at file `datasets.py`.

You can download CIFAR-10-C [here](https://zenodo.org/record/2535967/files/CIFAR-10-C.tar?download=1).

Please modify the `base_folder` variable in `datasets.py` to set your CIFAR-10-C location.

## Models

We upload the pre-trained ResNet-18 model for CIFAR-10 [here](https://www.dropbox.com/s/c1hnagihlqs7in6/deterministic-resnet18-cifar10-1227.pkl?dl=0).

Please modify the `config` function in `utils.py` to set the pre-trained model location.

## Metrics

All the metrics in this paper are located in the file `metrics.py`. In this paper, we use ECE to evaluate the calibration performance. 

## Evaluation

When all the preparation is done, you can evaluate the methods in our paper by the following scripts.

```
python outputs.py --model resnet18 --dataset cifar10

```

We store the output tensors of all the corrupted data at directory `root_path`, which accelerates the computation. You can set the `root_path` acorrding to your environment.

```

python ts_eval.py --model resnet18 --dataset cifar10

```

You can obtain all the results of `base`, `ts`, `ets`, `pts` and `augts` for 18 corrupted types.

Or you can run the script `run.sh` in the terminal directly.