# Evaluation

**Repository Path**: lmcins/evaluation

## Basic Information

- **Project Name**: Evaluation
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2023-05-05
- **Last Updated**: 2024-04-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Evaluation of Zero-shot models

## 1. Prepare the config file (YAML Format)

```yaml
base_dir: R:/Proteus/data # 数据目录
protein_names:
  - UBC9_HUMAN # 蛋白质名
  - TIM_SULSO
  - RASH_HUMAN[2] #可以用这种方式指定保留的最大点位数
  - P84126_THETH
  - MTH3_HAEAESTABILIZED
  - MK01_HUMAN
  - KKA2_KLEPN
  - BLAT_ECOLX
  - BG_STRSQ
  - B3VI55_LIPST
  - AMIE_PSEAE
  - T7:
      - tm # 可以用这种方式指定实验
metrics:
  - spearman
  - top_5 # 指标
models:
  esm1b: # 模型名
    - esm1b_t33_650M_UR50S
  esm1v:
    - esm1v_t33_650M_UR90S_1
  esmif:
    - esm_if1_gvp4_t16_142M_UR50
  esm2-15B:
    - esm2_t48_15B_UR50D
  esm2-3B:
    - esm2_t36_3B_UR50D
  tranception-300M:
    - tranception_300M
  tranception-700M:
    - tranception_700M
  ensemble: # 支持ensemble
    - esm1v_t33_650M_UR90S_1
    - tranception_700M
    - tranception_300M
    - esm2_t36_3B_UR50D
    - esm2_t48_15B_UR50D
    - esm_if1_gvp4_t16_142M_UR50
```

## 2. train the SVM

1. data_prepare.py: prepare the dataset 
2. normalization.py: do minmax normalization
3. svm_train.py: train


## 3. Run the evaluation
```shell
python report.py --config configs/example.yaml --output cat.xlsx
```