# dataProducerVerification

**Repository Path**: suHong79/dataProducerVerification

## Basic Information

- **Project Name**: dataProducerVerification
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MulanPSL-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-01-25
- **Last Updated**: 2026-01-25

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

Temperature Simulation and Method Comparison to verify that we should check the status of data producer first – Usage Guide
1. Overview

This project consists of two main components:

temp_system_sim.py
A temperature simulation system implemented as a lightweight web service (Flask, port 6000).
It generates temperature sequences together with ground-truth abnormal timestamps and exposes
system-level diagnostic interfaces.

compareDifferentMethod.py
An experimental script that queries the simulation system, interacts with a large language model (LLM),
and compares three inference methods (M1, M2, M3) for detecting the abnormal timestamp.

2. Requirements

Python 3.8+ (recommended: 3.10+)

Required packages:

pip install flask requests

3. Running the Temperature Simulation System
3.1 Start the service
python temp_system_sim.py


The service runs locally at:

http://127.0.0.1:6000

3.2 Health check
curl http://127.0.0.1:6000/health

4. Simulation System APIs (temp_system_sim)
4.1 Reset and generate a new trial

Endpoint: POST /reset

Description: Generates a new temperature sequence and abnormal timestamps.

Example:

curl -X POST http://127.0.0.1:6000/reset


Optional reproducibility using a seed:

curl "http://127.0.0.1:6000/reset?seed=123"

4.2 Get temperature sequence

Endpoint: GET /getTemps

Return: Temperature list of length 8.

curl http://127.0.0.1:6000/getTemps

4.3 Get sensor intrinsic abnormal time

Endpoint: GET /getSensorAbTime

Description: Returns the intrinsic sensor failure time
(-1 means no intrinsic sensor failure).

curl http://127.0.0.1:6000/getSensorAbTime

4.4 Get related object abnormal time

Endpoint: GET /getSensorRelatedObjectAbTime

Description: Returns the abnormal time of the related object
(-1 means no object failure).

curl http://127.0.0.1:6000/getSensorRelatedObjectAbTime

4.5 Get ground-truth temperature abnormal timestamp

Endpoint: GET /getAbnormalTimeStamp

Description: Returns the earliest time when either the sensor
or the related object becomes abnormal.

curl http://127.0.0.1:6000/getAbnormalTimeStamp

5. Running the Comparison Experiment
5.1 Configure the LLM

Edit the following fields in compareDifferentMethod.py:

LLM_URL = "https://api.deepseek.com/chat/completions"
API_KEY = "sk-xxxxxx"
MODEL_NAME = "deepseek-chat"
SIM_BASE = "http://127.0.0.1:6000"


If using a local OpenAI-compatible LLM service (e.g., vLLM),
replace LLM_URL with the local endpoint.

5.2 Run the experiment

Ensure temp_system_sim.py is running, then execute:

python compareDifferentMethod.py


The script:

Runs 23 independent trials

Compares three methods:

M1: Temperature-only inference

M2: Temperature + optional sensor abnormal time

M3: Temperature + optional sensor and object abnormal times

Records accuracy, Precision/Recall/F1, penalized MAE, and LLM call counts

6. Output File: testData.txt

The experiment outputs results to testData.txt in JSON Lines format.
Each line corresponds to one trial, followed by a final summary block.

Example (simplified):

{
  "trial": 1,
  "temps": [...],
  "true_abnormal_ts": 2,
  "pred": {
    "M1": {"ts": 2, "diff": 0.0, "llm_calls": 1},
    "M2": {"ts": 2, "diff": 0.0, "llm_calls": 2},
    "M3": {"ts": 2, "diff": 0.0, "llm_calls": 2}
  }
}


ts: predicted abnormal timestamp

diff: error magnitude (penalized if one side predicts no abnormality)

llm_calls: number of LLM API calls used in the trial

7. Notes and Common Issues

Why M2 and M3 may perform similarly:
If sensor and object abnormal times are highly correlated, object-level information may
be largely redundant, limiting the marginal benefit of M3.

LLM output must be JSON-only:
The scripts enforce a strict JSON protocol to avoid parsing errors.
If invalid output occurs, the script automatically requests a corrected response.

Port conflicts:
Ensure port 6000 is free before starting temp_system_sim.py.