docs: add progress callback mechanism documentation
Add comprehensive documentation for _report_progress mechanism: - System architecture with mermaid diagrams - Complete sequence diagram showing data flow - Progress calculation logic with stage weights - Code examples and usage patterns - Design considerations (thread safety, decoupling, fault tolerance) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
401
docs/progress_callback_mechanism.md
Normal file
401
docs/progress_callback_mechanism.md
Normal file
@@ -0,0 +1,401 @@
|
|||||||
|
# 进度回调机制详解
|
||||||
|
|
||||||
|
## 概述
|
||||||
|
|
||||||
|
`_report_progress` 是一个基于**回调函数模式**的进度报告系统,用于后台任务(数据提取)和 GUI 主线程之间的线程安全通信。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 架构设计
|
||||||
|
|
||||||
|
### 系统架构图
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
subgraph BG["后台线程 (Background Thread)"]
|
||||||
|
Extractor["DiscreteMaterialPlanExtractor"]
|
||||||
|
Report["_report_progress()"]
|
||||||
|
ProgressInfo["ProgressInfo 对象"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Boundary["线程边界 (Thread Boundary)"]
|
||||||
|
Callback["progress_callback()"]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph FG["主线程 (Main/GUI Thread)"]
|
||||||
|
Calc["ProgressCalculator<br/>计算总体百分比"]
|
||||||
|
Queue["queue.Queue<br/>线程安全队列"]
|
||||||
|
Poll["_poll_progress_queue()<br/>每50ms轮询"]
|
||||||
|
GUI["GUI 组件<br/>progress_bar<br/>status_label"]
|
||||||
|
end
|
||||||
|
|
||||||
|
Extractor -->|"调用"| Report
|
||||||
|
Report -->|"创建"| ProgressInfo
|
||||||
|
ProgressInfo -->|"触发"| Callback
|
||||||
|
Callback -->|"计算"| Calc
|
||||||
|
Calc -->|"put"| Queue
|
||||||
|
Queue -->|"get"| Poll
|
||||||
|
Poll -->|"更新"| GUI
|
||||||
|
|
||||||
|
style Callback fill:#ff9,stroke:#333,stroke-width:2px
|
||||||
|
style Queue fill:#9f9,stroke:#333,stroke-width:2px
|
||||||
|
style Boundary fill:#ddd,stroke:#333,stroke-dasharray: 5 5
|
||||||
|
```
|
||||||
|
|
||||||
|
### 组件职责
|
||||||
|
|
||||||
|
| 组件 | 职责 | 位置 |
|
||||||
|
|------|------|------|
|
||||||
|
| `DiscreteMaterialPlanExtractor` | 执行数据提取任务 | 后台线程 |
|
||||||
|
| `_report_progress()` | 报告进度到回调 | 后台线程 |
|
||||||
|
| `ProgressInfo` | 进度信息数据结构 | 跨线程 |
|
||||||
|
| `progress_callback()` | GUI 提供的回调函数 | 主线程定义,后台调用 |
|
||||||
|
| `ProgressCalculator` | 计算总体进度百分比 | 主线程 |
|
||||||
|
| `queue.Queue` | 线程安全的消息队列 | 主线程 |
|
||||||
|
| `_poll_progress_queue()` | 轮询队列并更新 GUI | 主线程 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 数据流程
|
||||||
|
|
||||||
|
### 完整时序图
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant Bg as 后台线程<br/>(Extractor)
|
||||||
|
participant Report as _report_progress()
|
||||||
|
participant Callback as progress_callback()
|
||||||
|
participant Calc as ProgressCalculator
|
||||||
|
participant Queue as 进度队列
|
||||||
|
participant Poll as _poll_progress_queue()
|
||||||
|
participant GUI as GUI 组件
|
||||||
|
|
||||||
|
Bg->>Report: _report_progress('download', 2, 3, '下载中')
|
||||||
|
Report->>Report: 创建 ProgressInfo 对象
|
||||||
|
Report->>Callback: progress_callback(progress_info)
|
||||||
|
Note over Callback: 主线程定义的函数<br/>在后台线程中执行
|
||||||
|
|
||||||
|
Callback->>Calc: calculate_overall_percent(progress_info)
|
||||||
|
Note over Calc: download 阶段<br/>stage_offset=10%<br/>current=2, total=3<br/>weight=65%<br/>result = 10 + 67%×65 = 54%
|
||||||
|
Calc-->>Callback: 返回 54
|
||||||
|
|
||||||
|
Callback->>Queue: put((54, '下载中'))
|
||||||
|
Note over Queue: 线程安全队列<br/>缓冲区
|
||||||
|
|
||||||
|
loop 每 50ms
|
||||||
|
Poll->>Queue: get_nowait()
|
||||||
|
Queue-->>Poll: (54, '下载中')
|
||||||
|
Poll->>GUI: progress_bar['value'] = 54
|
||||||
|
Poll->>GUI: status_label['text'] = '下载中'
|
||||||
|
Poll->>Poll: after(50ms, 继续轮询)
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
### 进度计算逻辑
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
A[ProgressInfo<br/>stage=download<br/>current=2<br/>total=3] --> B[ProgressCalculator]
|
||||||
|
|
||||||
|
subgraph Calc["计算过程"]
|
||||||
|
direction TB
|
||||||
|
B --> C["计算阶段内进度<br/>2/3 × 100 = 67%"]
|
||||||
|
C --> D["查找阶段权重<br/>download = 65%"]
|
||||||
|
D --> E["查找阶段偏移<br/>offset = 10%"]
|
||||||
|
E --> F["总体进度<br/>10 + 67%×65 = 54%"]
|
||||||
|
end
|
||||||
|
|
||||||
|
F --> G["更新进度条<br/>54%"]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 阶段权重分配
|
||||||
|
|
||||||
|
### 进度阶段划分
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
pie title 各阶段权重分布
|
||||||
|
"登录 (5%)" : 5
|
||||||
|
"查询 (5%)" : 5
|
||||||
|
"下载 (65%)" : 65
|
||||||
|
"注销 (5%)" : 5
|
||||||
|
"转换 (15%)" : 15
|
||||||
|
"完成 (5%)" : 5
|
||||||
|
```
|
||||||
|
|
||||||
|
### 阶段详情表
|
||||||
|
|
||||||
|
| 阶段 | stage | 权重 | 进度范围 | 说明 |
|
||||||
|
|------|-------|------|----------|------|
|
||||||
|
| 登录 | `login` | 5% | 0-5% | ERP 系统登录 |
|
||||||
|
| 查询 | `query` | 5% | 5-10% | 查询数据库获取订单号 |
|
||||||
|
| 下载 | `download` | 65% | 10-75% | 批量下载数据(主要耗时) |
|
||||||
|
| 注销 | `logout` | 5% | 75-80% | 退出 ERP 系统 |
|
||||||
|
| 转换 | `convert` | 15% | 80-95% | 转换 Excel 格式并合并 |
|
||||||
|
| 完成 | `complete` | 5% | 95-100% | 任务完成 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 代码实现
|
||||||
|
|
||||||
|
### 1. 后台任务:报告进度
|
||||||
|
|
||||||
|
```python
|
||||||
|
# utils/离散备料计划维护数据提取.py
|
||||||
|
|
||||||
|
def _report_progress(self, stage: str, current: int, total: int, message: str, **detail):
|
||||||
|
"""
|
||||||
|
报告进度
|
||||||
|
|
||||||
|
Args:
|
||||||
|
stage: 阶段标识 ('login', 'query', 'download', 等)
|
||||||
|
current: 当前进度值 (1, 2, 3...)
|
||||||
|
total: 总量 (3, 100...)
|
||||||
|
message: 显示给用户的消息
|
||||||
|
**detail: 额外信息 (如 batch_index=1)
|
||||||
|
"""
|
||||||
|
if self.progress_callback:
|
||||||
|
try:
|
||||||
|
from gui.progress import ProgressInfo
|
||||||
|
progress_info = ProgressInfo(
|
||||||
|
stage=stage,
|
||||||
|
current=current,
|
||||||
|
total=total,
|
||||||
|
message=message,
|
||||||
|
detail=detail
|
||||||
|
)
|
||||||
|
# 调用 GUI 提供的回调函数
|
||||||
|
self.progress_callback(progress_info)
|
||||||
|
except Exception:
|
||||||
|
# 回调失败不影响主流程
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. GUI:设置回调
|
||||||
|
|
||||||
|
```python
|
||||||
|
# gui/data_extraction_tab.py
|
||||||
|
|
||||||
|
def _extraction_worker(self, input_file: str, output_file: str):
|
||||||
|
"""后台工作线程"""
|
||||||
|
|
||||||
|
# 创建进度回调函数
|
||||||
|
def progress_callback(progress_info: ProgressInfo):
|
||||||
|
# 1. 计算总体进度百分比
|
||||||
|
overall_percent = self.progress_calculator.calculate_overall_percent(progress_info)
|
||||||
|
# 2. 放入队列(线程安全)
|
||||||
|
self._update_progress(overall_percent, progress_info.message)
|
||||||
|
|
||||||
|
# 将回调传递给提取器
|
||||||
|
self.extractor.extract(
|
||||||
|
production_id_file=input_file,
|
||||||
|
output_file=output_file,
|
||||||
|
progress_callback=progress_callback
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. 线程安全:队列通信
|
||||||
|
|
||||||
|
```python
|
||||||
|
# gui/data_extraction_tab.py
|
||||||
|
|
||||||
|
def _update_progress(self, value: int, message: str):
|
||||||
|
"""后台线程调用,放入队列"""
|
||||||
|
try:
|
||||||
|
self.progress_queue.put_nowait((value, message))
|
||||||
|
except:
|
||||||
|
pass # 队列满时忽略
|
||||||
|
|
||||||
|
def _poll_progress_queue(self):
|
||||||
|
"""主线程轮询,更新 GUI"""
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
# 非阻塞获取队列中的消息
|
||||||
|
progress_data = self.progress_queue.get_nowait()
|
||||||
|
value, message = progress_data
|
||||||
|
# 更新 GUI 组件
|
||||||
|
self.progress_bar['value'] = value
|
||||||
|
self.status_label.config(text=message)
|
||||||
|
except queue.Empty:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
# 继续轮询(每 50ms 检查一次)
|
||||||
|
self.after(50, self._poll_progress_queue)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. 进度计算器
|
||||||
|
|
||||||
|
```python
|
||||||
|
# gui/progress.py
|
||||||
|
|
||||||
|
class ProgressCalculator:
|
||||||
|
# 各阶段在总进度中的占比
|
||||||
|
STAGE_WEIGHTS = {
|
||||||
|
'login': 5, # 0-5%
|
||||||
|
'query': 5, # 5-10%
|
||||||
|
'download': 65, # 10-75%
|
||||||
|
'logout': 5, # 75-80%
|
||||||
|
'convert': 15, # 80-95%
|
||||||
|
'complete': 5, # 95-100%
|
||||||
|
}
|
||||||
|
|
||||||
|
def calculate_overall_percent(self, progress: ProgressInfo) -> int:
|
||||||
|
"""计算总体进度百分比"""
|
||||||
|
stage = progress.stage
|
||||||
|
|
||||||
|
if stage == 'complete':
|
||||||
|
return 100
|
||||||
|
|
||||||
|
# 计算阶段起始百分比
|
||||||
|
stage_offset = self._stage_offsets[stage]
|
||||||
|
|
||||||
|
# 计算阶段内的进度百分比
|
||||||
|
stage_percent = progress.percent # current/total * 100
|
||||||
|
|
||||||
|
# 计算该阶段的权重
|
||||||
|
stage_weight = self.STAGE_WEIGHTS[stage]
|
||||||
|
|
||||||
|
# 总进度 = 阶段偏移 + (阶段内进度 × 阶段权重 / 100)
|
||||||
|
overall = stage_offset + int(stage_percent * stage_weight / 100)
|
||||||
|
|
||||||
|
return min(overall, 100)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 设计要点
|
||||||
|
|
||||||
|
### 1. 线程安全
|
||||||
|
|
||||||
|
**问题**:Tkinter 不是线程安全的,后台线程不能直接操作 GUI。
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
A[后台线程] -->|"❌ 直接调用 GUI"| B[崩溃/未定义行为]
|
||||||
|
A -->|"✅ 写入队列"| C[queue.Queue]
|
||||||
|
C -->|"主线程读取"| D[GUI 更新]
|
||||||
|
```
|
||||||
|
|
||||||
|
**解决方案**:使用 `queue.Queue` 作为缓冲区。
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 后台线程:只写入队列
|
||||||
|
self.progress_queue.put_nowait((value, message))
|
||||||
|
|
||||||
|
# 主线程:从队列读取并更新 GUI
|
||||||
|
progress_data = self.progress_queue.get_nowait()
|
||||||
|
self.progress_bar['value'] = progress_data[0]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. 解耦设计
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
A[提取器] -->|"不需要知道 GUI"| B[回调接口]
|
||||||
|
B -->|"由 GUI 提供"| C[实现]
|
||||||
|
C -->|"可以替换"| D[测试回调<br/>日志回调<br/>GUI 回调]
|
||||||
|
```
|
||||||
|
|
||||||
|
**好处**:
|
||||||
|
- 提取器代码不依赖 GUI
|
||||||
|
- 易于测试(可以传入测试回调)
|
||||||
|
- 灵活扩展(不同场景使用不同回调)
|
||||||
|
|
||||||
|
### 3. 容错处理
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _report_progress(self, ...):
|
||||||
|
if self.progress_callback:
|
||||||
|
try:
|
||||||
|
# 调用回调
|
||||||
|
self.progress_callback(progress_info)
|
||||||
|
except Exception:
|
||||||
|
# 回调失败不影响主流程
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
**保证**:进度报告失败不会中断数据提取任务。
|
||||||
|
|
||||||
|
### 4. 准确的进度反映
|
||||||
|
|
||||||
|
**问题**:不同阶段耗时差异大(登录 3 秒,下载 60 秒)
|
||||||
|
|
||||||
|
**解决方案**:为每个阶段分配不同权重。
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
gantt
|
||||||
|
title 数据提取各阶段耗时示例
|
||||||
|
dateFormat X
|
||||||
|
axisFormat %s
|
||||||
|
|
||||||
|
section 任务
|
||||||
|
登录 :0, 3
|
||||||
|
查询 :3, 5
|
||||||
|
下载第1批 :5, 25
|
||||||
|
下载第2批 :25, 45
|
||||||
|
下载第3批 :45, 65
|
||||||
|
注销 :65, 68
|
||||||
|
转换 :68, 72
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 使用示例
|
||||||
|
|
||||||
|
### 在提取器中报告进度
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 下载批次
|
||||||
|
for batch_index, order_ids_batch in enumerate(self.group_order_ids(order_ids, self.batch_size)):
|
||||||
|
# 报告批次开始
|
||||||
|
self._report_progress(
|
||||||
|
'download',
|
||||||
|
batch_index,
|
||||||
|
total_batches,
|
||||||
|
f'正在下载第 {batch_index + 1}/{total_batches} 批',
|
||||||
|
batch_index=batch_index + 1
|
||||||
|
)
|
||||||
|
|
||||||
|
# 执行下载
|
||||||
|
downloaded_file = self.download_batch(...)
|
||||||
|
|
||||||
|
# 报告批次完成
|
||||||
|
self._report_progress(
|
||||||
|
'download',
|
||||||
|
batch_index + 1,
|
||||||
|
total_batches,
|
||||||
|
f'第 {batch_index + 1} 批下载完成'
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 在 GUI 中接收进度
|
||||||
|
|
||||||
|
```python
|
||||||
|
from gui.progress import ProgressInfo, ProgressCalculator
|
||||||
|
|
||||||
|
class DataExtractionTab(ttk.Frame):
|
||||||
|
def __init__(self, ...):
|
||||||
|
self.progress_calculator = ProgressCalculator()
|
||||||
|
self.progress_queue = queue.Queue()
|
||||||
|
self._poll_progress_queue()
|
||||||
|
|
||||||
|
def progress_callback(self, progress_info: ProgressInfo):
|
||||||
|
"""后台任务调用的回调函数"""
|
||||||
|
overall_percent = self.progress_calculator.calculate_overall_percent(progress_info)
|
||||||
|
self._update_progress(overall_percent, progress_info.message)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 总结
|
||||||
|
|
||||||
|
`_report_progress` 机制实现了:
|
||||||
|
|
||||||
|
1. **线程安全**:通过队列跨线程通信
|
||||||
|
2. **解耦设计**:提取器与 GUI 分离
|
||||||
|
3. **准确反映**:权重分配适配实际耗时
|
||||||
|
4. **容错能力**:回调失败不影响主流程
|
||||||
|
5. **易于测试**:可注入测试回调
|
||||||
|
|
||||||
|
这种模式适用于任何需要长时间运行任务并实时报告进度的场景。
|
||||||
Reference in New Issue
Block a user