c34a300f0b
Add comprehensive documentation for _report_progress mechanism: - System architecture with mermaid diagrams - Complete sequence diagram showing data flow - Progress calculation logic with stage weights - Code examples and usage patterns - Design considerations (thread safety, decoupling, fault tolerance) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
402 lines
11 KiB
Markdown
402 lines
11 KiB
Markdown
# 进度回调机制详解
|
||
|
||
## 概述
|
||
|
||
`_report_progress` 是一个基于**回调函数模式**的进度报告系统，用于后台任务（数据提取）和 GUI 主线程之间的线程安全通信。
|
||
|
||
---
|
||
|
||
## 架构设计
|
||
|
||
### 系统架构图
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
subgraph BG["后台线程 (Background Thread)"]
|
||
Extractor["DiscreteMaterialPlanExtractor"]
|
||
Report["_report_progress()"]
|
||
ProgressInfo["ProgressInfo 对象"]
|
||
end
|
||
|
||
subgraph Boundary["线程边界 (Thread Boundary)"]
|
||
Callback["progress_callback()"]
|
||
end
|
||
|
||
subgraph FG["主线程 (Main/GUI Thread)"]
|
||
Calc["ProgressCalculator<br/>计算总体百分比"]
|
||
Queue["queue.Queue<br/>线程安全队列"]
|
||
Poll["_poll_progress_queue()<br/>每50ms轮询"]
|
||
GUI["GUI 组件<br/>progress_bar<br/>status_label"]
|
||
end
|
||
|
||
Extractor -->|"调用"| Report
|
||
Report -->|"创建"| ProgressInfo
|
||
ProgressInfo -->|"触发"| Callback
|
||
Callback -->|"计算"| Calc
|
||
Calc -->|"put"| Queue
|
||
Queue -->|"get"| Poll
|
||
Poll -->|"更新"| GUI
|
||
|
||
style Callback fill:#ff9,stroke:#333,stroke-width:2px
|
||
style Queue fill:#9f9,stroke:#333,stroke-width:2px
|
||
style Boundary fill:#ddd,stroke:#333,stroke-dasharray: 5 5
|
||
```
|
||
|
||
### 组件职责
|
||
|
||
| 组件 | 职责 | 位置 |
|
||
|------|------|------|
|
||
| `DiscreteMaterialPlanExtractor` | 执行数据提取任务 | 后台线程 |
|
||
| `_report_progress()` | 报告进度到回调 | 后台线程 |
|
||
| `ProgressInfo` | 进度信息数据结构 | 跨线程 |
|
||
| `progress_callback()` | GUI 提供的回调函数 | 主线程定义，后台调用 |
|
||
| `ProgressCalculator` | 计算总体进度百分比 | 主线程 |
|
||
| `queue.Queue` | 线程安全的消息队列 | 主线程 |
|
||
| `_poll_progress_queue()` | 轮询队列并更新 GUI | 主线程 |
|
||
|
||
---
|
||
|
||
## 数据流程
|
||
|
||
### 完整时序图
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Bg as 后台线程<br/>(Extractor)
|
||
participant Report as _report_progress()
|
||
participant Callback as progress_callback()
|
||
participant Calc as ProgressCalculator
|
||
participant Queue as 进度队列
|
||
participant Poll as _poll_progress_queue()
|
||
participant GUI as GUI 组件
|
||
|
||
Bg->>Report: _report_progress('download', 2, 3, '下载中')
|
||
Report->>Report: 创建 ProgressInfo 对象
|
||
Report->>Callback: progress_callback(progress_info)
|
||
Note over Callback: 主线程定义的函数<br/>在后台线程中执行
|
||
|
||
Callback->>Calc: calculate_overall_percent(progress_info)
|
||
Note over Calc: download 阶段<br/>stage_offset=10%<br/>current=2, total=3<br/>weight=65%<br/>result = 10 + 67%×65 = 54%
|
||
Calc-->>Callback: 返回 54
|
||
|
||
Callback->>Queue: put((54, '下载中'))
|
||
Note over Queue: 线程安全队列<br/>缓冲区
|
||
|
||
loop 每 50ms
|
||
Poll->>Queue: get_nowait()
|
||
Queue-->>Poll: (54, '下载中')
|
||
Poll->>GUI: progress_bar['value'] = 54
|
||
Poll->>GUI: status_label['text'] = '下载中'
|
||
Poll->>Poll: after(50ms, 继续轮询)
|
||
end
|
||
```
|
||
|
||
### 进度计算逻辑
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
A[ProgressInfo<br/>stage=download<br/>current=2<br/>total=3] --> B[ProgressCalculator]
|
||
|
||
subgraph Calc["计算过程"]
|
||
direction TB
|
||
B --> C["计算阶段内进度<br/>2/3 × 100 = 67%"]
|
||
C --> D["查找阶段权重<br/>download = 65%"]
|
||
D --> E["查找阶段偏移<br/>offset = 10%"]
|
||
E --> F["总体进度<br/>10 + 67%×65 = 54%"]
|
||
end
|
||
|
||
F --> G["更新进度条<br/>54%"]
|
||
```
|
||
|
||
---
|
||
|
||
## 阶段权重分配
|
||
|
||
### 进度阶段划分
|
||
|
||
```mermaid
|
||
pie title 各阶段权重分布
|
||
"登录 (5%)" : 5
|
||
"查询 (5%)" : 5
|
||
"下载 (65%)" : 65
|
||
"注销 (5%)" : 5
|
||
"转换 (15%)" : 15
|
||
"完成 (5%)" : 5
|
||
```
|
||
|
||
### 阶段详情表
|
||
|
||
| 阶段 | stage | 权重 | 进度范围 | 说明 |
|
||
|------|-------|------|----------|------|
|
||
| 登录 | `login` | 5% | 0-5% | ERP 系统登录 |
|
||
| 查询 | `query` | 5% | 5-10% | 查询数据库获取订单号 |
|
||
| 下载 | `download` | 65% | 10-75% | 批量下载数据（主要耗时） |
|
||
| 注销 | `logout` | 5% | 75-80% | 退出 ERP 系统 |
|
||
| 转换 | `convert` | 15% | 80-95% | 转换 Excel 格式并合并 |
|
||
| 完成 | `complete` | 5% | 95-100% | 任务完成 |
|
||
|
||
---
|
||
|
||
## 代码实现
|
||
|
||
### 1. 后台任务：报告进度
|
||
|
||
```python
|
||
# utils/离散备料计划维护数据提取.py
|
||
|
||
def _report_progress(self, stage: str, current: int, total: int, message: str, **detail):
|
||
"""
|
||
报告进度
|
||
|
||
Args:
|
||
stage: 阶段标识 ('login', 'query', 'download', 等)
|
||
current: 当前进度值 (1, 2, 3...)
|
||
total: 总量 (3, 100...)
|
||
message: 显示给用户的消息
|
||
**detail: 额外信息 (如 batch_index=1)
|
||
"""
|
||
if self.progress_callback:
|
||
try:
|
||
from gui.progress import ProgressInfo
|
||
progress_info = ProgressInfo(
|
||
stage=stage,
|
||
current=current,
|
||
total=total,
|
||
message=message,
|
||
detail=detail
|
||
)
|
||
# 调用 GUI 提供的回调函数
|
||
self.progress_callback(progress_info)
|
||
except Exception:
|
||
# 回调失败不影响主流程
|
||
pass
|
||
```
|
||
|
||
### 2. GUI：设置回调
|
||
|
||
```python
|
||
# gui/data_extraction_tab.py
|
||
|
||
def _extraction_worker(self, input_file: str, output_file: str):
|
||
"""后台工作线程"""
|
||
|
||
# 创建进度回调函数
|
||
def progress_callback(progress_info: ProgressInfo):
|
||
# 1. 计算总体进度百分比
|
||
overall_percent = self.progress_calculator.calculate_overall_percent(progress_info)
|
||
# 2. 放入队列（线程安全）
|
||
self._update_progress(overall_percent, progress_info.message)
|
||
|
||
# 将回调传递给提取器
|
||
self.extractor.extract(
|
||
production_id_file=input_file,
|
||
output_file=output_file,
|
||
progress_callback=progress_callback
|
||
)
|
||
```
|
||
|
||
### 3. 线程安全：队列通信
|
||
|
||
```python
|
||
# gui/data_extraction_tab.py
|
||
|
||
def _update_progress(self, value: int, message: str):
|
||
"""后台线程调用，放入队列"""
|
||
try:
|
||
self.progress_queue.put_nowait((value, message))
|
||
except:
|
||
pass # 队列满时忽略
|
||
|
||
def _poll_progress_queue(self):
|
||
"""主线程轮询，更新 GUI"""
|
||
try:
|
||
while True:
|
||
# 非阻塞获取队列中的消息
|
||
progress_data = self.progress_queue.get_nowait()
|
||
value, message = progress_data
|
||
# 更新 GUI 组件
|
||
self.progress_bar['value'] = value
|
||
self.status_label.config(text=message)
|
||
except queue.Empty:
|
||
pass
|
||
finally:
|
||
# 继续轮询（每 50ms 检查一次）
|
||
self.after(50, self._poll_progress_queue)
|
||
```
|
||
|
||
### 4. 进度计算器
|
||
|
||
```python
|
||
# gui/progress.py
|
||
|
||
class ProgressCalculator:
|
||
# 各阶段在总进度中的占比
|
||
STAGE_WEIGHTS = {
|
||
'login': 5, # 0-5%
|
||
'query': 5, # 5-10%
|
||
'download': 65, # 10-75%
|
||
'logout': 5, # 75-80%
|
||
'convert': 15, # 80-95%
|
||
'complete': 5, # 95-100%
|
||
}
|
||
|
||
def calculate_overall_percent(self, progress: ProgressInfo) -> int:
|
||
"""计算总体进度百分比"""
|
||
stage = progress.stage
|
||
|
||
if stage == 'complete':
|
||
return 100
|
||
|
||
# 计算阶段起始百分比
|
||
stage_offset = self._stage_offsets[stage]
|
||
|
||
# 计算阶段内的进度百分比
|
||
stage_percent = progress.percent # current/total * 100
|
||
|
||
# 计算该阶段的权重
|
||
stage_weight = self.STAGE_WEIGHTS[stage]
|
||
|
||
# 总进度 = 阶段偏移 + (阶段内进度 × 阶段权重 / 100)
|
||
overall = stage_offset + int(stage_percent * stage_weight / 100)
|
||
|
||
return min(overall, 100)
|
||
```
|
||
|
||
---
|
||
|
||
## 设计要点
|
||
|
||
### 1. 线程安全
|
||
|
||
**问题**：Tkinter 不是线程安全的，后台线程不能直接操作 GUI。
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
A[后台线程] -->|"❌ 直接调用 GUI"| B[崩溃/未定义行为]
|
||
A -->|"✅ 写入队列"| C[queue.Queue]
|
||
C -->|"主线程读取"| D[GUI 更新]
|
||
```
|
||
|
||
**解决方案**：使用 `queue.Queue` 作为缓冲区。
|
||
|
||
```python
|
||
# 后台线程：只写入队列
|
||
self.progress_queue.put_nowait((value, message))
|
||
|
||
# 主线程：从队列读取并更新 GUI
|
||
progress_data = self.progress_queue.get_nowait()
|
||
self.progress_bar['value'] = progress_data[0]
|
||
```
|
||
|
||
### 2. 解耦设计
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
A[提取器] -->|"不需要知道 GUI"| B[回调接口]
|
||
B -->|"由 GUI 提供"| C[实现]
|
||
C -->|"可以替换"| D[测试回调<br/>日志回调<br/>GUI 回调]
|
||
```
|
||
|
||
**好处**：
|
||
- 提取器代码不依赖 GUI
|
||
- 易于测试（可以传入测试回调）
|
||
- 灵活扩展（不同场景使用不同回调）
|
||
|
||
### 3. 容错处理
|
||
|
||
```python
|
||
def _report_progress(self, ...):
|
||
if self.progress_callback:
|
||
try:
|
||
# 调用回调
|
||
self.progress_callback(progress_info)
|
||
except Exception:
|
||
# 回调失败不影响主流程
|
||
pass
|
||
```
|
||
|
||
**保证**：进度报告失败不会中断数据提取任务。
|
||
|
||
### 4. 准确的进度反映
|
||
|
||
**问题**：不同阶段耗时差异大（登录 3 秒，下载 60 秒）
|
||
|
||
**解决方案**：为每个阶段分配不同权重。
|
||
|
||
```mermaid
|
||
gantt
|
||
title 数据提取各阶段耗时示例
|
||
dateFormat X
|
||
axisFormat %s
|
||
|
||
section 任务
|
||
登录 :0, 3
|
||
查询 :3, 5
|
||
下载第1批 :5, 25
|
||
下载第2批 :25, 45
|
||
下载第3批 :45, 65
|
||
注销 :65, 68
|
||
转换 :68, 72
|
||
```
|
||
|
||
---
|
||
|
||
## 使用示例
|
||
|
||
### 在提取器中报告进度
|
||
|
||
```python
|
||
# 下载批次
|
||
for batch_index, order_ids_batch in enumerate(self.group_order_ids(order_ids, self.batch_size)):
|
||
# 报告批次开始
|
||
self._report_progress(
|
||
'download',
|
||
batch_index,
|
||
total_batches,
|
||
f'正在下载第 {batch_index + 1}/{total_batches} 批',
|
||
batch_index=batch_index + 1
|
||
)
|
||
|
||
# 执行下载
|
||
downloaded_file = self.download_batch(...)
|
||
|
||
# 报告批次完成
|
||
self._report_progress(
|
||
'download',
|
||
batch_index + 1,
|
||
total_batches,
|
||
f'第 {batch_index + 1} 批下载完成'
|
||
)
|
||
```
|
||
|
||
### 在 GUI 中接收进度
|
||
|
||
```python
|
||
from gui.progress import ProgressInfo, ProgressCalculator
|
||
|
||
class DataExtractionTab(ttk.Frame):
|
||
def __init__(self, ...):
|
||
self.progress_calculator = ProgressCalculator()
|
||
self.progress_queue = queue.Queue()
|
||
self._poll_progress_queue()
|
||
|
||
def progress_callback(self, progress_info: ProgressInfo):
|
||
"""后台任务调用的回调函数"""
|
||
overall_percent = self.progress_calculator.calculate_overall_percent(progress_info)
|
||
self._update_progress(overall_percent, progress_info.message)
|
||
```
|
||
|
||
---
|
||
|
||
## 总结
|
||
|
||
`_report_progress` 机制实现了：
|
||
|
||
1. **线程安全**：通过队列跨线程通信
|
||
2. **解耦设计**：提取器与 GUI 分离
|
||
3. **准确反映**：权重分配适配实际耗时
|
||
4. **容错能力**：回调失败不影响主流程
|
||
5. **易于测试**：可注入测试回调
|
||
|
||
这种模式适用于任何需要长时间运行任务并实时报告进度的场景。
|