683 lines
30 KiB
Markdown
683 lines
30 KiB
Markdown
# 指定日期下载接口(开发者)Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 给 `POST /tasks` 加可选 `date`(YYYY-MM-DD),支持顺心/中通/韵达/安能指定一个过去日期下载应到/实到数据;不传 date 时行为不变(走 offset)。
|
||
|
||
**Architecture:** 复用现有 `force` 透传链路加一个 `date` 字段:`TaskRequest.date` → `task_spec["date"]` → `dispatch_task` → `handler(ctx, force, date)` → 各站 `impl(page, force, date)`。各站把"算 target 日期"的来源从 `today - offset` 改为"有 date 用 date,否则 today - offset"。中通把 date 折算成 effective offset 以复用已验证的跨月翻页。
|
||
|
||
**Tech Stack:** Python 3.10、FastAPI、Playwright、SQLite(state_store)、PostgreSQL(store,可选入库)。
|
||
|
||
## Global Constraints
|
||
|
||
- **环境**:所有 Python 用 `D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe`(项目虚拟环境,无需激活)。PYTHONPATH 含 `InboundVerify` 根。
|
||
- **格式化/自检(每改一个 .py 必做)**:`python -m black <file>`(target py310)+ `python -m py_compile <file>`。Black 若提示 "Python 3.10 cannot parse code formatted for 3.15",加 `--target-version py310`;"left unchanged" 即合格。
|
||
- **无 pytest**:本项目无单元测试框架(见 `InboundVerify/CLAUDE.md`)。每个任务的"验证"= Black + py_compile;端到端(API 校验、各站实测)集中在 Task 8(需重启服务加载新代码)。
|
||
- **submodule 工作流**:改动在 `InboundVerify` submodule(`dev` 分支)。每个 Task 末尾在 submodule 内 `git add <file> && git commit`。**push 到 origin/dev + 父仓库 bump** 统一在 Task 8(项目约定不自动 push,等用户确认;但 plan 内 commit 步骤照写)。
|
||
- **顺序安全**:Task 1-5(站点)只给 impl 加 `date=None` 形参 + date 逻辑,**date 默认 None 时走原 offset 路径,向后兼容**;Task 6(runtime)才把 date 从 task_spec 透传进 impl;Task 7(server)才允许 date 入队。任一 Task 完成后系统均可正常运行。
|
||
- **合法性边界**(来自 spec):`date` 传了才校验——格式 `YYYY-MM-DD`、`今天-90 ≤ date ≤ 今天`、百世不支持 date。非法返回 HTTP 400。
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
| 文件 | 责任 | 本计划改动 |
|
||
| --- | --- | --- |
|
||
| `inbound_verify/sites/baishi.py` | 百世下载(固定当天) | 入口加 `date=None` 形参(忽略) |
|
||
| `inbound_verify/sites/zto.py` | 中通下载(日历格子,跨月) | 入口+impl 加 `date`;date→effective offset 复用跨月 |
|
||
| `inbound_verify/sites/yunda.py` | 韵达下载(日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||
| `inbound_verify/sites/shunxin.py` | 顺心下载(双账号,日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||
| `inbound_verify/sites/anneng.py` | 安能下载(CDP,日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||
| `inbound_verify/runtime.py` | 任务派发/心跳共享核心 | handler/dispatch 透传 date;`_record_business_date` 用 date |
|
||
| `inbound_verify/cli/server.py` | FastAPI 服务 | `TaskRequest.date` + 合法性校验 + task_spec 透传 |
|
||
|
||
---
|
||
|
||
## Task 1: baishi.py — 入口加 date=None 形参(兼容)
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/sites/baishi.py:140`(`baishi_download_undelivered_data`)
|
||
|
||
**Interfaces:**
|
||
- Produces: `baishi_download_undelivered_data(page, force=False, date=None)` —— 后续 Task 6 的 `_web_handler` 会以 `download_func(pg, force=force, date=date)` 调用它,必须接受 `date` kwarg(百世忽略)。
|
||
|
||
- [ ] **Step 1: 改签名**
|
||
|
||
把 `def baishi_download_undelivered_data(page, force=False):` 改为:
|
||
|
||
```python
|
||
def baishi_download_undelivered_data(page, force=False, date=None):
|
||
"""百世:应到未到数据下载(固定当天;date 形参仅为对齐统一透传签名,忽略)。"""
|
||
```
|
||
|
||
(函数体不动;`date` 不使用。)
|
||
|
||
- [ ] **Step 2: Black + py_compile**
|
||
|
||
```bash
|
||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/sites/baishi.py"
|
||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||
```
|
||
Expected: black "left unchanged" 或 reformat 后通过;OK。
|
||
|
||
- [ ] **Step 3: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/baishi.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(baishi): accept date kwarg (ignored) for unified dispatch signature" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 2: zto.py — date 折算成 effective offset(复用跨月)
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/sites/zto.py` — `zto_expected_download`、`zto_expected_download_impl`、`zto_actual_download`、`zto_actual_download_impl`
|
||
|
||
**Interfaces:**
|
||
- Produces: `zto_expected_download(page, force=False, date=None)` / `zto_actual_download(page, force=False, date=None)`,impl 同签名。Task 6 的 `_web_handler` 以 `download_func(pg, force=force, date=date)` 调用。
|
||
|
||
- [ ] **Step 1: expected 入口加 date 并透传**
|
||
|
||
`def zto_expected_download(page, force=False):` 及其 `with_retry` 改为:
|
||
|
||
```python
|
||
def zto_expected_download(page, force=False, date=None):
|
||
"""中通:应到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||
|
||
return with_retry(
|
||
"中通",
|
||
"应到",
|
||
lambda: zto_expected_download_impl(page, force=force, date=date),
|
||
lambda: zto_reset(page),
|
||
)
|
||
```
|
||
|
||
- [ ] **Step 2: expected impl 加 date,date→effective offset**
|
||
|
||
`def zto_expected_download_impl(page, force=False):` 改签名加 `date=None`。其内"读取服务端日期偏移"段(`offset = state_store.get_offset("中通")` 与随后的 `print(...偏移...)`)改为:
|
||
|
||
```python
|
||
# 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||
offset = state_store.get_offset("中通")
|
||
if date:
|
||
# 指定日期:折算成相对今天的有效偏移,复用下方 target_time 计算与跨月翻月
|
||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||
offset = (datetime.now().date() - target_date).days
|
||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||
else:
|
||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||
```
|
||
|
||
(其后的 `target_time = today_time - offset * 86400000` 与跨月翻月逻辑**不动**——date 经折算后走同一条路径。)
|
||
|
||
- [ ] **Step 3: actual 入口加 date 并透传**
|
||
|
||
`def zto_actual_download(page, force=False):` 及其 `with_retry` 改为:
|
||
|
||
```python
|
||
def zto_actual_download(page, force=False, date=None):
|
||
"""中通:实到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||
|
||
return with_retry(
|
||
"中通",
|
||
"实到",
|
||
lambda: zto_actual_download_impl(page, date=date),
|
||
lambda: zto_reset(page),
|
||
)
|
||
```
|
||
|
||
> 注:`zto_actual_download_impl` 现签名 `(page)`(无 force,actual 不去重),Step 4 给它加 `date`。
|
||
|
||
- [ ] **Step 4: actual impl 加 date,date→effective offset**
|
||
|
||
`def zto_actual_download_impl(page):` 改为 `def zto_actual_download_impl(page, date=None):`。其内"# 2. 读取服务端日期偏移"段(`offset = state_store.get_offset("中通", "actual")` 与随后的 `print`)改为:
|
||
|
||
```python
|
||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||
offset = state_store.get_offset("中通", "actual")
|
||
if date:
|
||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||
offset = (datetime.now().date() - target_date).days
|
||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||
else:
|
||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||
```
|
||
|
||
(其后 `target_time = today_time - offset * 86400000` 与跨月翻月不动。)
|
||
|
||
- [ ] **Step 5: Black + py_compile**
|
||
|
||
```bash
|
||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/sites/zto.py"
|
||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||
```
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/zto.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(zto): support date arg via effective-offset (reuses cross-month nav)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 3: yunda.py — date→target(expected + actual)
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/sites/yunda.py` — `yunda_expected_download(_impl)`、`yunda_actual_download(_impl)`
|
||
|
||
**Interfaces:**
|
||
- Produces: `yunda_expected_download(page, force=False, date=None)` / `yunda_actual_download(page, force=False, date=None)`,impl 同加 `date=None`。
|
||
|
||
- [ ] **Step 1: expected 入口透传 date**
|
||
|
||
```python
|
||
def yunda_expected_download(page, force=False, date=None):
|
||
...
|
||
return with_retry(
|
||
"韵达",
|
||
"应到",
|
||
lambda: yunda_expected_download_impl(page, force=force, date=date),
|
||
lambda: yunda_reset(page),
|
||
)
|
||
```
|
||
|
||
- [ ] **Step 2: expected impl 加 date + date→target**
|
||
|
||
`def yunda_expected_download_impl(page, force=False):` → `def yunda_expected_download_impl(page, force=False, date=None):`。其内日期段(`offset = state_store.get_offset("韵达")` 起几行)改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("韵达")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
target_ymd = f"{target.year}-{target.month}-{target.day}"
|
||
start_date_ymd = target_ymd
|
||
today_ymd = target_ymd
|
||
print(f">> 设置查询日期: [{target_ymd}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||
```
|
||
|
||
- [ ] **Step 3: actual 入口透传 date**
|
||
|
||
```python
|
||
def yunda_actual_download(page, force=False, date=None):
|
||
...
|
||
return with_retry(
|
||
"韵达",
|
||
"实到",
|
||
lambda: yunda_actual_download_impl(page, date=date),
|
||
lambda: yunda_reset(page),
|
||
)
|
||
```
|
||
|
||
- [ ] **Step 4: actual impl 加 date + date→target**
|
||
|
||
`def yunda_actual_download_impl(page):` → `def yunda_actual_download_impl(page, date=None):`。其内日期段改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("韵达", "actual")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
start_date = target # 单日范围:起止同日
|
||
today = target # 让下方"截止时间"选择器也指向 target
|
||
print(
|
||
f">> 设置实到查询日期: [{target.year}-{target.month}-{target.day}]"
|
||
f"({'指定 ' + date if date else f'偏移 {offset},0=今天'})"
|
||
)
|
||
```
|
||
|
||
- [ ] **Step 5: Black + py_compile**(同 Task 2 命令,文件换 yunda.py)
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/yunda.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(yunda): support date arg (date takes precedence over offset)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 4: shunxin.py — date→target(双账号透传)
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/sites/shunxin.py` — `shunxin_expected_download(_impl)`、`shunxin_actual_download(_impl)`
|
||
|
||
**Interfaces:**
|
||
- Produces: `shunxin_expected_download(pages, foreground=True, force=False, date=None)` / `shunxin_actual_download(pages, foreground=True, force=False, date=None)`。Task 6 的 `_web_handler` 以 `download_func(pg, foreground=ctx.foreground, force=force, date=date)` 调用(pg 是 page 列表)。
|
||
|
||
- [ ] **Step 1: expected 入口加 date 并向 impl 透传**
|
||
|
||
`def shunxin_expected_download(pages, foreground=True, force=False):` → 加 `date=None`。在函数内调用 `shunxin_expected_download_impl(page, out_tag=..., force=force)` 的位置,加上 `date=date`:
|
||
|
||
```python
|
||
def shunxin_expected_download(pages, foreground=True, force=False, date=None):
|
||
...
|
||
# 对每个账号调用 impl 时透传 date:
|
||
... shunxin_expected_download_impl(page, out_tag=归属地, force=force, date=date) ...
|
||
```
|
||
|
||
> 执行者:用 Grep 定位 `shunxin_expected_download_impl(page,` 的调用处(在 `shunxin_expected_download` 函数体内,对每个账号调用一次),给每处加 `date=date`。签名行只加 `date=None`,其余函数体(归属地读取、去重校验、merge)不动。
|
||
|
||
- [ ] **Step 2: expected impl 加 date + date→target**
|
||
|
||
`def shunxin_expected_download_impl(page, out_tag="", force=False):` → 加 `date=None`。其内日期段(`offset = state_store.get_offset("顺心")` 起几行)改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("顺心")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
target_str = target.strftime("%Y-%m-%d")
|
||
start_date_str = target_str
|
||
today_str = target_str
|
||
print(f">> 正在设置查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})...")
|
||
```
|
||
|
||
- [ ] **Step 3: actual 入口加 date 并透传**
|
||
|
||
`def shunxin_actual_download(pages, foreground=True, force=False):` → 加 `date=None`;调用 `shunxin_actual_download_impl(page, out_tag=...)` 处加 `date=date`。
|
||
|
||
- [ ] **Step 4: actual impl 加 date + date→target**
|
||
|
||
`def shunxin_actual_download_impl(page, out_tag=""):` → `def shunxin_actual_download_impl(page, out_tag="", date=None):`。其内日期段(`offset = state_store.get_offset("顺心", "actual")` 起几行)改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("顺心", "actual")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
target_str = target.strftime("%Y-%m-%d")
|
||
start_date_str = target_str
|
||
today_str = target_str
|
||
print(f">> 正在设置查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})...")
|
||
```
|
||
|
||
- [ ] **Step 5: Black + py_compile**(文件 shunxin.py)
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/shunxin.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(shunxin): support date arg, propagate to both accounts" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 5: anneng.py — date→target(CDP,expected + actual)
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/sites/anneng.py` — `anneng_expected_download(_impl)`、`anneng_actual_download(_impl)`
|
||
|
||
**Interfaces:**
|
||
- Produces: `anneng_expected_download(force=False, date=None)` / `anneng_actual_download(force=False, date=None)`。Task 6 的 TASK_HANDLERS 安能项以 `lambda ctx, force=False, date=None: anneng.anneng_expected_download(force=force, date=date)` 调用。
|
||
|
||
- [ ] **Step 1: expected 入口 + impl 加 date,date→target**
|
||
|
||
```python
|
||
def anneng_expected_download(force=False, date=None):
|
||
return with_retry(
|
||
"安能", "应到", lambda: anneng_expected_download_impl(force=force, date=date), anneng_reset
|
||
)
|
||
|
||
|
||
def anneng_expected_download_impl(force=False, date=None):
|
||
```
|
||
|
||
expected impl 内日期段(`offset = state_store.get_offset("安能")` 起几行)改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("安能")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
target_str = f"{target.year}-{target.month:02d}-{target.day:02d}"
|
||
start_str = target_str
|
||
today_str = target_str
|
||
print(f">> 查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||
```
|
||
|
||
- [ ] **Step 2: actual 入口 + impl 加 date,date→target**
|
||
|
||
```python
|
||
def anneng_actual_download(force=False, date=None):
|
||
return with_retry("安能", "实到", lambda: anneng_actual_download_impl(date=date), anneng_reset)
|
||
|
||
|
||
def anneng_actual_download_impl(date=None):
|
||
```
|
||
|
||
actual impl 内日期段(`offset = state_store.get_offset("安能", "actual")` 起几行)改为:
|
||
|
||
```python
|
||
offset = state_store.get_offset("安能", "actual")
|
||
today = datetime.now()
|
||
if date:
|
||
target = datetime.strptime(date, "%Y-%m-%d")
|
||
else:
|
||
target = today - timedelta(days=offset)
|
||
start_str = f"{target.year}\{target.month:02d}/{target.day:02d} 00:00:00"
|
||
end_str = f"{target.year}\{target.month:02d}/{target.day:02d} 23:59:59"
|
||
print(f">> 扫描日期: [{start_str} 至 {end_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||
```
|
||
|
||
> 注:actual 的 `start_str/end_str` 沿用现有 `{year}\{month}/{day}` 格式(含反斜杠,站点如此),只把 target 来源改成 date。
|
||
|
||
- [ ] **Step 3: Black + py_compile**(文件 anneng.py)
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/anneng.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(anneng): support date arg (date takes precedence over offset)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 6: runtime.py — 全链路透传 date + _record_business_date 用 date
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/runtime.py` — `_web_handler`、`_site_undelivered_handler`、`TASK_HANDLERS` 安能项、`dispatch_task`、`_record_business_date`
|
||
|
||
**Interfaces:**
|
||
- Consumes: Task 1-5 产出的各站 `*(..., date=None)` 签名。
|
||
- Produces: `dispatch_task` 从 `task_spec["date"]` 读 date 透传给 handler 与 `_record_business_date`;handler 签名 `(ctx, force=False, date=None)`。
|
||
|
||
- [ ] **Step 1: _web_handler 透传 date**
|
||
|
||
```python
|
||
def _web_handler(site, download_func):
|
||
def handler(ctx, force=False, date=None):
|
||
pg = ctx.pages_map[site]
|
||
if isinstance(pg, list):
|
||
# 顺心双账号:置顶与否交给 shunxin_download 在逐账号循环里按 foreground 决定
|
||
return download_func(pg, foreground=ctx.foreground, force=force, date=date)
|
||
if ctx.foreground:
|
||
pg.bring_to_front()
|
||
return download_func(pg, force=force, date=date)
|
||
|
||
return handler
|
||
```
|
||
|
||
- [ ] **Step 2: _site_undelivered_handler 透传 date(连下 expected+actual 共用同一 date)**
|
||
|
||
```python
|
||
def _site_undelivered_handler(site):
|
||
def handler(ctx, force=False, date=None):
|
||
exp_ok = TASK_HANDLERS[(site, "expected")](ctx, force, date) is not False
|
||
act_ok = (
|
||
(TASK_HANDLERS[(site, "actual")](ctx, force, date) is not False)
|
||
if exp_ok
|
||
else False
|
||
)
|
||
if exp_ok and act_ok:
|
||
return compare.write_site_file(site)
|
||
stale = os.path.join(DOWNLOAD_DIR, SITE_UNDELIVERED_FILE.format(name=site))
|
||
if os.path.exists(stale):
|
||
os.remove(stale)
|
||
return False
|
||
|
||
return handler
|
||
```
|
||
|
||
- [ ] **Step 3: TASK_HANDLERS 安能项透传 date**
|
||
|
||
```python
|
||
("安能", "expected"): lambda ctx, force=False, date=None: anneng.anneng_expected_download(
|
||
force=force, date=date
|
||
),
|
||
("安能", "actual"): lambda ctx, force=False, date=None: anneng.anneng_actual_download(
|
||
force=force, date=date
|
||
),
|
||
```
|
||
|
||
(`__compare__` 项与百世/网页项不动——百世经 `_web_handler` 已透传 date,baishi 忽略。)
|
||
|
||
- [ ] **Step 4: dispatch_task 透传 date**
|
||
|
||
在 `dispatch_task` 内,把 `ret = handler(ctx, bool(task_spec.get("force", False)))` 改为:
|
||
|
||
```python
|
||
ret = handler(
|
||
ctx, bool(task_spec.get("force", False)), task_spec.get("date")
|
||
)
|
||
if ret is False:
|
||
return (state_store.TASK_FAILED, "任务执行失败(重试耗尽)")
|
||
_record_business_date(site, kind, task_spec.get("date"))
|
||
_persist_to_db(site, kind)
|
||
return (state_store.TASK_SUCCESS, None)
|
||
```
|
||
|
||
- [ ] **Step 5: _record_business_date 接受 date**
|
||
|
||
```python
|
||
def _record_business_date(site, kind, date=None):
|
||
"""下载成功后,把本次数据的业务日期快照写进状态库(供前端/报告显示「是哪天的数据」)。
|
||
有 date 用 date;否则 = 下载当天 − 该数据对应的日期偏移。__compare__ 无数据概念,跳过。"""
|
||
if site == "__compare__":
|
||
return
|
||
today = datetime.now().date()
|
||
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||
|
||
def _write(k, biz_or_off):
|
||
# biz_or_off: int=offset(today−offset);str=已确定的业务日期(date)
|
||
biz = (
|
||
(today - timedelta(days=biz_or_off)).strftime("%Y-%m-%d")
|
||
if isinstance(biz_or_off, int)
|
||
else biz_or_off
|
||
)
|
||
try:
|
||
state_store.set_data_state(
|
||
site, k, ready=True, generated_at=now, business_date=biz
|
||
)
|
||
except Exception as e:
|
||
print(f">> [状态] 写业务日期失败 {site}/{k}: {e}")
|
||
|
||
def off(kind_key):
|
||
return state_store.get_offset(site, kind_key)
|
||
|
||
if kind == "expected":
|
||
_write("expected", date if date else off("expected"))
|
||
elif kind == "actual":
|
||
_write("actual", date if date else off("actual"))
|
||
elif site == "百世":
|
||
_write("undelivered", 0)
|
||
else: # 4 站 undelivered:连带补写 expected/actual/undelivered 三列
|
||
_write("expected", date if date else off("expected"))
|
||
_write("actual", date if date else off("actual"))
|
||
_write("undelivered", date if date else off("expected"))
|
||
```
|
||
|
||
- [ ] **Step 6: Black + py_compile**
|
||
|
||
```bash
|
||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/runtime.py"
|
||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||
```
|
||
|
||
- [ ] **Step 7: 冒烟(date=None 行为不变)**
|
||
|
||
服务仍跑旧代码,但 runtime 模块可独立 import 校验:
|
||
|
||
```bash
|
||
PYTHONPATH="D:/projects/LogisticsHubIPA/InboundVerify" "$PY" -c "import inbound_verify.runtime as r; import inspect; print('handler', inspect.signature(r._web_handler('中通', lambda *a, **k: None))); print('rec', inspect.signature(r._record_business_date))"
|
||
```
|
||
Expected: handler 含 `(ctx, force=False, date=None)`;rec 含 `(site, kind, date=None)`。
|
||
|
||
- [ ] **Step 8: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/runtime.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(runtime): propagate date through dispatch chain and business-date snapshot" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 7: server.py — TaskRequest.date + 合法性校验 + task_spec
|
||
|
||
**Files:**
|
||
- Modify: `inbound_verify/cli/server.py` — 顶部 import、`TaskRequest`、`create_task`
|
||
|
||
**Interfaces:**
|
||
- Produces: `POST /tasks` 接受 `{site, kind, force, date}`;合法 date 入队为 `task_spec["date"]`(YYYY-MM-DD),非法返回 400。
|
||
|
||
- [ ] **Step 1: 顶部 import 加 timedelta**
|
||
|
||
`from datetime import datetime` → `from datetime import datetime, timedelta`
|
||
|
||
- [ ] **Step 2: TaskRequest 加 date**
|
||
|
||
```python
|
||
class TaskRequest(BaseModel):
|
||
site: str
|
||
kind: str
|
||
force: bool = False
|
||
date: Optional[str] = None # YYYY-MM-DD;指定则下载该日数据,否则走站点 offset
|
||
```
|
||
|
||
- [ ] **Step 3: create_task 加合法性校验 + task_spec 透传 date**
|
||
|
||
```python
|
||
@app.post("/tasks")
|
||
def create_task(req: TaskRequest):
|
||
"""提交任务 {site, kind, force, date?} → 入队,返回 task_id。"""
|
||
if not worker_state["ready"]:
|
||
raise HTTPException(
|
||
status_code=409, detail="后端尚未就绪,请等待各站点登录完成后再操作"
|
||
)
|
||
if (req.site, req.kind) not in TASK_HANDLERS:
|
||
raise HTTPException(status_code=400, detail=f"无效任务: {req.site}/{req.kind}")
|
||
# 指定日期合法性校验(仅在传了 date 时)
|
||
if req.date:
|
||
try:
|
||
target_date = datetime.strptime(req.date, "%Y-%m-%d").date()
|
||
except ValueError:
|
||
raise HTTPException(
|
||
status_code=400, detail=f"date 格式非法,需 YYYY-MM-DD: {req.date}"
|
||
)
|
||
today = datetime.now().date()
|
||
if target_date > today:
|
||
raise HTTPException(
|
||
status_code=400, detail=f"date 不可为未来日期: {req.date}"
|
||
)
|
||
if target_date < today - timedelta(days=90):
|
||
raise HTTPException(
|
||
status_code=400, detail=f"date 超出 90 天回溯上限: {req.date}"
|
||
)
|
||
if req.site == "百世":
|
||
raise HTTPException(
|
||
status_code=400, detail="百世固定下载当天,不支持指定日期"
|
||
)
|
||
task_id = state_store.create_task(req.site, req.kind)
|
||
spec = {"site": req.site, "kind": req.kind, "force": req.force}
|
||
if req.date:
|
||
spec["date"] = req.date
|
||
task_queue.put((task_id, spec))
|
||
return {"task_id": task_id}
|
||
```
|
||
|
||
- [ ] **Step 4: Black + py_compile**
|
||
|
||
```bash
|
||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/cli/server.py"
|
||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||
```
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/cli/server.py
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(server): add date field to POST /tasks with legality validation" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 8: 端到端验证 + 收尾(push / 父 bump)
|
||
|
||
**Files:** 无代码改动;验证 + 提交推送。
|
||
|
||
- [ ] **Step 1: 重启服务加载全部新代码**
|
||
|
||
停掉旧服务进程,重启(中通调试模式,config.yaml 已是中通):
|
||
|
||
```bash
|
||
PYTHONPATH="D:/projects/LogisticsHubIPA/InboundVerify" "D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe" -m inbound_verify.cli.server
|
||
```
|
||
(后台运行;等中通登录就绪。)
|
||
|
||
- [ ] **Step 2: 合法性校验端到端(API 400/202)**
|
||
|
||
用 Python urllib(避免 curl 中文编码问题)逐一验证,期望:
|
||
|
||
| 请求 | 期望 |
|
||
| --- | --- |
|
||
| `{site:"中通", kind:"expected", date:"2026-06-14"}` | 202 + task_id |
|
||
| `date:"2099-01-01"`(未来) | 400 "不可为未来日期" |
|
||
| `date:"2020-01-01"`(超 90 天) | 400 "超出 90 天回溯上限" |
|
||
| `date:"2026/06/14"`(格式错) | 400 "格式非法" |
|
||
| `{site:"百世", kind:"undelivered", date:"2026-06-14"}` | 400 "百世…不支持指定日期" |
|
||
| `{site:"中通", kind:"expected"}`(不传 date) | 202(走 offset,行为不变) |
|
||
|
||
```bash
|
||
"D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe" - <<'PYEOF'
|
||
import json, urllib.request, urllib.error
|
||
def post(body):
|
||
data = json.dumps(body).encode("utf-8")
|
||
req = urllib.request.Request("http://127.0.0.1:8000/tasks", data=data,
|
||
headers={"Content-Type":"application/json"}, method="POST")
|
||
try:
|
||
print(body, "->", urllib.request.urlopen(req, timeout=10).read().decode())
|
||
except urllib.error.HTTPError as e:
|
||
print(body, "->", e.code, e.read().decode())
|
||
post({"site":"中通","kind":"expected","date":"2099-01-01"})
|
||
post({"site":"中通","kind":"expected","date":"2020-01-01"})
|
||
post({"site":"中通","kind":"expected","date":"2026/06/14"})
|
||
post({"site":"百世","kind":"undelivered","date":"2026-06-14"})
|
||
PYEOF
|
||
```
|
||
|
||
- [ ] **Step 3: 中通跨月 date 实测(已知 OK)**
|
||
|
||
触发 `{site:"中通", kind:"expected", date:"2026-06-14"}`,观察 worker 日志:出现 `指定日期 2026-06-14(折算偏移 …)` + `偏移日期跨月,正在向前翻月导航` + 查询/下载成功。任务 `success`。
|
||
|
||
- [ ] **Step 4: 顺心/韵达/安能 date 实测**
|
||
|
||
对顺心/韵达/安能各触发一个 expected `date`(取一个近 1 周内的过去日期,确保站点有数据且控件能接受)。观察日志:`指定日期 …` + 正常查询下载。**若某站日期控件不接受字符串而需日历翻月**,记录现象,按中通同法(_zto_flip 模式)追加改动(可能产生新 Task)。
|
||
|
||
- [ ] **Step 5: 业务日期快照校验**
|
||
|
||
下载后 `curl -s http://127.0.0.1:8000/status`,确认对应站 `expected_business_date` == 指定 date。
|
||
|
||
- [ ] **Step 6: push submodule dev + 父仓库 bump**
|
||
|
||
```bash
|
||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" push origin dev
|
||
git -C "D:/projects/LogisticsHubIPA" add InboundVerify
|
||
git -C "D:/projects/LogisticsHubIPA" commit -m "chore: bump InboundVerify submodule (date-specific download API)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||
git -C "D:/projects/LogisticsHubIPA" push origin master
|
||
```
|
||
|
||
---
|
||
|
||
## Self-Review(plan 作者自检)
|
||
|
||
1. **Spec 覆盖**:接口契约→Task 7;透传链路→Task 6;中通 effective offset→Task 2;韵达/顺心/安能 date→target→Task 3/4/5;百世兼容→Task 1;业务日期快照→Task 6 Step 5;周期调度(不动)→无需 task(spec 明示);合法性校验→Task 7;验证→Task 8。✅ 全覆盖。
|
||
2. **占位符**:无 TBD/TODO;每步含完整代码或精确命令。✅
|
||
3. **类型/签名一致**:`date=None` 贯穿 server→runtime→sites;actual impl(yunda/shunxin/anneng)原本无 force,本计划只加 `date`,与 runtime 调用 `download_func(pg, force=, date=)` / 安能 `lambda(ctx,force,date)` 一致;shunxin 双账号 `(pages, foreground, force, date)` 与 `_web_handler` 的 list 分支一致。✅
|
||
4. **顺序安全**:Task 1-5 向后兼容(date=None 走 offset);Task 6 启用透传但 server 未传 date(仍 None);Task 7 启用 date。✅
|