Compare commits
9 Commits
a68fd46c51
...
4f0ef69739
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4f0ef69739 | ||
|
|
5e2ef72afd | ||
|
|
fcb75643d0 | ||
|
|
cda764a305 | ||
|
|
e51aab2e0b | ||
|
|
bacccc43ab | ||
|
|
e240d92ad9 | ||
|
|
30e9203fca | ||
|
|
93e48118bf |
682
docs/2026-07-29-指定日期下载接口-plan.md
Normal file
682
docs/2026-07-29-指定日期下载接口-plan.md
Normal file
@@ -0,0 +1,682 @@
|
||||
# 指定日期下载接口(开发者)Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** 给 `POST /tasks` 加可选 `date`(YYYY-MM-DD),支持顺心/中通/韵达/安能指定一个过去日期下载应到/实到数据;不传 date 时行为不变(走 offset)。
|
||||
|
||||
**Architecture:** 复用现有 `force` 透传链路加一个 `date` 字段:`TaskRequest.date` → `task_spec["date"]` → `dispatch_task` → `handler(ctx, force, date)` → 各站 `impl(page, force, date)`。各站把"算 target 日期"的来源从 `today - offset` 改为"有 date 用 date,否则 today - offset"。中通把 date 折算成 effective offset 以复用已验证的跨月翻页。
|
||||
|
||||
**Tech Stack:** Python 3.10、FastAPI、Playwright、SQLite(state_store)、PostgreSQL(store,可选入库)。
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **环境**:所有 Python 用 `D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe`(项目虚拟环境,无需激活)。PYTHONPATH 含 `InboundVerify` 根。
|
||||
- **格式化/自检(每改一个 .py 必做)**:`python -m black <file>`(target py310)+ `python -m py_compile <file>`。Black 若提示 "Python 3.10 cannot parse code formatted for 3.15",加 `--target-version py310`;"left unchanged" 即合格。
|
||||
- **无 pytest**:本项目无单元测试框架(见 `InboundVerify/CLAUDE.md`)。每个任务的"验证"= Black + py_compile;端到端(API 校验、各站实测)集中在 Task 8(需重启服务加载新代码)。
|
||||
- **submodule 工作流**:改动在 `InboundVerify` submodule(`dev` 分支)。每个 Task 末尾在 submodule 内 `git add <file> && git commit`。**push 到 origin/dev + 父仓库 bump** 统一在 Task 8(项目约定不自动 push,等用户确认;但 plan 内 commit 步骤照写)。
|
||||
- **顺序安全**:Task 1-5(站点)只给 impl 加 `date=None` 形参 + date 逻辑,**date 默认 None 时走原 offset 路径,向后兼容**;Task 6(runtime)才把 date 从 task_spec 透传进 impl;Task 7(server)才允许 date 入队。任一 Task 完成后系统均可正常运行。
|
||||
- **合法性边界**(来自 spec):`date` 传了才校验——格式 `YYYY-MM-DD`、`今天-90 ≤ date ≤ 今天`、百世不支持 date。非法返回 HTTP 400。
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| 文件 | 责任 | 本计划改动 |
|
||||
| --- | --- | --- |
|
||||
| `inbound_verify/sites/baishi.py` | 百世下载(固定当天) | 入口加 `date=None` 形参(忽略) |
|
||||
| `inbound_verify/sites/zto.py` | 中通下载(日历格子,跨月) | 入口+impl 加 `date`;date→effective offset 复用跨月 |
|
||||
| `inbound_verify/sites/yunda.py` | 韵达下载(日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||||
| `inbound_verify/sites/shunxin.py` | 顺心下载(双账号,日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||||
| `inbound_verify/sites/anneng.py` | 安能下载(CDP,日期字符串) | expected/actual 入口+impl 加 `date`;date→target |
|
||||
| `inbound_verify/runtime.py` | 任务派发/心跳共享核心 | handler/dispatch 透传 date;`_record_business_date` 用 date |
|
||||
| `inbound_verify/cli/server.py` | FastAPI 服务 | `TaskRequest.date` + 合法性校验 + task_spec 透传 |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: baishi.py — 入口加 date=None 形参(兼容)
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/sites/baishi.py:140`(`baishi_download_undelivered_data`)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `baishi_download_undelivered_data(page, force=False, date=None)` —— 后续 Task 6 的 `_web_handler` 会以 `download_func(pg, force=force, date=date)` 调用它,必须接受 `date` kwarg(百世忽略)。
|
||||
|
||||
- [ ] **Step 1: 改签名**
|
||||
|
||||
把 `def baishi_download_undelivered_data(page, force=False):` 改为:
|
||||
|
||||
```python
|
||||
def baishi_download_undelivered_data(page, force=False, date=None):
|
||||
"""百世:应到未到数据下载(固定当天;date 形参仅为对齐统一透传签名,忽略)。"""
|
||||
```
|
||||
|
||||
(函数体不动;`date` 不使用。)
|
||||
|
||||
- [ ] **Step 2: Black + py_compile**
|
||||
|
||||
```bash
|
||||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/sites/baishi.py"
|
||||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||||
```
|
||||
Expected: black "left unchanged" 或 reformat 后通过;OK。
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/baishi.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(baishi): accept date kwarg (ignored) for unified dispatch signature" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: zto.py — date 折算成 effective offset(复用跨月)
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/sites/zto.py` — `zto_expected_download`、`zto_expected_download_impl`、`zto_actual_download`、`zto_actual_download_impl`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `zto_expected_download(page, force=False, date=None)` / `zto_actual_download(page, force=False, date=None)`,impl 同签名。Task 6 的 `_web_handler` 以 `download_func(pg, force=force, date=date)` 调用。
|
||||
|
||||
- [ ] **Step 1: expected 入口加 date 并透传**
|
||||
|
||||
`def zto_expected_download(page, force=False):` 及其 `with_retry` 改为:
|
||||
|
||||
```python
|
||||
def zto_expected_download(page, force=False, date=None):
|
||||
"""中通:应到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"中通",
|
||||
"应到",
|
||||
lambda: zto_expected_download_impl(page, force=force, date=date),
|
||||
lambda: zto_reset(page),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: expected impl 加 date,date→effective offset**
|
||||
|
||||
`def zto_expected_download_impl(page, force=False):` 改签名加 `date=None`。其内"读取服务端日期偏移"段(`offset = state_store.get_offset("中通")` 与随后的 `print(...偏移...)`)改为:
|
||||
|
||||
```python
|
||||
# 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||||
offset = state_store.get_offset("中通")
|
||||
if date:
|
||||
# 指定日期:折算成相对今天的有效偏移,复用下方 target_time 计算与跨月翻月
|
||||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||||
offset = (datetime.now().date() - target_date).days
|
||||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||||
else:
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
```
|
||||
|
||||
(其后的 `target_time = today_time - offset * 86400000` 与跨月翻月逻辑**不动**——date 经折算后走同一条路径。)
|
||||
|
||||
- [ ] **Step 3: actual 入口加 date 并透传**
|
||||
|
||||
`def zto_actual_download(page, force=False):` 及其 `with_retry` 改为:
|
||||
|
||||
```python
|
||||
def zto_actual_download(page, force=False, date=None):
|
||||
"""中通:实到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"中通",
|
||||
"实到",
|
||||
lambda: zto_actual_download_impl(page, date=date),
|
||||
lambda: zto_reset(page),
|
||||
)
|
||||
```
|
||||
|
||||
> 注:`zto_actual_download_impl` 现签名 `(page)`(无 force,actual 不去重),Step 4 给它加 `date`。
|
||||
|
||||
- [ ] **Step 4: actual impl 加 date,date→effective offset**
|
||||
|
||||
`def zto_actual_download_impl(page):` 改为 `def zto_actual_download_impl(page, date=None):`。其内"# 2. 读取服务端日期偏移"段(`offset = state_store.get_offset("中通", "actual")` 与随后的 `print`)改为:
|
||||
|
||||
```python
|
||||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||||
offset = state_store.get_offset("中通", "actual")
|
||||
if date:
|
||||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||||
offset = (datetime.now().date() - target_date).days
|
||||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||||
else:
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
```
|
||||
|
||||
(其后 `target_time = today_time - offset * 86400000` 与跨月翻月不动。)
|
||||
|
||||
- [ ] **Step 5: Black + py_compile**
|
||||
|
||||
```bash
|
||||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/sites/zto.py"
|
||||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/zto.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(zto): support date arg via effective-offset (reuses cross-month nav)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: yunda.py — date→target(expected + actual)
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/sites/yunda.py` — `yunda_expected_download(_impl)`、`yunda_actual_download(_impl)`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `yunda_expected_download(page, force=False, date=None)` / `yunda_actual_download(page, force=False, date=None)`,impl 同加 `date=None`。
|
||||
|
||||
- [ ] **Step 1: expected 入口透传 date**
|
||||
|
||||
```python
|
||||
def yunda_expected_download(page, force=False, date=None):
|
||||
...
|
||||
return with_retry(
|
||||
"韵达",
|
||||
"应到",
|
||||
lambda: yunda_expected_download_impl(page, force=force, date=date),
|
||||
lambda: yunda_reset(page),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: expected impl 加 date + date→target**
|
||||
|
||||
`def yunda_expected_download_impl(page, force=False):` → `def yunda_expected_download_impl(page, force=False, date=None):`。其内日期段(`offset = state_store.get_offset("韵达")` 起几行)改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("韵达")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_ymd = f"{target.year}-{target.month}-{target.day}"
|
||||
start_date_ymd = target_ymd
|
||||
today_ymd = target_ymd
|
||||
print(f">> 设置查询日期: [{target_ymd}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||||
```
|
||||
|
||||
- [ ] **Step 3: actual 入口透传 date**
|
||||
|
||||
```python
|
||||
def yunda_actual_download(page, force=False, date=None):
|
||||
...
|
||||
return with_retry(
|
||||
"韵达",
|
||||
"实到",
|
||||
lambda: yunda_actual_download_impl(page, date=date),
|
||||
lambda: yunda_reset(page),
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: actual impl 加 date + date→target**
|
||||
|
||||
`def yunda_actual_download_impl(page):` → `def yunda_actual_download_impl(page, date=None):`。其内日期段改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("韵达", "actual")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
start_date = target # 单日范围:起止同日
|
||||
today = target # 让下方"截止时间"选择器也指向 target
|
||||
print(
|
||||
f">> 设置实到查询日期: [{target.year}-{target.month}-{target.day}]"
|
||||
f"({'指定 ' + date if date else f'偏移 {offset},0=今天'})"
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Black + py_compile**(同 Task 2 命令,文件换 yunda.py)
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/yunda.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(yunda): support date arg (date takes precedence over offset)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: shunxin.py — date→target(双账号透传)
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/sites/shunxin.py` — `shunxin_expected_download(_impl)`、`shunxin_actual_download(_impl)`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `shunxin_expected_download(pages, foreground=True, force=False, date=None)` / `shunxin_actual_download(pages, foreground=True, force=False, date=None)`。Task 6 的 `_web_handler` 以 `download_func(pg, foreground=ctx.foreground, force=force, date=date)` 调用(pg 是 page 列表)。
|
||||
|
||||
- [ ] **Step 1: expected 入口加 date 并向 impl 透传**
|
||||
|
||||
`def shunxin_expected_download(pages, foreground=True, force=False):` → 加 `date=None`。在函数内调用 `shunxin_expected_download_impl(page, out_tag=..., force=force)` 的位置,加上 `date=date`:
|
||||
|
||||
```python
|
||||
def shunxin_expected_download(pages, foreground=True, force=False, date=None):
|
||||
...
|
||||
# 对每个账号调用 impl 时透传 date:
|
||||
... shunxin_expected_download_impl(page, out_tag=归属地, force=force, date=date) ...
|
||||
```
|
||||
|
||||
> 执行者:用 Grep 定位 `shunxin_expected_download_impl(page,` 的调用处(在 `shunxin_expected_download` 函数体内,对每个账号调用一次),给每处加 `date=date`。签名行只加 `date=None`,其余函数体(归属地读取、去重校验、merge)不动。
|
||||
|
||||
- [ ] **Step 2: expected impl 加 date + date→target**
|
||||
|
||||
`def shunxin_expected_download_impl(page, out_tag="", force=False):` → 加 `date=None`。其内日期段(`offset = state_store.get_offset("顺心")` 起几行)改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("顺心")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = target.strftime("%Y-%m-%d")
|
||||
start_date_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 正在设置查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})...")
|
||||
```
|
||||
|
||||
- [ ] **Step 3: actual 入口加 date 并透传**
|
||||
|
||||
`def shunxin_actual_download(pages, foreground=True, force=False):` → 加 `date=None`;调用 `shunxin_actual_download_impl(page, out_tag=...)` 处加 `date=date`。
|
||||
|
||||
- [ ] **Step 4: actual impl 加 date + date→target**
|
||||
|
||||
`def shunxin_actual_download_impl(page, out_tag=""):` → `def shunxin_actual_download_impl(page, out_tag="", date=None):`。其内日期段(`offset = state_store.get_offset("顺心", "actual")` 起几行)改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("顺心", "actual")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = target.strftime("%Y-%m-%d")
|
||||
start_date_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 正在设置查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})...")
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Black + py_compile**(文件 shunxin.py)
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/shunxin.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(shunxin): support date arg, propagate to both accounts" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: anneng.py — date→target(CDP,expected + actual)
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/sites/anneng.py` — `anneng_expected_download(_impl)`、`anneng_actual_download(_impl)`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `anneng_expected_download(force=False, date=None)` / `anneng_actual_download(force=False, date=None)`。Task 6 的 TASK_HANDLERS 安能项以 `lambda ctx, force=False, date=None: anneng.anneng_expected_download(force=force, date=date)` 调用。
|
||||
|
||||
- [ ] **Step 1: expected 入口 + impl 加 date,date→target**
|
||||
|
||||
```python
|
||||
def anneng_expected_download(force=False, date=None):
|
||||
return with_retry(
|
||||
"安能", "应到", lambda: anneng_expected_download_impl(force=force, date=date), anneng_reset
|
||||
)
|
||||
|
||||
|
||||
def anneng_expected_download_impl(force=False, date=None):
|
||||
```
|
||||
|
||||
expected impl 内日期段(`offset = state_store.get_offset("安能")` 起几行)改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("安能")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = f"{target.year}-{target.month:02d}-{target.day:02d}"
|
||||
start_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 查询日期: [{target_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: actual 入口 + impl 加 date,date→target**
|
||||
|
||||
```python
|
||||
def anneng_actual_download(force=False, date=None):
|
||||
return with_retry("安能", "实到", lambda: anneng_actual_download_impl(date=date), anneng_reset)
|
||||
|
||||
|
||||
def anneng_actual_download_impl(date=None):
|
||||
```
|
||||
|
||||
actual impl 内日期段(`offset = state_store.get_offset("安能", "actual")` 起几行)改为:
|
||||
|
||||
```python
|
||||
offset = state_store.get_offset("安能", "actual")
|
||||
today = datetime.now()
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
start_str = f"{target.year}\{target.month:02d}/{target.day:02d} 00:00:00"
|
||||
end_str = f"{target.year}\{target.month:02d}/{target.day:02d} 23:59:59"
|
||||
print(f">> 扫描日期: [{start_str} 至 {end_str}]({'指定 ' + date if date else f'偏移 {offset},0=今天'})")
|
||||
```
|
||||
|
||||
> 注:actual 的 `start_str/end_str` 沿用现有 `{year}\{month}/{day}` 格式(含反斜杠,站点如此),只把 target 来源改成 date。
|
||||
|
||||
- [ ] **Step 3: Black + py_compile**(文件 anneng.py)
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/sites/anneng.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(anneng): support date arg (date takes precedence over offset)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: runtime.py — 全链路透传 date + _record_business_date 用 date
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/runtime.py` — `_web_handler`、`_site_undelivered_handler`、`TASK_HANDLERS` 安能项、`dispatch_task`、`_record_business_date`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 1-5 产出的各站 `*(..., date=None)` 签名。
|
||||
- Produces: `dispatch_task` 从 `task_spec["date"]` 读 date 透传给 handler 与 `_record_business_date`;handler 签名 `(ctx, force=False, date=None)`。
|
||||
|
||||
- [ ] **Step 1: _web_handler 透传 date**
|
||||
|
||||
```python
|
||||
def _web_handler(site, download_func):
|
||||
def handler(ctx, force=False, date=None):
|
||||
pg = ctx.pages_map[site]
|
||||
if isinstance(pg, list):
|
||||
# 顺心双账号:置顶与否交给 shunxin_download 在逐账号循环里按 foreground 决定
|
||||
return download_func(pg, foreground=ctx.foreground, force=force, date=date)
|
||||
if ctx.foreground:
|
||||
pg.bring_to_front()
|
||||
return download_func(pg, force=force, date=date)
|
||||
|
||||
return handler
|
||||
```
|
||||
|
||||
- [ ] **Step 2: _site_undelivered_handler 透传 date(连下 expected+actual 共用同一 date)**
|
||||
|
||||
```python
|
||||
def _site_undelivered_handler(site):
|
||||
def handler(ctx, force=False, date=None):
|
||||
exp_ok = TASK_HANDLERS[(site, "expected")](ctx, force, date) is not False
|
||||
act_ok = (
|
||||
(TASK_HANDLERS[(site, "actual")](ctx, force, date) is not False)
|
||||
if exp_ok
|
||||
else False
|
||||
)
|
||||
if exp_ok and act_ok:
|
||||
return compare.write_site_file(site)
|
||||
stale = os.path.join(DOWNLOAD_DIR, SITE_UNDELIVERED_FILE.format(name=site))
|
||||
if os.path.exists(stale):
|
||||
os.remove(stale)
|
||||
return False
|
||||
|
||||
return handler
|
||||
```
|
||||
|
||||
- [ ] **Step 3: TASK_HANDLERS 安能项透传 date**
|
||||
|
||||
```python
|
||||
("安能", "expected"): lambda ctx, force=False, date=None: anneng.anneng_expected_download(
|
||||
force=force, date=date
|
||||
),
|
||||
("安能", "actual"): lambda ctx, force=False, date=None: anneng.anneng_actual_download(
|
||||
force=force, date=date
|
||||
),
|
||||
```
|
||||
|
||||
(`__compare__` 项与百世/网页项不动——百世经 `_web_handler` 已透传 date,baishi 忽略。)
|
||||
|
||||
- [ ] **Step 4: dispatch_task 透传 date**
|
||||
|
||||
在 `dispatch_task` 内,把 `ret = handler(ctx, bool(task_spec.get("force", False)))` 改为:
|
||||
|
||||
```python
|
||||
ret = handler(
|
||||
ctx, bool(task_spec.get("force", False)), task_spec.get("date")
|
||||
)
|
||||
if ret is False:
|
||||
return (state_store.TASK_FAILED, "任务执行失败(重试耗尽)")
|
||||
_record_business_date(site, kind, task_spec.get("date"))
|
||||
_persist_to_db(site, kind)
|
||||
return (state_store.TASK_SUCCESS, None)
|
||||
```
|
||||
|
||||
- [ ] **Step 5: _record_business_date 接受 date**
|
||||
|
||||
```python
|
||||
def _record_business_date(site, kind, date=None):
|
||||
"""下载成功后,把本次数据的业务日期快照写进状态库(供前端/报告显示「是哪天的数据」)。
|
||||
有 date 用 date;否则 = 下载当天 − 该数据对应的日期偏移。__compare__ 无数据概念,跳过。"""
|
||||
if site == "__compare__":
|
||||
return
|
||||
today = datetime.now().date()
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
def _write(k, biz_or_off):
|
||||
# biz_or_off: int=offset(today−offset);str=已确定的业务日期(date)
|
||||
biz = (
|
||||
(today - timedelta(days=biz_or_off)).strftime("%Y-%m-%d")
|
||||
if isinstance(biz_or_off, int)
|
||||
else biz_or_off
|
||||
)
|
||||
try:
|
||||
state_store.set_data_state(
|
||||
site, k, ready=True, generated_at=now, business_date=biz
|
||||
)
|
||||
except Exception as e:
|
||||
print(f">> [状态] 写业务日期失败 {site}/{k}: {e}")
|
||||
|
||||
def off(kind_key):
|
||||
return state_store.get_offset(site, kind_key)
|
||||
|
||||
if kind == "expected":
|
||||
_write("expected", date if date else off("expected"))
|
||||
elif kind == "actual":
|
||||
_write("actual", date if date else off("actual"))
|
||||
elif site == "百世":
|
||||
_write("undelivered", 0)
|
||||
else: # 4 站 undelivered:连带补写 expected/actual/undelivered 三列
|
||||
_write("expected", date if date else off("expected"))
|
||||
_write("actual", date if date else off("actual"))
|
||||
_write("undelivered", date if date else off("expected"))
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Black + py_compile**
|
||||
|
||||
```bash
|
||||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/runtime.py"
|
||||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||||
```
|
||||
|
||||
- [ ] **Step 7: 冒烟(date=None 行为不变)**
|
||||
|
||||
服务仍跑旧代码,但 runtime 模块可独立 import 校验:
|
||||
|
||||
```bash
|
||||
PYTHONPATH="D:/projects/LogisticsHubIPA/InboundVerify" "$PY" -c "import inbound_verify.runtime as r; import inspect; print('handler', inspect.signature(r._web_handler('中通', lambda *a, **k: None))); print('rec', inspect.signature(r._record_business_date))"
|
||||
```
|
||||
Expected: handler 含 `(ctx, force=False, date=None)`;rec 含 `(site, kind, date=None)`。
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/runtime.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(runtime): propagate date through dispatch chain and business-date snapshot" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: server.py — TaskRequest.date + 合法性校验 + task_spec
|
||||
|
||||
**Files:**
|
||||
- Modify: `inbound_verify/cli/server.py` — 顶部 import、`TaskRequest`、`create_task`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `POST /tasks` 接受 `{site, kind, force, date}`;合法 date 入队为 `task_spec["date"]`(YYYY-MM-DD),非法返回 400。
|
||||
|
||||
- [ ] **Step 1: 顶部 import 加 timedelta**
|
||||
|
||||
`from datetime import datetime` → `from datetime import datetime, timedelta`
|
||||
|
||||
- [ ] **Step 2: TaskRequest 加 date**
|
||||
|
||||
```python
|
||||
class TaskRequest(BaseModel):
|
||||
site: str
|
||||
kind: str
|
||||
force: bool = False
|
||||
date: Optional[str] = None # YYYY-MM-DD;指定则下载该日数据,否则走站点 offset
|
||||
```
|
||||
|
||||
- [ ] **Step 3: create_task 加合法性校验 + task_spec 透传 date**
|
||||
|
||||
```python
|
||||
@app.post("/tasks")
|
||||
def create_task(req: TaskRequest):
|
||||
"""提交任务 {site, kind, force, date?} → 入队,返回 task_id。"""
|
||||
if not worker_state["ready"]:
|
||||
raise HTTPException(
|
||||
status_code=409, detail="后端尚未就绪,请等待各站点登录完成后再操作"
|
||||
)
|
||||
if (req.site, req.kind) not in TASK_HANDLERS:
|
||||
raise HTTPException(status_code=400, detail=f"无效任务: {req.site}/{req.kind}")
|
||||
# 指定日期合法性校验(仅在传了 date 时)
|
||||
if req.date:
|
||||
try:
|
||||
target_date = datetime.strptime(req.date, "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 格式非法,需 YYYY-MM-DD: {req.date}"
|
||||
)
|
||||
today = datetime.now().date()
|
||||
if target_date > today:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 不可为未来日期: {req.date}"
|
||||
)
|
||||
if target_date < today - timedelta(days=90):
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 超出 90 天回溯上限: {req.date}"
|
||||
)
|
||||
if req.site == "百世":
|
||||
raise HTTPException(
|
||||
status_code=400, detail="百世固定下载当天,不支持指定日期"
|
||||
)
|
||||
task_id = state_store.create_task(req.site, req.kind)
|
||||
spec = {"site": req.site, "kind": req.kind, "force": req.force}
|
||||
if req.date:
|
||||
spec["date"] = req.date
|
||||
task_queue.put((task_id, spec))
|
||||
return {"task_id": task_id}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Black + py_compile**
|
||||
|
||||
```bash
|
||||
PY="D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe"
|
||||
F="D:/projects/LogisticsHubIPA/InboundVerify/inbound_verify/cli/server.py"
|
||||
"$PY" -m black "$F" && "$PY" -m py_compile "$F" && echo OK
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" add inbound_verify/cli/server.py
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" commit -m "feat(server): add date field to POST /tasks with legality validation" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: 端到端验证 + 收尾(push / 父 bump)
|
||||
|
||||
**Files:** 无代码改动;验证 + 提交推送。
|
||||
|
||||
- [ ] **Step 1: 重启服务加载全部新代码**
|
||||
|
||||
停掉旧服务进程,重启(中通调试模式,config.yaml 已是中通):
|
||||
|
||||
```bash
|
||||
PYTHONPATH="D:/projects/LogisticsHubIPA/InboundVerify" "D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe" -m inbound_verify.cli.server
|
||||
```
|
||||
(后台运行;等中通登录就绪。)
|
||||
|
||||
- [ ] **Step 2: 合法性校验端到端(API 400/202)**
|
||||
|
||||
用 Python urllib(避免 curl 中文编码问题)逐一验证,期望:
|
||||
|
||||
| 请求 | 期望 |
|
||||
| --- | --- |
|
||||
| `{site:"中通", kind:"expected", date:"2026-06-14"}` | 202 + task_id |
|
||||
| `date:"2099-01-01"`(未来) | 400 "不可为未来日期" |
|
||||
| `date:"2020-01-01"`(超 90 天) | 400 "超出 90 天回溯上限" |
|
||||
| `date:"2026/06/14"`(格式错) | 400 "格式非法" |
|
||||
| `{site:"百世", kind:"undelivered", date:"2026-06-14"}` | 400 "百世…不支持指定日期" |
|
||||
| `{site:"中通", kind:"expected"}`(不传 date) | 202(走 offset,行为不变) |
|
||||
|
||||
```bash
|
||||
"D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe" - <<'PYEOF'
|
||||
import json, urllib.request, urllib.error
|
||||
def post(body):
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
req = urllib.request.Request("http://127.0.0.1:8000/tasks", data=data,
|
||||
headers={"Content-Type":"application/json"}, method="POST")
|
||||
try:
|
||||
print(body, "->", urllib.request.urlopen(req, timeout=10).read().decode())
|
||||
except urllib.error.HTTPError as e:
|
||||
print(body, "->", e.code, e.read().decode())
|
||||
post({"site":"中通","kind":"expected","date":"2099-01-01"})
|
||||
post({"site":"中通","kind":"expected","date":"2020-01-01"})
|
||||
post({"site":"中通","kind":"expected","date":"2026/06/14"})
|
||||
post({"site":"百世","kind":"undelivered","date":"2026-06-14"})
|
||||
PYEOF
|
||||
```
|
||||
|
||||
- [ ] **Step 3: 中通跨月 date 实测(已知 OK)**
|
||||
|
||||
触发 `{site:"中通", kind:"expected", date:"2026-06-14"}`,观察 worker 日志:出现 `指定日期 2026-06-14(折算偏移 …)` + `偏移日期跨月,正在向前翻月导航` + 查询/下载成功。任务 `success`。
|
||||
|
||||
- [ ] **Step 4: 顺心/韵达/安能 date 实测**
|
||||
|
||||
对顺心/韵达/安能各触发一个 expected `date`(取一个近 1 周内的过去日期,确保站点有数据且控件能接受)。观察日志:`指定日期 …` + 正常查询下载。**若某站日期控件不接受字符串而需日历翻月**,记录现象,按中通同法(_zto_flip 模式)追加改动(可能产生新 Task)。
|
||||
|
||||
- [ ] **Step 5: 业务日期快照校验**
|
||||
|
||||
下载后 `curl -s http://127.0.0.1:8000/status`,确认对应站 `expected_business_date` == 指定 date。
|
||||
|
||||
- [ ] **Step 6: push submodule dev + 父仓库 bump**
|
||||
|
||||
```bash
|
||||
git -C "D:/projects/LogisticsHubIPA/InboundVerify" push origin dev
|
||||
git -C "D:/projects/LogisticsHubIPA" add InboundVerify
|
||||
git -C "D:/projects/LogisticsHubIPA" commit -m "chore: bump InboundVerify submodule (date-specific download API)" -m "Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
git -C "D:/projects/LogisticsHubIPA" push origin master
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review(plan 作者自检)
|
||||
|
||||
1. **Spec 覆盖**:接口契约→Task 7;透传链路→Task 6;中通 effective offset→Task 2;韵达/顺心/安能 date→target→Task 3/4/5;百世兼容→Task 1;业务日期快照→Task 6 Step 5;周期调度(不动)→无需 task(spec 明示);合法性校验→Task 7;验证→Task 8。✅ 全覆盖。
|
||||
2. **占位符**:无 TBD/TODO;每步含完整代码或精确命令。✅
|
||||
3. **类型/签名一致**:`date=None` 贯穿 server→runtime→sites;actual impl(yunda/shunxin/anneng)原本无 force,本计划只加 `date`,与 runtime 调用 `download_func(pg, force=, date=)` / 安能 `lambda(ctx,force,date)` 一致;shunxin 双账号 `(pages, foreground, force, date)` 与 `_web_handler` 的 list 分支一致。✅
|
||||
4. **顺序安全**:Task 1-5 向后兼容(date=None 走 offset);Task 6 启用透传但 server 未传 date(仍 None);Task 7 启用 date。✅
|
||||
@@ -17,7 +17,7 @@ import queue
|
||||
import threading
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Dict, Optional
|
||||
|
||||
import uvicorn
|
||||
@@ -190,11 +190,12 @@ class TaskRequest(BaseModel):
|
||||
force: bool = (
|
||||
False # 强制重下:忽略已落库去重,重新提交所有班次/交接单的导出任务(默认关)
|
||||
)
|
||||
date: Optional[str] = None # YYYY-MM-DD;指定则下载该日数据,否则走站点 offset
|
||||
|
||||
|
||||
@app.post("/tasks")
|
||||
def create_task(req: TaskRequest):
|
||||
"""提交任务 {site, kind} → 入队,返回 task_id。"""
|
||||
"""提交任务 {site, kind, force, date?} → 入队,返回 task_id。"""
|
||||
# 【P0】后端未就绪时直接拒绝,避免任务在 worker 启动前入队卡死
|
||||
if not worker_state["ready"]:
|
||||
raise HTTPException(
|
||||
@@ -202,8 +203,32 @@ def create_task(req: TaskRequest):
|
||||
)
|
||||
if (req.site, req.kind) not in TASK_HANDLERS:
|
||||
raise HTTPException(status_code=400, detail=f"无效任务: {req.site}/{req.kind}")
|
||||
# 指定日期合法性校验(仅在传了 date 时)
|
||||
if req.date:
|
||||
try:
|
||||
target_date = datetime.strptime(req.date, "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 格式非法,需 YYYY-MM-DD: {req.date}"
|
||||
)
|
||||
today = datetime.now().date()
|
||||
if target_date > today:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 不可为未来日期: {req.date}"
|
||||
)
|
||||
if target_date < today - timedelta(days=31):
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"date 超出 31 天回溯上限: {req.date}"
|
||||
)
|
||||
if req.site == "百世":
|
||||
raise HTTPException(
|
||||
status_code=400, detail="百世固定下载当天,不支持指定日期"
|
||||
)
|
||||
task_id = state_store.create_task(req.site, req.kind)
|
||||
task_queue.put((task_id, {"site": req.site, "kind": req.kind, "force": req.force}))
|
||||
spec = {"site": req.site, "kind": req.kind, "force": req.force}
|
||||
if req.date:
|
||||
spec["date"] = req.date
|
||||
task_queue.put((task_id, spec))
|
||||
return {"task_id": task_id}
|
||||
|
||||
|
||||
|
||||
@@ -454,14 +454,14 @@ def _web_handler(site, download_func):
|
||||
焦点;交互模式置顶便于调试。顺心是 page 列表,置顶标志透传给 shunxin_download。
|
||||
"""
|
||||
|
||||
def handler(ctx, force=False):
|
||||
def handler(ctx, force=False, date=None):
|
||||
pg = ctx.pages_map[site]
|
||||
if isinstance(pg, list):
|
||||
# 顺心双账号:置顶与否交给 shunxin_download 在逐账号循环里按 foreground 决定
|
||||
return download_func(pg, foreground=ctx.foreground, force=force)
|
||||
return download_func(pg, foreground=ctx.foreground, force=force, date=date)
|
||||
if ctx.foreground:
|
||||
pg.bring_to_front()
|
||||
return download_func(pg, force=force)
|
||||
return download_func(pg, force=force, date=date)
|
||||
|
||||
return handler
|
||||
|
||||
@@ -470,11 +470,11 @@ def _site_undelivered_handler(site):
|
||||
"""4 站未到:下应到+实到 → 比对写 downloads/<站>-未到数据.xlsx。
|
||||
任一下载失败 → 清掉旧未到文件、返回 False(前端不展示陈旧未到)。"""
|
||||
|
||||
def handler(ctx, force=False):
|
||||
def handler(ctx, force=False, date=None):
|
||||
# 各站下载入口约定返回 True/False;顺心历史返回 None(视为成功,与 dispatch 一致)
|
||||
exp_ok = TASK_HANDLERS[(site, "expected")](ctx, force) is not False
|
||||
exp_ok = TASK_HANDLERS[(site, "expected")](ctx, force, date) is not False
|
||||
act_ok = (
|
||||
(TASK_HANDLERS[(site, "actual")](ctx, force) is not False)
|
||||
(TASK_HANDLERS[(site, "actual")](ctx, force, date) is not False)
|
||||
if exp_ok
|
||||
else False
|
||||
)
|
||||
@@ -504,24 +504,32 @@ TASK_HANDLERS = {
|
||||
("韵达", "expected"): _web_handler("韵达", yunda.yunda_expected_download),
|
||||
("韵达", "actual"): _web_handler("韵达", yunda.yunda_actual_download),
|
||||
("韵达", "undelivered"): _site_undelivered_handler("韵达"),
|
||||
("安能", "expected"): lambda ctx, force=False: anneng.anneng_expected_download(
|
||||
force=force
|
||||
(
|
||||
"安能",
|
||||
"expected",
|
||||
): lambda ctx, force=False, date=None: anneng.anneng_expected_download(
|
||||
force=force, date=date
|
||||
),
|
||||
("安能", "actual"): lambda ctx, force=False: anneng.anneng_actual_download(
|
||||
force=force
|
||||
(
|
||||
"安能",
|
||||
"actual",
|
||||
): lambda ctx, force=False, date=None: anneng.anneng_actual_download(
|
||||
force=force, date=date
|
||||
),
|
||||
("安能", "undelivered"): _site_undelivered_handler("安能"),
|
||||
("__compare__", "compare"): lambda ctx, force=False: (compare.main() or True),
|
||||
("__compare__", "compare"): lambda ctx, force=False, date=None: (
|
||||
compare.main() or True
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _record_business_date(site, kind):
|
||||
def _record_business_date(site, kind, date=None):
|
||||
"""下载成功后,把本次数据的业务日期快照写进状态库(供前端/报告显示「是哪天的数据」)。
|
||||
业务日期 = 下载当天 − 该数据对应的日期偏移。__compare__ 无数据概念,跳过。
|
||||
有 date 用 date;否则 = 下载当天 − 该数据对应的日期偏移。__compare__ 无数据概念,跳过。
|
||||
|
||||
kind → 写入:
|
||||
expected/actual:各写自己一列(偏移各取其列)。
|
||||
undelivered:百世直供(恒 0)写 undelivered;4 站未到由 _site_undelivered_handler
|
||||
expected/actual:各写自己一列。
|
||||
undelivered:百世直供(恒当天)写 undelivered;4 站未到由 _site_undelivered_handler
|
||||
内部连带下了 expected+actual(不经 dispatch,无业务日期写入),故此处一并补写
|
||||
expected/actual/undelivered 三列——actual 用 actual 偏移、未到跟随 expected 偏移。
|
||||
顺带置 ready=True,让前端不必等心跳即可反映下载成功;写入失败仅告警、不影响任务判定。"""
|
||||
@@ -530,8 +538,13 @@ def _record_business_date(site, kind):
|
||||
today = datetime.now().date()
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
def _write(k, off):
|
||||
biz = (today - timedelta(days=off)).strftime("%Y-%m-%d")
|
||||
def _write(k, biz_or_off):
|
||||
# biz_or_off: int=偏移(today−off);str=已确定业务日期(date)
|
||||
biz = (
|
||||
(today - timedelta(days=biz_or_off)).strftime("%Y-%m-%d")
|
||||
if isinstance(biz_or_off, int)
|
||||
else biz_or_off
|
||||
)
|
||||
try:
|
||||
state_store.set_data_state(
|
||||
site, k, ready=True, generated_at=now, business_date=biz
|
||||
@@ -539,16 +552,19 @@ def _record_business_date(site, kind):
|
||||
except Exception as e:
|
||||
print(f">> [状态] 写业务日期失败 {site}/{k}: {e}")
|
||||
|
||||
def off(kind_key):
|
||||
return state_store.get_offset(site, kind_key)
|
||||
|
||||
if kind == "expected":
|
||||
_write("expected", state_store.get_offset(site, "expected"))
|
||||
_write("expected", date if date else off("expected"))
|
||||
elif kind == "actual":
|
||||
_write("actual", state_store.get_offset(site, "actual"))
|
||||
_write("actual", date if date else off("actual"))
|
||||
elif site == "百世":
|
||||
_write("undelivered", 0)
|
||||
else: # 4 站 undelivered:连带补写 expected/actual/undelivered 三列
|
||||
_write("expected", state_store.get_offset(site, "expected"))
|
||||
_write("actual", state_store.get_offset(site, "actual"))
|
||||
_write("undelivered", state_store.get_offset(site, "expected"))
|
||||
_write("expected", date if date else off("expected"))
|
||||
_write("actual", date if date else off("actual"))
|
||||
_write("undelivered", date if date else off("expected"))
|
||||
|
||||
|
||||
def _persist_to_db(site, kind):
|
||||
@@ -600,10 +616,10 @@ def dispatch_task(ctx, task_spec):
|
||||
if handler is None:
|
||||
return (state_store.TASK_FAILED, f"未知任务: {site}/{kind}")
|
||||
try:
|
||||
ret = handler(ctx, bool(task_spec.get("force", False)))
|
||||
ret = handler(ctx, bool(task_spec.get("force", False)), task_spec.get("date"))
|
||||
if ret is False:
|
||||
return (state_store.TASK_FAILED, "任务执行失败(重试耗尽)")
|
||||
_record_business_date(site, kind)
|
||||
_record_business_date(site, kind, task_spec.get("date"))
|
||||
_persist_to_db(site, kind)
|
||||
return (state_store.TASK_SUCCESS, None)
|
||||
except Exception as e:
|
||||
|
||||
@@ -855,15 +855,18 @@ def _load_query_days():
|
||||
return max(1, days)
|
||||
|
||||
|
||||
def anneng_expected_download(force=False):
|
||||
def anneng_expected_download(force=False, date=None):
|
||||
"""安能:应到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"安能", "应到", lambda: anneng_expected_download_impl(force=force), anneng_reset
|
||||
"安能",
|
||||
"应到",
|
||||
lambda: anneng_expected_download_impl(force=force, date=date),
|
||||
anneng_reset,
|
||||
)
|
||||
|
||||
|
||||
def anneng_expected_download_impl(force=False):
|
||||
def anneng_expected_download_impl(force=False, date=None):
|
||||
"""安能:应到货物数据(运单信息)下载,完整流程(单次执行,无重试;供自动化测试用)。"""
|
||||
print("\n▶ 开始执行【安能 - 应到货物数据下载】任务 ...")
|
||||
download_dir = DOWNLOAD_DIR
|
||||
@@ -874,11 +877,15 @@ def anneng_expected_download_impl(force=False):
|
||||
|
||||
offset = state_store.get_offset("安能")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = f"{target.year}-{target.month:02d}-{target.day:02d}"
|
||||
start_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 查询日期: [{target_str}](偏移 {offset},0=今天)")
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(f">> 查询日期: [{target_str}]({src})")
|
||||
|
||||
main_cdp = find_main_page_cdp()
|
||||
export_times = []
|
||||
@@ -1258,13 +1265,15 @@ def _save_actual(rows, download_dir):
|
||||
print("====================================================")
|
||||
|
||||
|
||||
def anneng_actual_download(force=False):
|
||||
def anneng_actual_download(force=False, date=None):
|
||||
"""安能:实到数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry("安能", "实到", anneng_actual_download_impl, anneng_reset)
|
||||
return with_retry(
|
||||
"安能", "实到", lambda: anneng_actual_download_impl(date=date), anneng_reset
|
||||
)
|
||||
|
||||
|
||||
def anneng_actual_download_impl():
|
||||
def anneng_actual_download_impl(date=None):
|
||||
"""安能:实到数据(网点到件扫描,子单)下载,完整流程(单次执行,无重试;供自动化测试用)。"""
|
||||
print("\n▶ 开始执行【安能 - 实到货物数据下载】任务 ...")
|
||||
download_dir = DOWNLOAD_DIR
|
||||
@@ -1275,10 +1284,14 @@ def anneng_actual_download_impl():
|
||||
|
||||
offset = state_store.get_offset("安能", "actual")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
start_str = f"{target.year}/{target.month:02d}/{target.day:02d} 00:00:00"
|
||||
end_str = f"{target.year}/{target.month:02d}/{target.day:02d} 23:59:59"
|
||||
print(f">> 扫描日期: [{start_str} 至 {end_str}](偏移 {offset},0=今天)")
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(f">> 扫描日期: [{start_str} 至 {end_str}]({src})")
|
||||
|
||||
main_cdp = find_main_page_cdp()
|
||||
try:
|
||||
|
||||
@@ -137,8 +137,10 @@ def _close_tab(page, tab_name):
|
||||
print(f" ⚠️ 关闭标签页【{tab_name}】时出错: {e}")
|
||||
|
||||
|
||||
def baishi_download_undelivered_data(page, force=False):
|
||||
"""百世:一键提取应到未到(当日未扫)数据(内部含异常兜底重试,路由层无感)。"""
|
||||
def baishi_download_undelivered_data(page, force=False, date=None):
|
||||
"""百世:一键提取应到未到(当日未扫)数据(内部含异常兜底重试,路由层无感)。
|
||||
|
||||
date 形参仅为对齐统一透传签名(百世固定下载当天),忽略。"""
|
||||
|
||||
return with_retry(
|
||||
"百世",
|
||||
|
||||
@@ -149,7 +149,7 @@ def shunxin_merge_final(kind, tags):
|
||||
pass
|
||||
|
||||
|
||||
def shunxin_expected_download(pages, foreground=True, force=False):
|
||||
def shunxin_expected_download(pages, foreground=True, force=False, date=None):
|
||||
"""顺心:应到货物数据下载(双账号/双归属地,内部含异常兜底重试与数据融合)。
|
||||
|
||||
pages 为该站点的 page 列表(双账号在同一窗口的各一个标签页)。
|
||||
@@ -175,8 +175,8 @@ def shunxin_expected_download(pages, foreground=True, force=False):
|
||||
ok = with_retry(
|
||||
f"顺心-{tag}",
|
||||
"应到",
|
||||
lambda p=pg, t=tag, f=force: shunxin_expected_download_impl(
|
||||
p, out_tag=t, force=f
|
||||
lambda p=pg, t=tag, f=force, d=date: shunxin_expected_download_impl(
|
||||
p, out_tag=t, force=f, date=d
|
||||
),
|
||||
lambda p=pg: shunxin_reset(p),
|
||||
)
|
||||
@@ -187,7 +187,7 @@ def shunxin_expected_download(pages, foreground=True, force=False):
|
||||
return True
|
||||
|
||||
|
||||
def shunxin_expected_download_impl(page, out_tag="", force=False):
|
||||
def shunxin_expected_download_impl(page, out_tag="", force=False, date=None):
|
||||
"""顺心:应到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。
|
||||
|
||||
out_tag 为归属地标签时,合并产物命名为「顺心-{out_tag}-应到货物数据.xlsx」,
|
||||
@@ -224,11 +224,15 @@ def shunxin_expected_download_impl(page, out_tag="", force=False):
|
||||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日范围:起止同日
|
||||
offset = state_store.get_offset("顺心")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = target.strftime("%Y-%m-%d")
|
||||
start_date_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 正在设置查询日期: [{target_str}](偏移 {offset},0=今天)...")
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(f">> 正在设置查询日期: [{target_str}]({src})...")
|
||||
|
||||
# 分两步精准呼出和点击时间控件
|
||||
print(" >> 设置起始时间...")
|
||||
@@ -516,7 +520,7 @@ def shunxin_expected_download_impl(page, out_tag="", force=False):
|
||||
return False
|
||||
|
||||
|
||||
def shunxin_actual_download(pages, foreground=True, force=False):
|
||||
def shunxin_actual_download(pages, foreground=True, force=False, date=None):
|
||||
"""顺心:实到货物数据下载(双账号/双归属地,内部含异常兜底重试与数据融合)。
|
||||
|
||||
与 shunxin_expected_download 同构:读归属地 → 去重校验 → 顺序各账号下载 →
|
||||
@@ -540,7 +544,9 @@ def shunxin_actual_download(pages, foreground=True, force=False):
|
||||
ok = with_retry(
|
||||
f"顺心-{tag}",
|
||||
"实到",
|
||||
lambda p=pg, t=tag: shunxin_actual_download_impl(p, out_tag=t),
|
||||
lambda p=pg, t=tag, d=date: shunxin_actual_download_impl(
|
||||
p, out_tag=t, date=d
|
||||
),
|
||||
lambda p=pg: shunxin_reset(p),
|
||||
)
|
||||
if not ok:
|
||||
@@ -550,7 +556,7 @@ def shunxin_actual_download(pages, foreground=True, force=False):
|
||||
return True
|
||||
|
||||
|
||||
def shunxin_actual_download_impl(page, out_tag=""):
|
||||
def shunxin_actual_download_impl(page, out_tag="", date=None):
|
||||
"""顺心:实到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。
|
||||
|
||||
out_tag 为归属地标签时,合并产物命名为「顺心-{out_tag}-实到货物数据.xlsx」,
|
||||
@@ -583,11 +589,15 @@ def shunxin_actual_download_impl(page, out_tag=""):
|
||||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日范围:起止同日
|
||||
offset = state_store.get_offset("顺心", "actual")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_str = target.strftime("%Y-%m-%d")
|
||||
start_date_str = target_str
|
||||
today_str = target_str
|
||||
print(f">> 正在设置查询日期: [{target_str}](偏移 {offset},0=今天)...")
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(f">> 正在设置查询日期: [{target_str}]({src})...")
|
||||
|
||||
# 分两步精准呼出和点击时间控件
|
||||
print(" >> 设置起始时间...")
|
||||
|
||||
@@ -169,18 +169,18 @@ def yunda_smart_menu_click(page, menu_path):
|
||||
page.wait_for_timeout(1000)
|
||||
|
||||
|
||||
def yunda_expected_download(page, force=False):
|
||||
def yunda_expected_download(page, force=False, date=None):
|
||||
"""韵达:应到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"韵达",
|
||||
"应到",
|
||||
lambda: yunda_expected_download_impl(page, force=force),
|
||||
lambda: yunda_expected_download_impl(page, force=force, date=date),
|
||||
lambda: yunda_reset(page),
|
||||
)
|
||||
|
||||
|
||||
def yunda_expected_download_impl(page, force=False):
|
||||
def yunda_expected_download_impl(page, force=False, date=None):
|
||||
"""韵达:应到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。"""
|
||||
print("\n▶ 开始执行【韵达 - 应到货物数据下载】任务...")
|
||||
|
||||
@@ -211,11 +211,15 @@ def yunda_expected_download_impl(page, force=False):
|
||||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日范围:起止同日
|
||||
offset = state_store.get_offset("韵达")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
target_ymd = f"{target.year}-{target.month}-{target.day}"
|
||||
start_date_ymd = target_ymd
|
||||
today_ymd = target_ymd
|
||||
print(f">> 设置查询日期: [{target_ymd}](偏移 {offset},0=今天)")
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(f">> 设置查询日期: [{target_ymd}]({src})")
|
||||
|
||||
# 设定起始时间
|
||||
print(" >> 设置起始时间...")
|
||||
@@ -422,18 +426,18 @@ def yunda_expected_download_impl(page, force=False):
|
||||
return False
|
||||
|
||||
|
||||
def yunda_actual_download(page, force=False):
|
||||
def yunda_actual_download(page, force=False, date=None):
|
||||
"""韵达:实到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"韵达",
|
||||
"实到",
|
||||
lambda: yunda_actual_download_impl(page),
|
||||
lambda: yunda_actual_download_impl(page, date=date),
|
||||
lambda: yunda_reset(page),
|
||||
)
|
||||
|
||||
|
||||
def yunda_actual_download_impl(page):
|
||||
def yunda_actual_download_impl(page, date=None):
|
||||
"""韵达:实到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。"""
|
||||
print("\n▶ 开始执行【韵达 - 实到货物数据下载】任务...")
|
||||
|
||||
@@ -464,12 +468,15 @@ def yunda_actual_download_impl(page):
|
||||
|
||||
offset = state_store.get_offset("韵达", "actual")
|
||||
today = datetime.now()
|
||||
target = today - timedelta(days=offset)
|
||||
if date:
|
||||
target = datetime.strptime(date, "%Y-%m-%d")
|
||||
else:
|
||||
target = today - timedelta(days=offset)
|
||||
start_date = target # 单日范围:起止同日
|
||||
today = target # 让下方"截止时间"选择器也指向 target
|
||||
src = f"指定 {date}" if date else f"偏移 {offset},0=今天"
|
||||
print(
|
||||
f">> 设置实到查询日期: [{target.year}-{target.month}-{target.day}]"
|
||||
f"(偏移 {offset},0=今天)"
|
||||
f">> 设置实到查询日期: [{target.year}-{target.month}-{target.day}]({src})"
|
||||
)
|
||||
|
||||
print(" >> 正在设定起始时间...")
|
||||
|
||||
@@ -125,18 +125,18 @@ def zto_smart_menu_click(page, menu_path):
|
||||
page.wait_for_timeout(1000)
|
||||
|
||||
|
||||
def zto_expected_download(page, force=False):
|
||||
def zto_expected_download(page, force=False, date=None):
|
||||
"""中通:应到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"中通",
|
||||
"应到",
|
||||
lambda: zto_expected_download_impl(page, force=force),
|
||||
lambda: zto_expected_download_impl(page, force=force, date=date),
|
||||
lambda: zto_reset(page),
|
||||
)
|
||||
|
||||
|
||||
def zto_expected_download_impl(page, force=False):
|
||||
def zto_expected_download_impl(page, force=False, date=None):
|
||||
"""中通:应到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。"""
|
||||
print("\n▶ 开始执行【中通 - 应到货物数据下载】任务...")
|
||||
|
||||
@@ -160,8 +160,13 @@ def zto_expected_download_impl(page, force=False):
|
||||
|
||||
# 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||||
offset = state_store.get_offset("中通")
|
||||
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
if date:
|
||||
# 指定日期:折算成相对今天的有效偏移,复用下方 target_time 计算与跨月翻月
|
||||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||||
offset = (datetime.now().date() - target_date).days
|
||||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||||
else:
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
ewb_frame.locator("#beginDate").click()
|
||||
page.wait_for_timeout(500)
|
||||
|
||||
@@ -373,18 +378,18 @@ def zto_expected_download_impl(page, force=False):
|
||||
return False
|
||||
|
||||
|
||||
def zto_actual_download(page, force=False):
|
||||
def zto_actual_download(page, force=False, date=None):
|
||||
"""中通:实到货物数据下载(内部含异常兜底重试,路由层无感)。"""
|
||||
|
||||
return with_retry(
|
||||
"中通",
|
||||
"实到",
|
||||
lambda: zto_actual_download_impl(page),
|
||||
lambda: zto_actual_download_impl(page, date=date),
|
||||
lambda: zto_reset(page),
|
||||
)
|
||||
|
||||
|
||||
def zto_actual_download_impl(page):
|
||||
def zto_actual_download_impl(page, date=None):
|
||||
"""中通:实到货物数据下载(单次执行,无重试;供自动化测试探测原始结果用)。"""
|
||||
print("\n▶ 开始执行【中通 - 实到货物数据下载】任务...")
|
||||
|
||||
@@ -408,8 +413,12 @@ def zto_actual_download_impl(page):
|
||||
|
||||
# 2. 读取服务端日期偏移(0=今天,1=昨天…),单日:起止同日
|
||||
offset = state_store.get_offset("中通", "actual")
|
||||
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
if date:
|
||||
target_date = datetime.strptime(date, "%Y-%m-%d").date()
|
||||
offset = (datetime.now().date() - target_date).days
|
||||
print(f">> 正在设定查询日期: 指定日期 {date}(折算偏移 {offset})...")
|
||||
else:
|
||||
print(f">> 正在设定查询日期: 偏移 {offset}(0=今天)...")
|
||||
arr_frame.locator("#daterange").click()
|
||||
page.wait_for_timeout(500)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user