Compare commits
10 Commits
b9f726c30a
...
79c84a5a0c
| Author | SHA1 | Date | |
|---|---|---|---|
|
79c84a5a0c | ||
|
|
62c44b262a | ||
|
|
23042c272b | ||
|
|
91ef8970b8 | ||
|
|
bd0df8909f | ||
|
|
17293bee79 | ||
|
|
5ac618c041 | ||
|
|
a07b9b435b | ||
|
|
7065b88269 | ||
|
|
0174e64a04 |
65
CLAUDE.md
65
CLAUDE.md
@@ -8,73 +8,82 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||||||
异常运单并汇总成 Excel。4 个网页站点 + 1 个 Electron 应用（安能）。
|
异常运单并汇总成 Excel。4 个网页站点 + 1 个 Electron 应用（安能）。
|
||||||
|
|
||||||
**两种运行模式**（阶段1 起）：
|
**两种运行模式**（阶段1 起）：
|
||||||
- **交互模式** `main_router.py`：人工调试 / 操作，交互菜单（登录、触发下载、[12] 状态盘）。
|
- **交互模式** `inbound_verify.cli.router`：人工调试 / 操作，交互菜单（登录、触发下载、[12] 状态盘）。
|
||||||
- **服务模式** `server.py`：常驻 + FastAPI，客户端经 HTTP 触发任务、查状态、下载数据（API 文档 `/docs`）。
|
- **服务模式** `inbound_verify.cli.server`：常驻 + FastAPI，客户端经 HTTP 触发任务、查状态、下载数据（API 文档 `/docs`）。
|
||||||
- 两者共享 `runtime.py`（启动 / 就绪 / 任务派发 / 心跳）与 `state_store.py`（SQLite 状态持久化）。
|
- 两者共享 `inbound_verify.runtime`（启动 / 就绪 / 任务派发 / 心跳）与 `inbound_verify.state_store`（SQLite 状态持久化）。
|
||||||
|
|
||||||
## 常用命令
|
## 常用命令
|
||||||
|
|
||||||
所有 Python 一律在项目虚拟环境 `.venv` 中运行（Windows 下可直接用
|
所有 Python 一律在项目虚拟环境 `.venv` 中运行（Windows 下可直接用
|
||||||
`.venv/Scripts/python.exe`，无需激活）。
|
`.venv/Scripts/python.exe`，无需激活）。首次 / 拉取新代码后需
|
||||||
|
`.venv/Scripts/python.exe -m pip install -e .`（以可编辑模式注册
|
||||||
|
`inbound-verify` 等命令）。
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 安装依赖（含安能 CDP 驱动所需的 websocket-client）
|
# 安装依赖（含安能 CDP 驱动所需的 websocket-client）+ 以可编辑模式注册命令
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
|
pip install -e .
|
||||||
playwright install chromium
|
playwright install chromium
|
||||||
|
|
||||||
# 运行主程序（交互式菜单，详见 main_router 的 run_multi_site_daemon）
|
# 运行主程序（交互式菜单，详见 inbound_verify.cli.router 的 run_multi_site_daemon）
|
||||||
.venv/Scripts/python.exe main_router.py
|
.venv/Scripts/python.exe -m inbound_verify.cli.router
|
||||||
|
# 装包后也可直接用命令：inbound-verify
|
||||||
|
|
||||||
# 服务模式（常驻 + FastAPI，客户端经 HTTP 触发；默认 :8000，API 文档见 /docs）
|
# 服务模式（常驻 + FastAPI，客户端经 HTTP 触发；默认 :8000，API 文档见 /docs）
|
||||||
.venv/Scripts/python.exe server.py
|
.venv/Scripts/python.exe -m inbound_verify.cli.server
|
||||||
|
# 或：inbound-verify-server
|
||||||
|
|
||||||
|
# DB CLI：建库 / 初始化 / 灌数据 / 全流程
|
||||||
|
.venv/Scripts/python.exe -m inbound_verify.store createdb # 或 init | ingest | all
|
||||||
|
# 或：inbound-verify-db createdb|init|ingest|all
|
||||||
|
|
||||||
# 单站点联调：在 config.yaml 设 debug.enabled=true + debug.target_site=顺心|百世|中通|韵达|安能
|
# 单站点联调：在 config.yaml 设 debug.enabled=true + debug.target_site=顺心|百世|中通|韵达|安能
|
||||||
# 网页站：只挂载该站；安能：只启动 Electron 应用。
|
# 网页站：只挂载该站；安能：只启动 Electron 应用。
|
||||||
|
|
||||||
# 安能独立运行（需先以 --remote-debugging-port=9222 启动「安能全网门户.exe」并手动登录）
|
# 安能独立运行（需先以 --remote-debugging-port=9222 启动「安能全网门户.exe」并手动登录）
|
||||||
.venv/Scripts/python.exe site_anneng.py expected # 或 actual
|
.venv/Scripts/python.exe -m inbound_verify.sites.anneng expected # 或 actual
|
||||||
|
|
||||||
# 格式化（全局规范：改完 Python 必须 Black）
|
# 格式化（全局规范：改完 Python 必须 Black）
|
||||||
.venv/Scripts/python.exe -m black <file.py>
|
.venv/Scripts/python.exe -m black inbound_verify
|
||||||
|
|
||||||
# 语法自检
|
# 语法自检
|
||||||
.venv/Scripts/python.exe -m py_compile <file.py>
|
.venv/Scripts/python.exe -m py_compile inbound_verify
|
||||||
```
|
```
|
||||||
|
|
||||||
**没有 pytest 测试套件。** "测试"指 `main_router` 菜单 **[8] 自动化测试**
|
**没有 pytest 测试套件。** "测试"指 `inbound_verify.cli.router` 菜单 **[8] 自动化测试**
|
||||||
（`run_automation_test`，按 `CROSS_TEST_SEQUENCE` 交叉跑通各站点流程）。
|
（`run_automation_test`，按 `CROSS_TEST_SEQUENCE` 交叉跑通各站点流程）。
|
||||||
|
|
||||||
## 架构（big picture）
|
## 架构（big picture）
|
||||||
|
|
||||||
### 运行模式与共享核心（阶段0/1 重构）
|
### 运行模式与共享核心（阶段0/1 重构）
|
||||||
- **`runtime.py`**：两种模式共享的核心——`launch_and_prepare`（启动 Playwright + 各站就绪 + 弹窗 + 心跳初值，阻塞至就绪）、`dispatch_task(ctx, {site,kind})`（派发任务，掉登录直接判 failed）、`run_heartbeat`、`probe_site_login/probe_data_file`、`RuntimeContext.stop()`。常量 `SITES_CONFIG/READY_SELECTORS/APP_SITES/HEARTBEAT_INTERVAL/DATA_FILENAMES` 在此。
|
- **`inbound_verify.runtime`**：两种模式共享的核心——`launch_and_prepare`（启动 Playwright + 各站就绪 + 弹窗 + 心跳初值，阻塞至就绪）、`dispatch_task(ctx, {site,kind})`（派发任务，掉登录直接判 failed）、`run_heartbeat`、`probe_site_login/probe_data_file`、`RuntimeContext.stop()`。常量 `SITES_CONFIG/READY_SELECTORS/APP_SITES/HEARTBEAT_INTERVAL/DATA_FILENAMES` 在此。
|
||||||
- **`state_store.py`**：SQLite 状态持久化（`state/state.db`）。`site_status`（登录态 + 数据态 + 时间戳，心跳刷新）、`task_history`（任务记录）。重启不丢。
|
- **`inbound_verify.state_store`**：SQLite 状态持久化（`state/state.db`）。`site_status`（登录态 + 数据态 + 时间戳，心跳刷新）、`task_history`（任务记录）。重启不丢。
|
||||||
- **`main_router.py`**：交互模式，菜单循环（input 后台线程 + `_await_command` + `dispatch_task` + 心跳）。
|
- **`inbound_verify.cli.router`**：交互模式，菜单循环（input 后台线程 + `_await_command` + `dispatch_task` + 心跳）。
|
||||||
- **`server.py`**：服务模式。**FastAPI（主线程）+ Playwright worker（独立线程）**——主线程处理 HTTP（绝不碰 Playwright），worker 独占 page 操作，经 `task_queue` + `state_store` 通信。API：`POST/GET /tasks`、`GET /status`、`GET /data/{file}`。
|
- **`inbound_verify.cli.server`**：服务模式。**FastAPI（主线程）+ Playwright worker（独立线程）**——主线程处理 HTTP（绝不碰 Playwright），worker 独占 page 操作，经 `task_queue` + `state_store` 通信。API：`POST/GET /tasks`、`GET /status`、`GET /data/{file}`。
|
||||||
- **关键线程约束**：Playwright sync 对象绑定创建它的线程；`launch_and_prepare`（含 `sync_playwright().start()`）必须在持有 Playwright 的线程调用（交互=主线程，服务=worker 线程）。FastAPI 路由绝不访问 page。
|
- **关键线程约束**：Playwright sync 对象绑定创建它的线程；`launch_and_prepare`（含 `sync_playwright().start()`）必须在持有 Playwright 的线程调用（交互=主线程，服务=worker 线程）。FastAPI 路由绝不访问 page。
|
||||||
- 站点模块 `site_*.py` 的 `with_retry` 返回 `True/False`（成功 / 放弃），供 `dispatch_task` 判成败。
|
- 站点模块 `inbound_verify.sites.*` 的 `with_retry` 返回 `True/False`（成功 / 放弃），供 `dispatch_task` 判成败。
|
||||||
|
|
||||||
### 两套驱动模态 —— 这是理解全局的关键
|
### 两套驱动模态 —— 这是理解全局的关键
|
||||||
- **网页 4 站**（顺心/百世/中通/韵达）：`main_router` 用 Playwright 开 chromium，
|
- **网页 4 站**（顺心/百世/中通/韵达）：`inbound_verify.cli.router` 用 Playwright 开 chromium，
|
||||||
每站一个 `page`，流程函数签名为 `xxx_download_impl(page)`。**例外：顺心是双账号**
|
每站一个 `page`，流程函数签名为 `xxx_download_impl(page)`。**例外：顺心是双账号**
|
||||||
——同一窗口开两个标签页（两个归属地账号），`pages_map["顺心"]` 存为 page **列表**，
|
——同一窗口开两个标签页（两个归属地账号），`pages_map["顺心"]` 存为 page **列表**，
|
||||||
`shunxin_download(pages)` 接收列表（详见下文「顺心双账号」）。
|
`shunxin_download(pages)` 接收列表（详见下文「顺心双账号」）。
|
||||||
- **安能**：Electron 桌面应用，**不走 Playwright**。`main_router` 以
|
- **安能**：Electron 桌面应用，**不走 Playwright**。`inbound_verify.cli.router` 以
|
||||||
`--remote-debugging-port=<动态空闲端口>` 启动 exe（`launch_anneng`），
|
`--remote-debugging-port=<动态空闲端口>` 启动 exe（`launch_anneng`），
|
||||||
通过 `site_anneng.set_cdp_port` 告知模块；`site_anneng.py` 用裸 CDP（websocket）
|
通过 `inbound_verify.sites.anneng.set_cdp_port` 告知模块；`inbound_verify.sites.anneng` 用裸 CDP（websocket）
|
||||||
驱动，业务 tab 是独立 webContents。这也是 `playwright-cli` 接管不了安能的原因
|
驱动，业务 tab 是独立 webContents。这也是 `playwright-cli` 接管不了安能的原因
|
||||||
（Electron 19 / Chrome 102 不支持 Playwright 要的 setDownloadBehavior）。
|
（Electron 19 / Chrome 102 不支持 Playwright 要的 setDownloadBehavior）。
|
||||||
|
|
||||||
### 分层：路由纯调度，站点模块自洽
|
### 分层：路由纯调度，站点模块自洽
|
||||||
- `main_router.py` **只调度**：启动浏览器/安能、就绪轮询、登录检测、菜单分发。
|
- `inbound_verify.cli.router` **只调度**：启动浏览器/安能、就绪轮询、登录检测、菜单分发。
|
||||||
菜单项直接调 `site_xxx.xxx_download(page)`（顺心传 page 列表），**不关心**重试/重置。
|
菜单项直接调 `sites.xxx_download(page)`（顺心传 page 列表），**不关心**重试/重置。
|
||||||
- 每个 `site_xxx.py` 对外只暴露"把任务做了"的入口，内部自洽：
|
- 每个 `inbound_verify.sites.*` 模块对外只暴露"把任务做了"的入口，内部自洽：
|
||||||
- `xxx_download(...)` —— 公开入口，= `with_retry(站点, 标签, xxx_download_impl, xxx_reset)`
|
- `xxx_download(...)` —— 公开入口，= `with_retry(站点, 标签, xxx_download_impl, xxx_reset)`
|
||||||
- `xxx_download_impl(...)` —— 单次执行、**无重试**（自动化测试刻意调它以探测原始失败）
|
- `xxx_download_impl(...)` —— 单次执行、**无重试**（自动化测试刻意调它以探测原始失败）
|
||||||
- `xxx_reset(...)` —— 重置回初始态（网页 = `page.goto(HOME_URL)`；安能 = 关业务 tab + 收菜单）
|
- `xxx_reset(...)` —— 重置回初始态（网页 = `page.goto(HOME_URL)`；安能 = 关业务 tab + 收菜单）
|
||||||
- `with_retry(...)` —— 重试逻辑**内联在每个站点模块**（不抽公共组件，现阶段刻意不优化结构）；
|
- `with_retry(...)` —— 重试逻辑**内联在每个站点模块**（不抽公共组件，现阶段刻意不优化结构）；
|
||||||
失败→重置→重试，最多 3 次（含首次），每次失败都重置（含最终放弃那次清场）
|
失败→重置→重试，最多 3 次（含首次），每次失败都重置（含最终放弃那次清场）
|
||||||
- `HOME_URL` —— 站点首页 URL，`main_router.SITES_CONFIG` 引用它（单一来源）
|
- `HOME_URL` —— 站点首页 URL，`runtime.SITES_CONFIG` 引用它（单一来源）
|
||||||
|
|
||||||
### 导出任务队列模式（顺心/中通/韵达/安能-应到 共用）
|
### 导出任务队列模式（顺心/中通/韵达/安能-应到 共用）
|
||||||
这些站的下载是异步的：提交导出（记 `export_times` 时间戳）→ 跳"导出任务管理"页轮询 →
|
这些站的下载是异步的：提交导出（记 `export_times` 时间戳）→ 跳"导出任务管理"页轮询 →
|
||||||
@@ -88,7 +97,7 @@ playwright install chromium
|
|||||||
### 顺心双账号（双归属地）
|
### 顺心双账号（双归属地）
|
||||||
顺心业务上要同时处理**两个归属地网点**（两个账号）。程序在同一窗口开两个标签页，
|
顺心业务上要同时处理**两个归属地网点**（两个账号）。程序在同一窗口开两个标签页，
|
||||||
人工分别登录两个账号（顺心站点支持同浏览器双账号并存，无需独立 context/窗口）。
|
人工分别登录两个账号（顺心站点支持同浏览器双账号并存，无需独立 context/窗口）。
|
||||||
- `main_router` 启动时为顺心开 2 个 `context.new_page()`，`pages_map["顺心"]` 为列表；
|
- `runtime` 启动时为顺心开 2 个 `context.new_page()`，`pages_map["顺心"]` 为列表；
|
||||||
就绪轮询要求**两个标签页都进主页**才算就绪；初始弹窗对两个标签页各处理一遍。
|
就绪轮询要求**两个标签页都进主页**才算就绪；初始弹窗对两个标签页各处理一遍。
|
||||||
- `shunxin_expected_download(pages)` / `shunxin_actual_download(pages)` 接收 page 列表：
|
- `shunxin_expected_download(pages)` / `shunxin_actual_download(pages)` 接收 page 列表：
|
||||||
先用 `shunxin_belonging(page)` 读各账号归属地（首页「切换网点」控件 `.site___3o7nH`），
|
先用 `shunxin_belonging(page)` 读各账号归属地（首页「切换网点」控件 `.site___3o7nH`），
|
||||||
@@ -100,16 +109,16 @@ playwright install chromium
|
|||||||
配合每账号独立 `export_times` + ≤40s 容差，B 不会误匹配 A 的任务。
|
配合每账号独立 `export_times` + ≤40s 容差，B 不会误匹配 A 的任务。
|
||||||
|
|
||||||
### 比对
|
### 比对
|
||||||
`expected_undelivered.py`（菜单 [9]）纯离线：读 `downloads/` 下各站应到/实到 xlsx，
|
`inbound_verify.expected_undelivered`（菜单 [9]）纯离线：读 `downloads/` 下各站应到/实到 xlsx，
|
||||||
比对生成 `output/应到未到数据.xlsx`（汇总 + 各站明细）。
|
比对生成 `output/应到未到数据.xlsx`（汇总 + 各站明细）。
|
||||||
|
|
||||||
### 路径
|
### 路径
|
||||||
`paths.py` 把 `DOWNLOAD_DIR` / `CONFIG_PATH` / `BASE_DIR` 全部锚定到项目目录，
|
`inbound_verify.paths` 把 `DOWNLOAD_DIR` / `CONFIG_PATH` / `BASE_DIR` 全部锚定到项目目录，
|
||||||
**不依赖运行时 cwd**——别用相对路径或 `os.getcwd()`。
|
**不依赖运行时 cwd**——别用相对路径或 `os.getcwd()`。
|
||||||
|
|
||||||
## 重要约定 / 易踩坑
|
## 重要约定 / 易踩坑
|
||||||
|
|
||||||
- **登录是手动的**：`main_router` 启动后会停在就绪轮询（`READY_SELECTORS` / `anneng_ready`），
|
- **登录是手动的**：`inbound_verify.cli.router` 启动后会停在就绪轮询（`READY_SELECTORS` / `anneng_ready`），
|
||||||
直到检测到所有站点进入工作台才进菜单。仅韵达支持凭 `config.yaml` 凭据自动登录。
|
直到检测到所有站点进入工作台才进菜单。仅韵达支持凭 `config.yaml` 凭据自动登录。
|
||||||
**顺心需登录两个账号**：同一窗口的两个标签页分别登录两个不同归属地账号，两个标签页
|
**顺心需登录两个账号**：同一窗口的两个标签页分别登录两个不同归属地账号，两个标签页
|
||||||
都进主页后才算就绪（顺心站点支持同浏览器双账号并存，故用同 context 标签页而非独立窗口）。
|
都进主页后才算就绪（顺心站点支持同浏览器双账号并存，故用同 context 标签页而非独立窗口）。
|
||||||
|
|||||||
48
README.md
48
README.md
@@ -32,19 +32,29 @@
|
|||||||
|
|
||||||
```
|
```
|
||||||
InboundVerify/
|
InboundVerify/
|
||||||
├── main_router.py # 主入口 / 调度层（菜单、启动浏览器与安能、就绪轮询、登录检测）
|
├── pyproject.toml # 打包 + 依赖 + console_scripts（inbound-verify 等）
|
||||||
├── site_shunxin.py # 顺心站点模块（流程 + 重置 + 重试，自洽）
|
├── inbound_verify/ # 源码包
|
||||||
├── site_baishi.py # 百世站点模块
|
│ ├── paths.py # 统一路径锚点（以项目目录为基准，不依赖 cwd）
|
||||||
├── site_zto.py # 中通站点模块
|
│ ├── runtime.py # 两种模式共享核心（启动 / 就绪 / 任务派发 / 心跳）
|
||||||
├── site_yunda.py # 韵达站点模块（含自动登录）
|
│ ├── state_store.py # SQLite 状态持久化（state/state.db）
|
||||||
├── site_anneng.py # 安能站点模块（Electron + CDP 驱动）
|
│ ├── expected_undelivered.py# 全站点应到未到离线比对，输出 output/应到未到数据.xlsx
|
||||||
├── expected_undelivered.py # 全站点应到未到离线比对，输出 output/应到未到数据.xlsx
|
│ ├── store.py # DB CLI 入口（createdb|init|ingest|all）
|
||||||
├── paths.py # 统一路径锚点（以本目录为基准，不依赖 cwd）
|
│ ├── sites/ # 各站点模块（流程 + 重置 + 重试，自洽）
|
||||||
|
│ │ ├── shunxin.py # 顺心（含双账号）
|
||||||
|
│ │ ├── baishi.py # 百世
|
||||||
|
│ │ ├── zto.py # 中通
|
||||||
|
│ │ ├── yunda.py # 韵达（含自动登录）
|
||||||
|
│ │ └── anneng.py # 安能（Electron + CDP 驱动）
|
||||||
|
│ └── cli/ # 命令行入口
|
||||||
|
│ ├── router.py # 交互菜单（调度层：启动 / 就绪轮询 / 登录检测 / 菜单分发）
|
||||||
|
│ └── server.py # FastAPI 服务模式（常驻 + HTTP 触发）
|
||||||
├── config.example.yaml # 配置模板
|
├── config.example.yaml # 配置模板
|
||||||
├── config.yaml # 真实配置（自行创建，已被 .gitignore 忽略）
|
├── config.yaml # 真实配置（自行创建，已被 .gitignore 忽略）
|
||||||
├── requirements.txt
|
├── schema.sql # 数据库表结构（store.py createdb / init 使用）
|
||||||
|
├── requirements.txt # pyproject 依赖的静态镜像
|
||||||
├── downloads/ # 各站点下载的原始数据
|
├── downloads/ # 各站点下载的原始数据
|
||||||
├── output/ # 比对报表输出
|
├── output/ # 比对报表输出
|
||||||
|
├── state/ # 运行状态持久化（state.db）
|
||||||
└── docs/ # 说明文档
|
└── docs/ # 说明文档
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -63,10 +73,13 @@ python -m venv .venv
|
|||||||
# 2. 安装依赖（含安能 CDP 驱动所需的 websocket-client）
|
# 2. 安装依赖（含安能 CDP 驱动所需的 websocket-client）
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
|
|
||||||
# 3. 安装 Playwright 浏览器内核（网页站点用）
|
# 3. 以可编辑模式安装本包（注册 inbound-verify 等命令）
|
||||||
|
pip install -e .
|
||||||
|
|
||||||
|
# 4. 安装 Playwright 浏览器内核（网页站点用）
|
||||||
playwright install chromium
|
playwright install chromium
|
||||||
|
|
||||||
# 4. 由模板创建本地配置并填入真实凭据
|
# 5. 由模板创建本地配置并填入真实凭据
|
||||||
cp config.example.yaml config.yaml
|
cp config.example.yaml config.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -96,7 +109,10 @@ cp config.example.yaml config.yaml
|
|||||||
## 五、运行
|
## 五、运行
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python main_router.py
|
# 交互菜单（任选其一）
|
||||||
|
python -m inbound_verify.cli.router
|
||||||
|
# 或装包后直接用命令：
|
||||||
|
inbound-verify
|
||||||
```
|
```
|
||||||
|
|
||||||
程序会：
|
程序会：
|
||||||
@@ -126,10 +142,10 @@ python main_router.py
|
|||||||
|
|
||||||
**分层原则：路由层只调度，站点模块自洽。**
|
**分层原则：路由层只调度，站点模块自洽。**
|
||||||
|
|
||||||
- **`main_router.py`（调度层）**：负责启动浏览器 / 安能、就绪轮询、登录检测、
|
- **`inbound_verify.cli.router`（调度层）**：负责启动浏览器 / 安能、就绪轮询、登录检测、
|
||||||
菜单分发。**不关心**"任务能否完成、失败怎么办"——只调
|
菜单分发。**不关心**"任务能否完成、失败怎么办"——只调
|
||||||
`site_xxx.xxx_download(page)` 然后等结果。
|
`inbound_verify.sites.xxx_download(page)` 然后等结果。
|
||||||
- **各 `site_xxx.py`（站点模块）**：每个模块对外只暴露一个"把任务做了"的入口
|
- **各 `inbound_verify.sites.*`（站点模块）**：每个模块对外只暴露一个"把任务做了"的入口
|
||||||
`xxx_download(...)`，内部自行处理一切：
|
`xxx_download(...)`，内部自行处理一切：
|
||||||
- `HOME_URL`：站点首页 URL（也供路由层 `SITES_CONFIG` 引用，单一来源）；
|
- `HOME_URL`：站点首页 URL（也供路由层 `SITES_CONFIG` 引用，单一来源）；
|
||||||
- `xxx_reset(...)`：异常兜底的重置（网页 = 跳首页 URL；安能 = 关业务 tab + 收菜单）；
|
- `xxx_reset(...)`：异常兜底的重置（网页 = 跳首页 URL；安能 = 关业务 tab + 收菜单）；
|
||||||
@@ -162,4 +178,4 @@ python main_router.py
|
|||||||
|
|
||||||
- 首次运行需手动登录各站点（程序会停在就绪轮询，直到检测到所有站点进入工作台）；
|
- 首次运行需手动登录各站点（程序会停在就绪轮询，直到检测到所有站点进入工作台）；
|
||||||
- 安能为单实例 Electron 应用：启动前请先关闭已打开的安能窗口；
|
- 安能为单实例 Electron 应用：启动前请先关闭已打开的安能窗口；
|
||||||
- 所有下载/输出路径以项目目录为基准（见 `paths.py`），与从哪个目录启动无关。
|
- 所有下载/输出路径以项目目录为基准（见 `inbound_verify.paths`），与从哪个目录启动无关。
|
||||||
|
|||||||
579
docs/superpowers/plans/2026-07-23-tier1-package-move.md
Normal file
579
docs/superpowers/plans/2026-07-23-tier1-package-move.md
Normal file
@@ -0,0 +1,579 @@
|
|||||||
|
# Tier 1: Package Move Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Move all 12 flat root-level Python modules into an installable `inbound_verify/` package (sites/, cli/ subpackages), rewrite every internal import to package-qualified form, and expose three console_script entry points — with **zero behavior change**.
|
||||||
|
|
||||||
|
**Architecture:** Pure mechanical relocation (git mv preserves history) + import rewrite + paths.py anchor fix + entry-point `main()` wrappers + `pyproject.toml` packaging. No logic changes. Module names kept for heavily-referenced modules (`state_store`, `expected_undelivered`, `runtime`, `paths`) to avoid call-site churn; only leaf entries (`db_store→store`, `main_router→cli/router`, `server→cli/server`) and site files (drop `site_` prefix) are renamed.
|
||||||
|
|
||||||
|
**Tech Stack:** Python ≥3.10, setuptools (PEP 517/621), pip editable install, psycopg3, FastAPI/uvicorn, Playwright.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- **Python ≥ 3.10** (`requires-python = ">=3.10"` in pyproject).
|
||||||
|
- **Package** import name `inbound_verify`; **distribution** name `inbound-verify`.
|
||||||
|
- **Dependency floors** (verbatim from spec): `pandas>=2.0.0`, `playwright>=1.40.0`, `openpyxl>=3.1.0`, `PyYAML>=6.0`, `websocket-client>=1.0.0`, `fastapi>=0.110.0`, `uvicorn>=0.27.0`, `apscheduler>=3.10.0`, `psycopg[binary]>=3.1`.
|
||||||
|
- **No test suite** (user decision). Verification = `compileall` + import smoke + grep-for-stale-refs + DB connectivity. No pytest.
|
||||||
|
- **Behavior must not change** in Tier 1 — pure move.
|
||||||
|
- **No auto-commit/push.** Every commit step below runs ONLY after the user explicitly says "提交/commit". Commit messages in English.
|
||||||
|
- **Black-format** every changed `.py` (global rule).
|
||||||
|
- All changes are **inside the `InboundVerify` git submodule**; the parent repo pointer bump is a separate parent-repo step, out of scope.
|
||||||
|
- All commands run from the `InboundVerify/` directory using the venv interpreter `.venv/Scripts/python.exe` (Windows; no activation needed).
|
||||||
|
|
||||||
|
**Reference spec:** `docs/superpowers/specs/2026-07-23-package-restructure-design.md` (§3 mapping table, §4 paths anchor, §5 import rules, §6 entry/packaging, §7 verification gate).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure (what each file becomes responsible for)
|
||||||
|
|
||||||
|
```
|
||||||
|
inbound_verify/
|
||||||
|
├── __init__.py # empty (package marker)
|
||||||
|
├── paths.py # path anchors → PROJECT ROOT (one dir above package)
|
||||||
|
├── runtime.py # orchestration core (unchanged logic)
|
||||||
|
├── state_store.py # SQLite state (unchanged; name kept)
|
||||||
|
├── expected_undelivered.py# offline compare (unchanged; name kept — Tier 2 renames to compare)
|
||||||
|
├── store.py # PostgreSQL persist + main() (was db_store.py)
|
||||||
|
├── sites/
|
||||||
|
│ ├── __init__.py # empty
|
||||||
|
│ ├── shunxin.py # (was site_shunxin.py)
|
||||||
|
│ ├── baishi.py # (was site_baishi.py)
|
||||||
|
│ ├── zto.py # (was site_zto.py)
|
||||||
|
│ ├── yunda.py # (was site_yunda.py)
|
||||||
|
│ └── anneng.py # (was site_anneng.py)
|
||||||
|
└── cli/
|
||||||
|
├── __init__.py # empty
|
||||||
|
├── router.py # interactive menu + main() (was main_router.py)
|
||||||
|
└── server.py # FastAPI service + main() (was server.py)
|
||||||
|
```
|
||||||
|
|
||||||
|
Root keeps: `pyproject.toml` (new), `config.yaml`, `config.example.yaml`, `schema.sql`, `requirements.txt`, `README.md`, `CLAUDE.md`, `docs/`, `downloads/`, `output/`, `state/`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 1: Package scaffold + pyproject + editable install
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `inbound_verify/__init__.py`, `inbound_verify/sites/__init__.py`, `inbound_verify/cli/__init__.py`
|
||||||
|
- Create: `pyproject.toml`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Produces: an importable (near-empty) `inbound_verify` package + console_script registration. The flat root scripts remain 100% functional after this task (untouched).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Create package marker files**
|
||||||
|
|
||||||
|
Create three empty files:
|
||||||
|
- `inbound_verify/__init__.py`
|
||||||
|
- `inbound_verify/sites/__init__.py`
|
||||||
|
- `inbound_verify/cli/__init__.py`
|
||||||
|
|
||||||
|
Each is a single comment line:
|
||||||
|
```python
|
||||||
|
# inbound_verify package
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Write pyproject.toml**
|
||||||
|
|
||||||
|
Create `pyproject.toml`:
|
||||||
|
```toml
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "inbound-verify"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "物流到货数据自动下载与应到未到核对工具"
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = [
|
||||||
|
"pandas>=2.0.0",
|
||||||
|
"playwright>=1.40.0",
|
||||||
|
"openpyxl>=3.1.0",
|
||||||
|
"PyYAML>=6.0",
|
||||||
|
"websocket-client>=1.0.0",
|
||||||
|
"fastapi>=0.110.0",
|
||||||
|
"uvicorn>=0.27.0",
|
||||||
|
"apscheduler>=3.10.0",
|
||||||
|
"psycopg[binary]>=3.1",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
inbound-verify = "inbound_verify.cli.router:main"
|
||||||
|
inbound-verify-server = "inbound_verify.cli.server:main"
|
||||||
|
inbound-verify-db = "inbound_verify.store:main"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
include = ["inbound_verify*"]
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Editable-install into the existing venv**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m pip install -e .
|
||||||
|
```
|
||||||
|
Expected: `Successfully installed inbound-verify-0.1.0` (deps already satisfied from earlier install — no network needed).
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify package imports**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify, inbound_verify.sites, inbound_verify.cli; print('package OK')"
|
||||||
|
```
|
||||||
|
Expected output: `package OK`
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit (only after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add inbound_verify/__init__.py inbound_verify/sites/__init__.py inbound_verify/cli/__init__.py pyproject.toml
|
||||||
|
git commit -m "chore: scaffold inbound_verify package and pyproject
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
> Do NOT commit until the user says to.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 2: Atomic move — relocate, rewrite imports, fix anchor, wire mains, verify
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Move (git mv): all 12 root `.py` modules → package locations (see Step 1)
|
||||||
|
- Modify: `inbound_verify/paths.py` (anchor), and import lines + call sites in every moved module
|
||||||
|
- Modify: entry `main()` in `cli/router.py`, `cli/server.py`, `store.py`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: the scaffold from Task 1 (package importable + editable install active).
|
||||||
|
- Produces: a fully functional `inbound_verify` package invokable via `python -m inbound_verify.cli.router`, `python -m inbound_verify.cli.server`, `python -m inbound_verify.store`, or the three console_scripts. Old root `.py` files are gone. The flat root scripts no longer exist — invocation switches to package form.
|
||||||
|
|
||||||
|
> **Why this is one task:** in a flat-import codebase, moving `paths.py` (imported by everyone) immediately breaks every importer until ALL moves + rewrites are complete. There is no intermediate state that imports cleanly, so the whole move is one atomic unit verified by the gate at the end. Each file-edit step below is followed by `py_compile` of that file to catch syntax errors as we go.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Relocate all 12 modules with git mv (history preserved)**
|
||||||
|
|
||||||
|
From the `InboundVerify/` directory:
|
||||||
|
```bash
|
||||||
|
git mv paths.py inbound_verify/paths.py
|
||||||
|
git mv runtime.py inbound_verify/runtime.py
|
||||||
|
git mv state_store.py inbound_verify/state_store.py
|
||||||
|
git mv expected_undelivered.py inbound_verify/expected_undelivered.py
|
||||||
|
git mv db_store.py inbound_verify/store.py
|
||||||
|
git mv main_router.py inbound_verify/cli/router.py
|
||||||
|
git mv server.py inbound_verify/cli/server.py
|
||||||
|
git mv site_shunxin.py inbound_verify/sites/shunxin.py
|
||||||
|
git mv site_baishi.py inbound_verify/sites/baishi.py
|
||||||
|
git mv site_zto.py inbound_verify/sites/zto.py
|
||||||
|
git mv site_yunda.py inbound_verify/sites/yunda.py
|
||||||
|
git mv site_anneng.py inbound_verify/sites/anneng.py
|
||||||
|
```
|
||||||
|
After this the tree is temporarily broken (imports unresolved) — expected. Continue.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Fix paths.py anchor to point at project root**
|
||||||
|
|
||||||
|
In `inbound_verify/paths.py`, replace:
|
||||||
|
```python
|
||||||
|
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
# __file__ = <root>/inbound_verify/paths.py → 上两级 = 项目根
|
||||||
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
```
|
||||||
|
(`DOWNLOAD_DIR`/`OUTPUT_DIR`/`CONFIG_PATH`/`STATE_DB_PATH` lines stay unchanged — they derive from BASE_DIR.)
|
||||||
|
|
||||||
|
Verify syntax:
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m py_compile inbound_verify/paths.py
|
||||||
|
```
|
||||||
|
Expected: no output (success).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Rewrite imports in state_store.py**
|
||||||
|
|
||||||
|
In `inbound_verify/state_store.py`, replace:
|
||||||
|
```python
|
||||||
|
from paths import STATE_DB_PATH
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import STATE_DB_PATH
|
||||||
|
```
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/state_store.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Rewrite imports + call sites in runtime.py**
|
||||||
|
|
||||||
|
In `inbound_verify/runtime.py`, replace the import block:
|
||||||
|
```python
|
||||||
|
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
|
import state_store
|
||||||
|
import site_shunxin
|
||||||
|
import site_baishi
|
||||||
|
import site_zto
|
||||||
|
import site_yunda
|
||||||
|
import site_anneng
|
||||||
|
import expected_undelivered # dispatch 的 compare 任务用
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
|
from inbound_verify import state_store
|
||||||
|
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
|
||||||
|
from inbound_verify import expected_undelivered # dispatch 的 compare 任务用
|
||||||
|
```
|
||||||
|
|
||||||
|
Then **drop the `site_` prefix at every call site** (28 references — 18 in this file). Apply these 5 replacements (all occurrences):
|
||||||
|
- `site_shunxin.` → `shunxin.`
|
||||||
|
- `site_baishi.` → `baishi.`
|
||||||
|
- `site_zto.` → `zto.`
|
||||||
|
- `site_yunda.` → `yunda.`
|
||||||
|
- `site_anneng.` → `anneng.`
|
||||||
|
|
||||||
|
Affected lines in runtime.py (for reference, all must change): 36, 37, 38, 39, 118, 139, 336, 408, 417, 463, 464, 467, 469, 470, 472, 473, 475, 476.
|
||||||
|
|
||||||
|
`state_store.` and `expected_undelivered.` call sites stay UNCHANGED (names kept). Confirm the key handler block now reads:
|
||||||
|
```python
|
||||||
|
TASK_HANDLERS = {
|
||||||
|
("顺心", "expected"): _web_handler("顺心", shunxin.shunxin_expected_download),
|
||||||
|
("顺心", "actual"): _web_handler("顺心", shunxin.shunxin_actual_download),
|
||||||
|
("顺心", "undelivered"): _site_undelivered_handler("顺心"),
|
||||||
|
("百世", "undelivered"): _web_handler(
|
||||||
|
"百世", baishi.baishi_download_undelivered_data
|
||||||
|
),
|
||||||
|
("中通", "expected"): _web_handler("中通", zto.zto_expected_download),
|
||||||
|
("中通", "actual"): _web_handler("中通", zto.zto_actual_download),
|
||||||
|
("中通", "undelivered"): _site_undelivered_handler("中通"),
|
||||||
|
("韵达", "expected"): _web_handler("韵达", yunda.yunda_expected_download),
|
||||||
|
("韵达", "actual"): _web_handler("韵达", yunda.yunda_actual_download),
|
||||||
|
("韵达", "undelivered"): _site_undelivered_handler("韵达"),
|
||||||
|
("安能", "expected"): lambda ctx: anneng.anneng_expected_download(),
|
||||||
|
("安能", "actual"): lambda ctx: anneng.anneng_actual_download(),
|
||||||
|
("安能", "undelivered"): _site_undelivered_handler("安能"),
|
||||||
|
("__compare__", "compare"): lambda ctx: (expected_undelivered.main() or True),
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/runtime.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Rewrite imports in expected_undelivered.py**
|
||||||
|
|
||||||
|
This module has TWO lazy `import state_store` statements (inside functions). Replace each occurrence of:
|
||||||
|
```python
|
||||||
|
import state_store
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify import state_store
|
||||||
|
```
|
||||||
|
(There is no `from paths import` here — the module has its own `BASE/DOWNLOADS/OUTPUT` constants; that dedup is Tier 2, not now.)
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/expected_undelivered.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Rewrite imports + rename _cli→main in store.py**
|
||||||
|
|
||||||
|
In `inbound_verify/store.py`, replace:
|
||||||
|
```python
|
||||||
|
from paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
|
||||||
|
```
|
||||||
|
Replace:
|
||||||
|
```python
|
||||||
|
import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源）
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源）
|
||||||
|
```
|
||||||
|
(`eu.` call sites stay unchanged.)
|
||||||
|
|
||||||
|
Rename the CLI entry: replace the function definition:
|
||||||
|
```python
|
||||||
|
def _cli():
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
def main():
|
||||||
|
```
|
||||||
|
And at the bottom replace:
|
||||||
|
```python
|
||||||
|
if __name__ == "__main__":
|
||||||
|
_cli()
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/store.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 7: Rewrite imports in all 5 site modules**
|
||||||
|
|
||||||
|
In each of `inbound_verify/sites/{shunxin,baishi,zto,yunda,anneng}.py`, replace:
|
||||||
|
```python
|
||||||
|
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
|
import state_store
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
|
from inbound_verify import state_store
|
||||||
|
```
|
||||||
|
(Each site module has exactly these two internal imports; `state_store.` call sites unchanged.)
|
||||||
|
Verify all five:
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m py_compile inbound_verify/sites/shunxin.py inbound_verify/sites/baishi.py inbound_verify/sites/zto.py inbound_verify/sites/yunda.py inbound_verify/sites/anneng.py
|
||||||
|
```
|
||||||
|
Expected: no output.
|
||||||
|
|
||||||
|
- [ ] **Step 8: Rewrite imports + call sites + add main() in cli/router.py**
|
||||||
|
|
||||||
|
In `inbound_verify/cli/router.py`, replace the import block (lines ~14–31):
|
||||||
|
```python
|
||||||
|
from paths import CONFIG_PATH
|
||||||
|
from runtime import (
|
||||||
|
APP_SITES,
|
||||||
|
HEARTBEAT_INTERVAL,
|
||||||
|
dispatch_task,
|
||||||
|
launch_and_prepare,
|
||||||
|
run_heartbeat,
|
||||||
|
)
|
||||||
|
|
||||||
|
import state_store
|
||||||
|
|
||||||
|
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
|
||||||
|
import site_shunxin
|
||||||
|
import site_baishi
|
||||||
|
import site_zto
|
||||||
|
import site_yunda
|
||||||
|
import site_anneng
|
||||||
|
import expected_undelivered
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import CONFIG_PATH
|
||||||
|
from inbound_verify.runtime import (
|
||||||
|
APP_SITES,
|
||||||
|
HEARTBEAT_INTERVAL,
|
||||||
|
dispatch_task,
|
||||||
|
launch_and_prepare,
|
||||||
|
run_heartbeat,
|
||||||
|
)
|
||||||
|
|
||||||
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
|
||||||
|
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
|
||||||
|
from inbound_verify import expected_undelivered
|
||||||
|
```
|
||||||
|
|
||||||
|
Drop the `site_` prefix at the 10 call sites (apply the same 5 replacements as Step 4). Affected lines: 60, 61, 64, 65, 68, 69, 72, 73, 86, 105. (`expected_undelivered.main()` at line 38 stays unchanged.)
|
||||||
|
|
||||||
|
Add an entry function and update the `__main__` guard. Replace:
|
||||||
|
```python
|
||||||
|
if __name__ == "__main__":
|
||||||
|
run_multi_site_daemon()
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
def main():
|
||||||
|
"""交互菜单模式入口。"""
|
||||||
|
run_multi_site_daemon()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/router.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 9: Rewrite imports + add main() in cli/server.py**
|
||||||
|
|
||||||
|
In `inbound_verify/cli/server.py`, replace:
|
||||||
|
```python
|
||||||
|
from paths import DOWNLOAD_DIR, OUTPUT_DIR
|
||||||
|
import state_store
|
||||||
|
from runtime import (
|
||||||
|
HEARTBEAT_INTERVAL,
|
||||||
|
TASK_HANDLERS,
|
||||||
|
dispatch_task,
|
||||||
|
launch_and_prepare,
|
||||||
|
run_heartbeat,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR
|
||||||
|
from inbound_verify import state_store
|
||||||
|
from inbound_verify.runtime import (
|
||||||
|
HEARTBEAT_INTERVAL,
|
||||||
|
TASK_HANDLERS,
|
||||||
|
dispatch_task,
|
||||||
|
launch_and_prepare,
|
||||||
|
run_heartbeat,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the bottom entry block:
|
||||||
|
```python
|
||||||
|
if __name__ == "__main__":
|
||||||
|
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
def main():
|
||||||
|
"""服务模式入口。传字符串导入路径（规范写法；不开 reload/workers 时进程内 import，行为等价）。"""
|
||||||
|
uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/server.py` → no output.
|
||||||
|
|
||||||
|
- [ ] **Step 10: Clean up stale filename comments**
|
||||||
|
|
||||||
|
Cosmetic but keeps grep clean (Step 12 depends on it). In each `inbound_verify/sites/*.py`, update the line-1 header `# site_xxx.py` → `# sites/xxx.py`. In `inbound_verify/sites/anneng.py`, update the standalone-run comment near the top:
|
||||||
|
```python
|
||||||
|
# .venv/Scripts/python.exe site_anneng.py
|
||||||
|
```
|
||||||
|
→
|
||||||
|
```python
|
||||||
|
# python -m inbound_verify.sites.anneng expected # 或 actual
|
||||||
|
```
|
||||||
|
And the comment at the `CDP_PORT` line referencing "独立运行 site_anneng.py" — update to "独立运行（python -m inbound_verify.sites.anneng）".
|
||||||
|
|
||||||
|
- [ ] **Step 11: Black-format all changed files**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m black inbound_verify
|
||||||
|
```
|
||||||
|
Expected: `reformatted ...` / `left unchanged` lines, exit 0.
|
||||||
|
|
||||||
|
- [ ] **Step 12: VERIFICATION GATE — run all five checks**
|
||||||
|
|
||||||
|
**12a. compileall (syntax across whole package):**
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m compileall inbound_verify
|
||||||
|
```
|
||||||
|
Expected: no errors.
|
||||||
|
|
||||||
|
**12b. Import smoke (catches every wrong import path / missed rewrite):**
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, inbound_verify.state_store; print('import smoke OK')"
|
||||||
|
```
|
||||||
|
Expected: `import smoke OK`. (The three entries transitively import sites + expected_undelivered.)
|
||||||
|
|
||||||
|
**12c. No stale `site_` references:**
|
||||||
|
```bash
|
||||||
|
grep -rn "site_shunxin\|site_baishi\|site_zto\|site_yunda\|site_anneng" inbound_verify || echo "no stale site_ refs OK"
|
||||||
|
```
|
||||||
|
Expected: `no stale site_ refs OK`.
|
||||||
|
|
||||||
|
**12d. No stale bare flat imports:**
|
||||||
|
```bash
|
||||||
|
grep -rnE "^from paths import|^from runtime import|^import site_|^import state_store$|^import expected_undelivered$" inbound_verify || echo "no stale flat imports OK"
|
||||||
|
```
|
||||||
|
Expected: `no stale flat imports OK`.
|
||||||
|
|
||||||
|
**12e. paths anchor points at project root + DB still connects:**
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -c "from inbound_verify.paths import BASE_DIR; print('BASE_DIR', BASE_DIR)"
|
||||||
|
.venv/Scripts/python.exe -c "from inbound_verify.store import _connect, _load_pg_config; c=_load_pg_config(); conn=_connect(c['dbname']); print('DB OK', conn.info.server_version); conn.close()"
|
||||||
|
```
|
||||||
|
Expected: `BASE_DIR` prints the `InboundVerify` project root (the dir containing `config.yaml`); `DB OK <pg version>`.
|
||||||
|
|
||||||
|
> If 12b fails with ModuleNotFoundError for a site module, run `.venv/Scripts/python.exe -m pip install -e .` again (editable finder refresh) and retry. If 12d still shows a line, that import was missed — rewrite it per Step 4/8 rules.
|
||||||
|
|
||||||
|
- [ ] **Step 13: Commit (only after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add -A inbound_verify
|
||||||
|
git commit -m "refactor: move flat modules into inbound_verify package (Tier 1, behavior-identical)
|
||||||
|
|
||||||
|
- relocate 12 root .py into inbound_verify/ (sites/, cli/ subpackages)
|
||||||
|
- rewrite all internal imports to package-qualified
|
||||||
|
- fix paths.py BASE_DIR to anchor at project root
|
||||||
|
- add main() entry wrappers; register console_scripts
|
||||||
|
- drop site_ prefix on site modules; keep state_store/expected_undelivered names
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
> Do NOT commit until the user says to. This commit is the **safety baseline**; the manual end-to-end test (spec §7 gate) runs against this state before any Tier 2/Tier 3 work.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 3: Docs sync (README + CLAUDE.md)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `README.md` (§二 directory tree, §三 env prep, §五 run)
|
||||||
|
- Modify: `CLAUDE.md` (常用命令 section)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: the completed package from Task 2 (docs must describe the real new layout/commands).
|
||||||
|
- Produces: documentation matching the new invocation model. No code impact.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update README §二 directory tree**
|
||||||
|
|
||||||
|
Replace the tree block (README lines ~33–49) with the actual new layout:
|
||||||
|
```
|
||||||
|
InboundVerify/
|
||||||
|
├── pyproject.toml # 打包 + 依赖 + console_scripts
|
||||||
|
├── inbound_verify/ # 源码包
|
||||||
|
│ ├── paths.py runtime.py state_store.py expected_undelivered.py store.py
|
||||||
|
│ ├── sites/ shunxin / baishi / zto / yunda / anneng
|
||||||
|
│ └── cli/ router（交互菜单）/ server（FastAPI 服务）
|
||||||
|
├── config.example.yaml / config.yaml
|
||||||
|
├── schema.sql
|
||||||
|
├── requirements.txt # pyproject 的静态镜像
|
||||||
|
├── downloads/ output/ state/
|
||||||
|
└── docs/
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Update README §三 env prep — add editable install**
|
||||||
|
|
||||||
|
In the env-prep command block (README lines ~58–71), after `pip install -r requirements.txt`, add:
|
||||||
|
```bash
|
||||||
|
# 4. 以可编辑模式安装本包（注册 inbound-verify 等命令）
|
||||||
|
pip install -e .
|
||||||
|
```
|
||||||
|
(renumber the subsequent `cp config.example.yaml config.yaml` step).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Update README §五 run — new commands**
|
||||||
|
|
||||||
|
Replace `python main_router.py` with:
|
||||||
|
```bash
|
||||||
|
# 交互菜单（任选其一）
|
||||||
|
python -m inbound_verify.cli.router
|
||||||
|
# 或装包后：inbound-verify
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Update CLAUDE.md 常用命令**
|
||||||
|
|
||||||
|
In the 常用命令 section, change every `.venv/Scripts/python.exe <module>.py` to the package form:
|
||||||
|
- `main_router.py` → `python -m inbound_verify.cli.router` (or `inbound-verify`)
|
||||||
|
- `server.py` → `python -m inbound_verify.cli.server` (or `inbound-verify-server`)
|
||||||
|
- `db_store.py createdb|init|ingest|all` → `python -m inbound_verify.store createdb|init|ingest|all` (or `inbound-verify-db ...`)
|
||||||
|
- `site_anneng.py expected|actual` → `python -m inbound_verify.sites.anneng expected|actual`
|
||||||
|
- `black`/`py_compile` targets → package paths (e.g. `-m black inbound_verify`)
|
||||||
|
|
||||||
|
Add a one-liner near the top of that section: `首次/拉取新代码后需 .venv/Scripts/python.exe -m pip install -e .`。
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify docs render + commands are real**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -n "main_router.py\|python server.py\|python db_store.py\|site_anneng.py\|site_shunxin" README.md CLAUDE.md || echo "no stale old-path commands OK"
|
||||||
|
```
|
||||||
|
Expected: `no stale old-path commands OK` (every old invocation updated).
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit (only after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add README.md CLAUDE.md
|
||||||
|
git commit -m "docs: update README and CLAUDE.md for package layout and console_scripts
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review (completed)
|
||||||
|
|
||||||
|
- **Spec coverage:** spec §3 (layout) → Task 1+2; §4 (paths anchor) → Task 2 Step 2; §5 (import rules) → Task 2 Steps 3–9; §6 (entry/packaging) → Task 1 Step 2 + Task 2 Steps 6/8/9; §7 (verification gate) → Task 2 Step 12; §10 (docs) → Task 3. All covered.
|
||||||
|
- **Placeholder scan:** none — every step has exact code or an exact command with expected output. The 28 call-site rewrites are given as a deterministic prefix-drop rule + enumerated line numbers + verification grep (complete, not a placeholder).
|
||||||
|
- **Type/name consistency:** kept-module names (`state_store`, `expected_undelivered`, `runtime`, `paths`) used consistently across all import rewrites and call sites; renamed entries (`store`, `cli/router`, `cli/server`) consistent with pyproject `[project.scripts]`. `main()` signature consistent across router/server/store and console_scripts.
|
||||||
403
docs/superpowers/plans/2026-07-23-tier2-domain-rename-dedup.md
Normal file
403
docs/superpowers/plans/2026-07-23-tier2-domain-rename-dedup.md
Normal file
@@ -0,0 +1,403 @@
|
|||||||
|
# Tier 2: domain extract + rename + path dedup — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Three targeted, behavior-preserving cleanups of the compare/config layer: (1) make `expected_undelivered` use `paths.DOWNLOAD_DIR/OUTPUT_DIR` instead of its own duplicate anchor; (2) extract shared site/file/column config into a new `domain` module; (3) rename `expected_undelivered` → `compare`.
|
||||||
|
|
||||||
|
**Architecture:** Pure refactor — move definitions, rewire imports, no logic change. Removes the duplicate path anchor that caused the Tier 1 hotfix bug (`17293be`), and the `db_store → expected_undelivered` coupling where the DB layer imported the whole compare engine just to read site config.
|
||||||
|
|
||||||
|
**Tech Stack:** Python ≥3.10, package `inbound_verify`, pandas, openpyxl, psycopg3.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- **Python ≥ 3.10**, package import name `inbound_verify`.
|
||||||
|
- **No behavior change** — pure refactor. Any logic change is a defect.
|
||||||
|
- **No test suite** (user decision). Per-task verification = `compileall` + **import smoke** (fresh process, no backend needed) + grep-for-stale-refs. NOT pytest. (The running backend is unaffected until a restart; a final end-to-end re-confirm is done once at the end of Tier 2.)
|
||||||
|
- **No auto-commit** (project rule). Each task's commit step runs only after the user says "提交". Commit messages in English.
|
||||||
|
- **Black-format** every changed `.py` (global rule).
|
||||||
|
- All changes inside the `InboundVerify` submodule; commits local on `dev`.
|
||||||
|
- Venv interpreter: `.venv/Scripts/python.exe` (absolute: `D:/projects/LogisticsHubIPA/InboundVerify/.venv/Scripts/python.exe`). Run commands from `D:/projects/LogisticsHubIPA/InboundVerify/`.
|
||||||
|
|
||||||
|
**Reference spec:** `docs/superpowers/specs/2026-07-23-package-restructure-design.md` §8 Tier 2.
|
||||||
|
|
||||||
|
**Out of scope (deferred):** `config.py` centralization (consolidating the 6 `open(CONFIG_PATH)+yaml.safe_load` sites in store/runtime×2/anneng/router). It is the churniest Tier 2 item (6 files) for the least marginal value — pure DRY, no pain addressed — and under no-tests more churn = more risk. Reconsider after the rest of Tier 2 lands.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
inbound_verify/
|
||||||
|
├── domain.py # NEW (Task 2): shared site/file/column config — leaf module
|
||||||
|
├── paths.py # unchanged (already the single anchor)
|
||||||
|
├── expected_undelivered.py# Task 1: use paths.* ; Task 2: import config from domain ; Task 3: renamed → compare.py
|
||||||
|
├── compare.py # (after Task 3) was expected_undelivered.py — compare engine + report
|
||||||
|
├── store.py # Task 2: import config from domain (not eu); Task 3: import _read_business_dates from compare
|
||||||
|
├── runtime.py # Task 2: SITE_UNDELIVERED_FILE from domain; Task 3: write_site_file from compare
|
||||||
|
└── cli/router.py # Task 3: compare.main()
|
||||||
|
```
|
||||||
|
|
||||||
|
**`domain.py` responsibility:** pure data — `ALL_REPORT_SITES`, `SITE_UNDELIVERED_FILE`, `BAISHI_FILE`, `BAISHI_COLUMNS`, `arrived_pieces_zhongtong`, `arrived_pieces_by_cols`, `STATIONS`, `_site_cfg`. No `state_store` dependency, no file I/O. Leaf module.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 1: Path dedup — expected_undelivered uses paths.DOWNLOAD_DIR/OUTPUT_DIR
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `inbound_verify/expected_undelivered.py` (lines 43-48 defs; usages at 146,147,298,338,374,400,669,675,676,690)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `paths.DOWNLOAD_DIR`, `paths.OUTPUT_DIR` (already exist, anchored at project root).
|
||||||
|
- Produces: `expected_undelivered` no longer defines `BASE/DOWNLOADS/OUTPUT`; keeps `OUTFILE` (now `= join(OUTPUT_DIR, "应到未到数据.xlsx")`). The 6 `DOWNLOADS` and 1 `OUTPUT` usages point at the `paths.*` constants — same resolved values as the current hotfixed anchor, so behavior identical.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the paths import to the top import block**
|
||||||
|
|
||||||
|
In `inbound_verify/expected_undelivered.py`, after the existing `import os` (top of file), add a line importing the two path constants. (The file currently has no `paths` import — it used its own BASE.)
|
||||||
|
|
||||||
|
Add:
|
||||||
|
```python
|
||||||
|
from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR
|
||||||
|
```
|
||||||
|
(place it alongside the other imports near the top, e.g. right after `import os`.)
|
||||||
|
|
||||||
|
- [ ] **Step 2: Replace the self-anchored BASE/DOWNLOADS/OUTPUT/OUTFILE block**
|
||||||
|
|
||||||
|
Replace this block (currently lines ~43-48):
|
||||||
|
```python
|
||||||
|
# 包内文件：上两级 = 项目根（与 paths.BASE_DIR 一致；站点下载落在 <root>/downloads）。
|
||||||
|
# 注：本模块自带锚点是 paths.py 的重复，Tier 2 计划改为直接引用 paths.DOWNLOAD_DIR/OUTPUT_DIR。
|
||||||
|
BASE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
DOWNLOADS = os.path.join(BASE, "downloads")
|
||||||
|
OUTPUT = os.path.join(BASE, "output")
|
||||||
|
OUTFILE = os.path.join(OUTPUT, "应到未到数据.xlsx")
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
# 比对报表输出文件（路径锚定统一走 paths.py）。
|
||||||
|
OUTFILE = os.path.join(OUTPUT_DIR, "应到未到数据.xlsx")
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Replace all `DOWNLOADS` usages with `DOWNLOAD_DIR`**
|
||||||
|
|
||||||
|
Apply `DOWNLOADS` → `DOWNLOAD_DIR` (6 occurrences, at lines 146, 147, 298, 338, 669, 675, 676). Use replace-all on the token `DOWNLOADS`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Replace the remaining `OUTPUT` usage (the makedirs line)**
|
||||||
|
|
||||||
|
At line ~374, replace:
|
||||||
|
```python
|
||||||
|
os.makedirs(OUTPUT, exist_ok=True)
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||||
|
```
|
||||||
|
(`OUTFILE` at lines 400/690 is unchanged — it's a different token, already redefined in Step 2.)
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify syntax + no stale BASE/DOWNLOADS/OUTPUT**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /d/projects/LogisticsHubIPA/InboundVerify
|
||||||
|
.venv/Scripts/python.exe -m py_compile inbound_verify/expected_undelivered.py
|
||||||
|
grep -nE "\b(BASE|DOWNLOADS|OUTPUT)\b" inbound_verify/expected_undelivered.py || echo "no stale BASE/DOWNLOADS/OUTPUT OK"
|
||||||
|
```
|
||||||
|
Expected: py_compile silent; grep prints `no stale BASE/DOWNLOADS/OUTPUT OK` (OUTFILE is a different token, won't match).
|
||||||
|
|
||||||
|
- [ ] **Step 6: Import smoke + paths equivalence**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -c "from inbound_verify import expected_undelivered as eu; from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR; import os; print('OUTFILE dir matches OUTPUT_DIR:', os.path.dirname(eu.OUTFILE)==OUTPUT_DIR)"
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime; print('import smoke OK')"
|
||||||
|
```
|
||||||
|
Expected: `OUTFILE dir matches OUTPUT_DIR: True` and `import smoke OK`.
|
||||||
|
|
||||||
|
- [ ] **Step 7: Black + commit (after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m black inbound_verify/expected_undelivered.py
|
||||||
|
git add inbound_verify/expected_undelivered.py
|
||||||
|
git commit -m "refactor: expected_undelivered uses paths.DOWNLOAD_DIR/OUTPUT_DIR (Tier 2)
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 2: Extract `domain.py` (shared site/file/column config)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `inbound_verify/domain.py`
|
||||||
|
- Modify: `inbound_verify/expected_undelivered.py` (remove the moved block, import from domain)
|
||||||
|
- Modify: `inbound_verify/store.py` (config from domain, not eu)
|
||||||
|
- Modify: `inbound_verify/runtime.py` (SITE_UNDELIVERED_FILE from domain)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: nothing new (moves existing definitions verbatim).
|
||||||
|
- Produces: `inbound_verify.domain` exposing `ALL_REPORT_SITES`, `SITE_UNDELIVERED_FILE`, `BAISHI_FILE`, `BAISHI_COLUMNS`, `arrived_pieces_zhongtong(df)`, `arrived_pieces_by_cols(wb_col, piece_col)`, `STATIONS` (list of dicts), `_site_cfg(name)`. These are the exact same objects that lived in `expected_undelivered` lines 50-135.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Create `inbound_verify/domain.py`**
|
||||||
|
|
||||||
|
Create `inbound_verify/domain.py` with this exact content (moved verbatim from expected_undelivered.py lines 50-135, plus the `defaultdict` import it needs):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""domain.py — 站点 / 文件名 / 列映射的共享配置（单一来源）。
|
||||||
|
|
||||||
|
比对（compare）与入库（store）都依赖这套配置；抽出独立 leaf 模块，
|
||||||
|
让 store 不必为读配置而依赖整个比对引擎。纯数据，无 state_store / 文件 IO 依赖。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from collections import defaultdict
|
||||||
|
|
||||||
|
# 汇总报表覆盖的全部站点（4 站在前、百世在末；汇总页图表只取 4 站）
|
||||||
|
ALL_REPORT_SITES = ["顺心", "中通", "韵达", "安能", "百世"]
|
||||||
|
# 4 站单站未到明细文件名（百世未到文件由站点直接产出，名为 BAISHI_FILE）
|
||||||
|
SITE_UNDELIVERED_FILE = "{name}-未到数据.xlsx"
|
||||||
|
BAISHI_FILE = "百世-应到未到货物数据.xlsx"
|
||||||
|
BAISHI_COLUMNS = ["类型", "子单号", "运单号", "最新扫描记录"]
|
||||||
|
|
||||||
|
|
||||||
|
def arrived_pieces_zhongtong(df):
|
||||||
|
"""中通：实到「运单号」为复合串（H + 运单号(12) + 总数(4) + 顺序(4)）。
|
||||||
|
基号 = v[:-8]（与应到表运单号对齐），单件 = 整串（每串即一件）。"""
|
||||||
|
res = defaultdict(set)
|
||||||
|
for v in df["运单号"]:
|
||||||
|
v = str(v).strip()
|
||||||
|
if len(v) > 8 and v[-4:].isdigit():
|
||||||
|
res[v[:-8]].add(v) # 以完整复合串作为“已到单号”存入
|
||||||
|
return res
|
||||||
|
|
||||||
|
|
||||||
|
def arrived_pieces_by_cols(wb_col, piece_col):
|
||||||
|
"""顺心 / 韵达 / 安能：按干净运单列分组，单件 = 子单号 / 扫描单号。
|
||||||
|
wb_col：实到表中与应到运单号对齐的干净列
|
||||||
|
（顺心=运单号 / 韵达=主单号 / 安能=所属单号）
|
||||||
|
piece_col：实到表中每件货物的单号列（子单号 / 扫描单号）"""
|
||||||
|
|
||||||
|
def parse(df):
|
||||||
|
res = defaultdict(set)
|
||||||
|
for m, s in zip(df[wb_col], df[piece_col]):
|
||||||
|
m, s = str(m).strip(), str(s).strip()
|
||||||
|
if m and s:
|
||||||
|
res[m].add(s)
|
||||||
|
return res
|
||||||
|
|
||||||
|
return parse
|
||||||
|
|
||||||
|
|
||||||
|
STATIONS = [
|
||||||
|
{
|
||||||
|
"name": "中通",
|
||||||
|
"exp": "中通-应到货物数据.xlsx",
|
||||||
|
"act": "中通-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数", # 应到件数口径：交接件数（非录单件数）
|
||||||
|
"exp_wb": "运单号", # 应到表运单号列（兼作去重键）
|
||||||
|
"exp_jd": "交接单号", # 未到数据需展示的交接单号
|
||||||
|
"arrived_pieces": arrived_pieces_zhongtong,
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "顺心",
|
||||||
|
"exp": "顺心-应到货物数据.xlsx",
|
||||||
|
"act": "顺心-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("运单号", "子单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "韵达",
|
||||||
|
"exp": "韵达-应到货物数据.xlsx",
|
||||||
|
"act": "韵达-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("主单号", "子单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "安能",
|
||||||
|
"exp": "安能-应到货物数据.xlsx",
|
||||||
|
"act": "安能-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("所属单号", "扫描单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _site_cfg(name):
|
||||||
|
"""按名称取 4 站配置（百世不在 STATIONS，返回 None）。"""
|
||||||
|
return next((c for c in STATIONS if c["name"] == name), None)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Remove the moved block from expected_undelivered.py and import from domain**
|
||||||
|
|
||||||
|
In `inbound_verify/expected_undelivered.py`:
|
||||||
|
- Add to the top import block:
|
||||||
|
```python
|
||||||
|
from inbound_verify.domain import (
|
||||||
|
ALL_REPORT_SITES,
|
||||||
|
BAISHI_COLUMNS,
|
||||||
|
BAISHI_FILE,
|
||||||
|
SITE_UNDELIVERED_FILE,
|
||||||
|
STATIONS,
|
||||||
|
_site_cfg,
|
||||||
|
arrived_pieces_by_cols,
|
||||||
|
arrived_pieces_zhongtong,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
- Delete the now-duplicated definitions (the block from `ALL_REPORT_SITES = ...` through the end of `_site_cfg`, i.e. old lines ~50-135 — the comment header `# 汇总报表...` through `return next(...)`). These now live in domain.py. The `from collections import defaultdict` import in expected_undelivered.py can stay (harmless) or be removed if unused — check with grep after.
|
||||||
|
|
||||||
|
- [ ] **Step 3: store.py — take config from domain, _read_business_dates still from eu**
|
||||||
|
|
||||||
|
In `inbound_verify/store.py`:
|
||||||
|
- Add to imports:
|
||||||
|
```python
|
||||||
|
from inbound_verify.domain import BAISHI_FILE, _site_cfg
|
||||||
|
```
|
||||||
|
- Replace `cfg = eu._site_cfg(site)` (2 occurrences, lines 264 and 296) → `cfg = _site_cfg(site)`.
|
||||||
|
- Replace `eu.BAISHI_FILE` (2 occurrences, lines 335 and 337) → `BAISHI_FILE`.
|
||||||
|
- Leave `eu._read_business_dates(...)` (line 213) unchanged — that's compare behavior, stays accessed via the compare module (`eu`). The `from inbound_verify import expected_undelivered as eu` import stays for this.
|
||||||
|
|
||||||
|
- [ ] **Step 4: runtime.py — SITE_UNDELIVERED_FILE from domain**
|
||||||
|
|
||||||
|
In `inbound_verify/runtime.py`:
|
||||||
|
- Add to imports:
|
||||||
|
```python
|
||||||
|
from inbound_verify.domain import SITE_UNDELIVERED_FILE
|
||||||
|
```
|
||||||
|
- Replace (line ~448):
|
||||||
|
```python
|
||||||
|
DOWNLOAD_DIR, expected_undelivered.SITE_UNDELIVERED_FILE.format(name=site)
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
DOWNLOAD_DIR, SITE_UNDELIVERED_FILE.format(name=site)
|
||||||
|
```
|
||||||
|
- Leave `expected_undelivered.write_site_file(site)` (line ~446) and `expected_undelivered.main()` (line ~476) unchanged (compare behavior).
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify — compileall + import smoke + domain is leaf**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /d/projects/LogisticsHubIPA/InboundVerify
|
||||||
|
.venv/Scripts/python.exe -m compileall -q inbound_verify
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.domain as d; print('STATIONS:', [s['name'] for s in d.STATIONS]); print('_site_cfg(中通):', d._site_cfg('中通')['exp']); print('BAISHI_FILE:', d.BAISHI_FILE)"
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, inbound_verify.expected_undelivered; print('import smoke OK')"
|
||||||
|
```
|
||||||
|
Expected: STATIONS lists `['中通', '顺心', '韵达', '安能']`; `_site_cfg('中通')['exp']` = `中通-应到货物数据.xlsx`; `BAISHI_FILE` = `百世-应到未到货物数据.xlsx`; `import smoke OK`.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Black + commit (after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m black inbound_verify/domain.py inbound_verify/expected_undelivered.py inbound_verify/store.py inbound_verify/runtime.py
|
||||||
|
git add inbound_verify/domain.py inbound_verify/expected_undelivered.py inbound_verify/store.py inbound_verify/runtime.py
|
||||||
|
git commit -m "refactor: extract domain.py (shared site/file/colmap config) from expected_undelivered
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 3: Rename `expected_undelivered.py` → `compare.py`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Move: `inbound_verify/expected_undelivered.py` → `inbound_verify/compare.py` (git mv)
|
||||||
|
- Modify: `inbound_verify/store.py`, `inbound_verify/runtime.py`, `inbound_verify/cli/router.py` (import + call-site rewrites)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: Task 2's `domain` (compare still uses it).
|
||||||
|
- Produces: module is `inbound_verify.compare`; all public names (`main`, `write_site_file`, `_read_business_dates`) unchanged. `expected_undelivered` no longer exists as a module name.
|
||||||
|
|
||||||
|
- [ ] **Step 1: git mv the module (history preserved)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /d/projects/LogisticsHubIPA/InboundVerify
|
||||||
|
git mv inbound_verify/expected_undelivered.py inbound_verify/compare.py
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: store.py — import compare instead of expected_undelivered**
|
||||||
|
|
||||||
|
In `inbound_verify/store.py`, replace:
|
||||||
|
```python
|
||||||
|
from inbound_verify import expected_undelivered as eu
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify import compare
|
||||||
|
```
|
||||||
|
and replace the one call site (line ~213):
|
||||||
|
```python
|
||||||
|
return eu._read_business_dates(ALL_SITES + ["百世"]) or {}
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
return compare._read_business_dates(ALL_SITES + ["百世"]) or {}
|
||||||
|
```
|
||||||
|
(`_site_cfg` and `BAISHI_FILE` already come from `domain` after Task 2 — no change there.)
|
||||||
|
|
||||||
|
- [ ] **Step 3: runtime.py — import compare, fix write_site_file/main**
|
||||||
|
|
||||||
|
In `inbound_verify/runtime.py`, replace:
|
||||||
|
```python
|
||||||
|
from inbound_verify import expected_undelivered
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify import compare
|
||||||
|
```
|
||||||
|
Replace `expected_undelivered.write_site_file(site)` (line ~446) → `compare.write_site_file(site)`.
|
||||||
|
Replace `(expected_undelivered.main() or True)` (line ~476) → `(compare.main() or True)`.
|
||||||
|
|
||||||
|
- [ ] **Step 4: cli/router.py — import compare, fix main()**
|
||||||
|
|
||||||
|
In `inbound_verify/cli/router.py`, replace:
|
||||||
|
```python
|
||||||
|
from inbound_verify import expected_undelivered
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
from inbound_verify import compare
|
||||||
|
```
|
||||||
|
Replace `expected_undelivered.main()` (line ~34, inside `run_undelivered_compare`) → `compare.main()`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Verify — no stale references + import smoke**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /d/projects/LogisticsHubIPA/InboundVerify
|
||||||
|
.venv/Scripts/python.exe -m compileall -q inbound_verify
|
||||||
|
grep -rn "expected_undelivered" inbound_verify || echo "no stale expected_undelivered refs OK"
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.compare, inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime; print('import smoke OK')"
|
||||||
|
```
|
||||||
|
Expected: compileall silent; grep prints `no stale expected_undelivered refs OK`; `import smoke OK`.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Black + commit (after user confirms)**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m black inbound_verify
|
||||||
|
git add -A inbound_verify
|
||||||
|
git commit -m "refactor: rename expected_undelivered to compare
|
||||||
|
|
||||||
|
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Final end-to-end re-confirm (once, after all 3 tasks; requires backend restart + re-login)
|
||||||
|
|
||||||
|
After Task 3 commits, the running backend still has the old modules loaded. To confirm end-to-end behavior is unchanged on the refactored code:
|
||||||
|
|
||||||
|
- [ ] **Restart backend** (TaskStop current → clean ms-playwright/anneng orphans → `python -m inbound_verify.cli.server`), re-login all sites.
|
||||||
|
- [ ] **Re-run the gate from the Tier 1 manual test:** trigger `undelivered` for 顺心/中通/韵达/安能 via API → expect 4× success + 4× `*-未到数据.xlsx`; trigger `__compare__` → expect `output/应到未到数据.xlsx`; run `python -m inbound_verify.store ingest` → expect rows UPSERTed. All green = Tier 2 behavior-identical, done.
|
||||||
|
|
||||||
|
> If you want to skip the re-login cost: the per-task import-smoke gates already prove the import graph is correct and the changes are behavior-preserving moves. The e2e re-confirm is belt-and-suspenders.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review (completed)
|
||||||
|
|
||||||
|
- **Spec coverage:** spec §8 Tier 2 — path dedup → Task 1; domain extract → Task 2; rename → compare → Task 3. config.py explicitly deferred (noted with rationale). All spec items addressed or consciously deferred.
|
||||||
|
- **Placeholder scan:** none — every step has exact code or exact commands with expected output. The domain.py content is the verbatim extracted block.
|
||||||
|
- **Type/name consistency:** `_site_cfg`, `BAISHI_FILE`, `SITE_UNDELIVERED_FILE`, `STATIONS`, `write_site_file`, `main`, `_read_business_dates` referenced consistently across tasks. Task 2 routes config to `domain` and leaves behavior (`_read_business_dates`, `write_site_file`, `main`) in compare — verified against store.py/runtime.py/router.py usages. Task 3 renames the module but preserves all public names.
|
||||||
274
docs/superpowers/specs/2026-07-23-package-restructure-design.md
Normal file
274
docs/superpowers/specs/2026-07-23-package-restructure-design.md
Normal file
@@ -0,0 +1,274 @@
|
|||||||
|
# InboundVerify 包化重构设计
|
||||||
|
|
||||||
|
- **日期**:2026-07-23
|
||||||
|
- **方案**:C(规范)—— `pyproject.toml` + `[project.scripts]` + 根级包 `inbound_verify/`,根目录不留 `.py` 薄壳
|
||||||
|
- **力度**:彻底(建包 + 定点小改进 + 大文件拆分),但**大文件拆分(Tier 3)前置手动测试闸门**
|
||||||
|
- **状态**:已与用户对齐,待 spec 评审
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 背景与目标
|
||||||
|
|
||||||
|
当前 13 个 Python 模块(共约 6500 行)全部平铺在仓库根目录,随脚本量增长结构混乱。本设计将其重组为一个规范的、可 `pip install -e .` 安装的 Python 包。
|
||||||
|
|
||||||
|
**目标**
|
||||||
|
|
||||||
|
1. 扁平脚本 → `inbound_verify/` 包(`sites/`、`cli/` 两个子包,其余平铺包根,避免一层只放一两个文件的过度嵌套)。
|
||||||
|
2. 全部内部 import 改为包内绝对引用。
|
||||||
|
3. `pyproject.toml` 打包,`[project.scripts]` 暴露命令入口;**根目录不留 `.py` 薄壳**(规范要求)。
|
||||||
|
4. 顺手抽离共享配置、消重(定点小改进,Tier 2)。
|
||||||
|
5. 大文件拆分作为**最后、可选、风险隔离**的阶段(Tier 3),且必须先过手动测试闸门。
|
||||||
|
|
||||||
|
**已确认约束**
|
||||||
|
|
||||||
|
- InboundVerify 是 git **子模块**;改动在子模块内提交,父仓库 `LogisticsHubIPA` 仅跟踪子模块指针。
|
||||||
|
- **不加测试套件**(用户决定);验证靠 `compileall` + 导入冒烟 + grep 查残留引用 + 手动端到端测试。
|
||||||
|
- 入口走规范:无根薄壳;一次性 `pip install -e .` 后用命令或 `python -m` 启动。
|
||||||
|
- **不做 src/ 布局**(内部工具、不发 PyPI、无测试,边际价值有限)。
|
||||||
|
- 本仓库约定:**不自动提交 / 不自动推送**;改动等用户明确说"提交"再 commit/push(本约定覆盖 brainstorming 默认的"写完即提交")。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 现状分析
|
||||||
|
|
||||||
|
### 2.1 依赖分层(自底向上)
|
||||||
|
|
||||||
|
```
|
||||||
|
paths ← 万物之基(被所有模块 import)
|
||||||
|
state_store ← SQLite 状态持久化
|
||||||
|
expected_undelivered ← 离线比对(兼职存站点/文件名/列映射共享配置,被 db_store 复用 —— 耦合点)
|
||||||
|
site_*.py(×5) ← 各依赖 paths + state_store
|
||||||
|
runtime ← 编排核心(启动/派发/心跳,依赖上面全部)
|
||||||
|
main_router / server / db_store ← 三入口(均有 __main__/CLI)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 关键发现
|
||||||
|
|
||||||
|
1. **`paths.py` 是地雷**:`BASE_DIR = dirname(abspath(__file__))`——paths.py 在哪,根就在哪。搬进子目录后必须改为上跳一级,否则 `downloads/`、`output/`、`config.yaml`、`state/state.db`、`schema.sql` 全部跑偏。
|
||||||
|
2. **import 全是扁平顶层**(`import site_shunxin`、`from paths import …`),且**没有任何地方按字符串名引用模块**(`TASK_HANDLERS` 用函数引用、`dispatch_task` 用 site/kind 字典),改写纯机械。
|
||||||
|
3. **重复代码**:`with_retry`、`_remove_if_exists` 在 5 个站点逐字重复(CLAUDE.md 注明"现阶段刻意不优化结构")。
|
||||||
|
4. **路径常量重复**:`expected_undelivered.py` 自带 `BASE/DOWNLOADS/OUTPUT`,与 `paths.py` 重复。
|
||||||
|
5. **耦合**:`db_store` 为读站点配置而 `import expected_undelivered`(为了配置而依赖整个比对引擎)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 目标目录结构
|
||||||
|
|
||||||
|
```
|
||||||
|
InboundVerify/
|
||||||
|
├── pyproject.toml # 新增:打包 + 依赖 + console_scripts
|
||||||
|
├── config.example.yaml
|
||||||
|
├── config.yaml # 留根(gitignored)
|
||||||
|
├── schema.sql # 留根(store 按绝对路径读)
|
||||||
|
├── requirements.txt # 保留为静态镜像(pyproject 为准)
|
||||||
|
├── README.md / CLAUDE.md / .gitignore
|
||||||
|
├── downloads/ output/ state/ # 运行时数据,留根
|
||||||
|
├── docs/
|
||||||
|
└── inbound_verify/ # ← 包
|
||||||
|
├── __init__.py
|
||||||
|
├── __main__.py # 可选:python -m inbound_verify → 交互菜单
|
||||||
|
├── paths.py # 锚点改为指向项目根(§4)
|
||||||
|
├── config.py # 新增(Tier2):集中 load_config()
|
||||||
|
├── domain.py # 新增(Tier2):从 expected_undelivered 抽出的共享站点/文件/列映射
|
||||||
|
├── runtime.py # ← runtime.py(编排核心,整体保留不拆)
|
||||||
|
├── state_store.py # ← state_store.py(保留名,仅搬运)
|
||||||
|
├── expected_undelivered.py # ← 搬运;Tier2 改名 compare.py + 抽 domain.py
|
||||||
|
├── store.py # ← db_store.py(叶子,改名无 churn)
|
||||||
|
├── sites/
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── shunxin.py baishi.py zto.py yunda.py # ← site_*.py(去 site_ 前缀)
|
||||||
|
│ └── anneng.py # ← site_anneng.py(Tier3 可选再拆成子包)
|
||||||
|
└── cli/
|
||||||
|
├── __init__.py
|
||||||
|
├── router.py # ← main_router.py(叶子,改名无 churn)
|
||||||
|
└── server.py # ← server.py(叶子)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.1 搬运映射表(全部 `git mv` 保历史)
|
||||||
|
|
||||||
|
| 现在 | Tier 1 后 | 调用点改动 |
|
||||||
|
|---|---|---|
|
||||||
|
| `paths.py` | `inbound_verify/paths.py` | 无(仅改 import 行 + 锚点) |
|
||||||
|
| `runtime.py` | `inbound_verify/runtime.py` | 无(保留名) |
|
||||||
|
| `state_store.py` | `inbound_verify/state_store.py` | 无(**保留名**,仅改 import 行) |
|
||||||
|
| `expected_undelivered.py` | `inbound_verify/expected_undelivered.py` | 无(Tier1 保留名;Tier2 改名 compare) |
|
||||||
|
| `db_store.py` | `inbound_verify/store.py` | 无(叶子,无人 import) |
|
||||||
|
| `main_router.py` | `inbound_verify/cli/router.py` | 无(叶子) |
|
||||||
|
| `server.py` | `inbound_verify/cli/server.py` | 无(叶子) |
|
||||||
|
| `site_{shunxin,baishi,zto,yunda,anneng}.py` | `inbound_verify/sites/{…}.py`(去 `site_` 前缀) | 改 `runtime` + `main_router` 调用点;**grep 查残留引用兜底** |
|
||||||
|
|
||||||
|
> **命名策略(降低无测试下的风险)**:Tier 1 只对**被多处裸名引用**的模块(`state_store`、`expected_undelivered`、`runtime`、`paths`)**保留原名**,做到"仅改 import 行、零调用点改动";只对**叶子入口**(`db_store`/`main_router`/`server`)和**站点文件**(去 `site_` 前缀)做改名。`expected_undelivered` 的改名(`→ compare.py`)推迟到 Tier 2——那时本就要为抽 `domain.py` 重做该文件及其调用方,把改名 churn 并入一个已经在改的批次。这样 Tier 1 的纯搬运可被 `compileall` + 导入冒烟 + grep 充分验证,不依赖手测。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. `paths.py` 锚点修正(唯一地雷,必须改对)
|
||||||
|
|
||||||
|
包搬进 `inbound_verify/` 后,`__file__` 多降一级。要把 `BASE_DIR` 继续指回项目根:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# inbound_verify/paths.py
|
||||||
|
import os
|
||||||
|
# __file__ = .../InboundVerify/inbound_verify/paths.py
|
||||||
|
# 上两级 = .../InboundVerify (= 项目根,config.yaml/downloads/schema.sql 所在)
|
||||||
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
DOWNLOAD_DIR = os.path.join(BASE_DIR, "downloads")
|
||||||
|
OUTPUT_DIR = os.path.join(BASE_DIR, "output")
|
||||||
|
CONFIG_PATH = os.path.join(BASE_DIR, "config.yaml")
|
||||||
|
STATE_DB_PATH = os.path.join(BASE_DIR, "state", "state.db")
|
||||||
|
```
|
||||||
|
|
||||||
|
editable 安装不会移动文件,`__file__` 仍指向源码树,两级上跳稳定指向项目根。`store.py` 的 `SCHEMA_PATH = join(BASE_DIR, "schema.sql")` 自动跟着对。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. import 改写规则
|
||||||
|
|
||||||
|
```python
|
||||||
|
from paths import ... → from inbound_verify.paths import ...
|
||||||
|
import state_store → from inbound_verify import state_store # 保留名,调用点 state_store.X 不变
|
||||||
|
from runtime import (...) → from inbound_verify.runtime import (...)
|
||||||
|
import site_shunxin → from inbound_verify.sites import shunxin # 5 站同理,调用点改 shunxin.X
|
||||||
|
import expected_undelivered → from inbound_verify import expected_undelivered # Tier1 保留名
|
||||||
|
import expected_undelivered as eu → from inbound_verify import expected_undelivered as eu # eu.X 不变
|
||||||
|
```
|
||||||
|
|
||||||
|
(Tier 2 后,`expected_undelivered` 改名 `compare`,`db_store` 的共享配置改 `from inbound_verify import domain`。)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 入口与打包
|
||||||
|
|
||||||
|
### 6.1 `main()` 包装(根目录不留薄壳)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# inbound_verify/cli/server.py 末尾
|
||||||
|
def main():
|
||||||
|
uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000)
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
|
||||||
|
> `uvicorn.run` 改传**字符串** `"inbound_verify.cli.server:app"`(规范写法;不开 `reload`/`workers` 时仍在当前进程 import,行为等价)。`router.py` 套 `main()` 调 `run_multi_site_daemon()`;`store.py` 已有 `_cli`,改名为 `main`。
|
||||||
|
|
||||||
|
### 6.2 `pyproject.toml`(骨架)
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "inbound-verify"
|
||||||
|
version = "0.1.0"
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = ["pandas>=2.0.0", "playwright>=1.40.0", "openpyxl>=3.1.0",
|
||||||
|
"PyYAML>=6.0", "websocket-client>=1.0.0", "fastapi>=0.110.0",
|
||||||
|
"uvicorn>=0.27.0", "apscheduler>=3.10.0", "psycopg[binary]>=3.1"]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
inbound-verify = "inbound_verify.cli.router:main"
|
||||||
|
inbound-verify-server = "inbound_verify.cli.server:main"
|
||||||
|
inbound-verify-db = "inbound_verify.store:main"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
include = ["inbound_verify*"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.3 装包后三种启动方式(都汇到同一个 `main()`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e . # 一次性
|
||||||
|
inbound-verify-server # 命令
|
||||||
|
python -m inbound_verify.cli.server # 兜底
|
||||||
|
uvicorn inbound_verify.cli.server:app # 生产最标准(端口/worker 命令行控)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 执行顺序与验证闸门(为"无测试"量身)
|
||||||
|
|
||||||
|
**安全原则**:Tier 1 必须先独立完成并验证为**行为等价**,再做任何动逻辑的改动。每段后必验证。验证职责分清——自动化部分我跑,端到端部分需你跑(真实登录/凭据/安能 Electron 只有你能提供)。
|
||||||
|
|
||||||
|
```
|
||||||
|
Tier 1 纯搬运(行为零改变)
|
||||||
|
├─ 我的自动验证: ① compileall 全过 ② 导入冒烟 ③ grep 查残留 site_ 引用 ④ DB 连通
|
||||||
|
└─ [等用户说"提交"] commit ← 安全基线
|
||||||
|
Tier 2 domain 抽取 / config 集中 / 路径消重 / expected_undelivered 改名 compare
|
||||||
|
├─ 我的自动验证: 同上
|
||||||
|
└─ [等用户说"提交"] commit
|
||||||
|
═══════ 手动测试闸门(你跑)═══════
|
||||||
|
全链路端到端:
|
||||||
|
起 inbound-verify → 登录 5 站(顺心双账号)→ 各站下载 → 比对(菜单 9)
|
||||||
|
→ inbound-verify-db ingest → 核对 output/应到未到数据.xlsx 与 PostgreSQL 三张表
|
||||||
|
通过? 否 → 修到通过
|
||||||
|
是 ↓
|
||||||
|
Tier 3 此时再定:anneng 拆不拆 / with_retry 抽不抽 base.py
|
||||||
|
└─ 每项后重跑我的自动验证,你按需复测
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.1 自动验证命令清单(我每个 Tier 后都跑)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/Scripts/python.exe -m compileall inbound_verify # ① 语法
|
||||||
|
.venv/Scripts/python.exe -c "import inbound_verify.cli.router, \
|
||||||
|
inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, \
|
||||||
|
inbound_verify.state_store" # ② 导入冒烟(三个入口会传递导入 sites/expected_undelivered 等;
|
||||||
|
# Tier2 改名后 expected_undelivered → compare,无需单独显式导入)
|
||||||
|
grep -rn "site_shunxin\|site_baishi\|site_zto\|site_yunda\|site_anneng" \
|
||||||
|
inbound_verify || echo "无残留 site_ 引用 ✓" # ③ 改名残留
|
||||||
|
.venv/Scripts/python.exe -c "from inbound_verify.store import _connect, _load_pg_config; \
|
||||||
|
c=_load_pg_config(); conn=_connect(c['dbname']); print('DB OK', conn.info.server_version)" # ④ DB
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 各 Tier 内容
|
||||||
|
|
||||||
|
### Tier 1 — 纯搬运(行为零改变)
|
||||||
|
|
||||||
|
1. 建包骨架 + 空白 `__init__.py`(`sites/`、`cli/`)。
|
||||||
|
2. `git mv` §3.1 表中所有文件到新位置。
|
||||||
|
3. 按 §5 规则机械改写全部 import;站点改名后改 `runtime` + `main_router` 调用点(`shunxin.shunxin_expected_download(...)` 等)。
|
||||||
|
4. 改 `paths.py` 锚点(§4)。
|
||||||
|
5. 三个入口加 `main()`(`store` 的 `_cli` 改名 `main`)。
|
||||||
|
6. 写 `pyproject.toml`,`.venv` 内 `pip install -e .`。
|
||||||
|
7. 跑 §7.1 四项自动验证。
|
||||||
|
8. 等用户说"提交"→ commit(子模块内)。
|
||||||
|
|
||||||
|
### Tier 2 — 定点小改进(每项后跑 §7.1)
|
||||||
|
|
||||||
|
- **抽 `domain.py`**:把 `expected_undelivered.py` 里的 `STATIONS`/`_site_cfg`/`ALL_REPORT_SITES`/`SITE_UNDELIVERED_FILE`/`BAISHI_FILE`/`BAISHI_COLUMNS`/`arrived_pieces_*` 移到 `inbound_verify/domain.py`;`store.py` 从依赖整个比对引擎改为 `from inbound_verify import domain`。**接缝最干净、收益明确,推荐做。**
|
||||||
|
- **`expected_undelivered.py` → `compare.py`** 改名,更新调用方(`store` 的 `as eu`、`runtime`、`cli/router`)。
|
||||||
|
- **加 `config.py`**:集中 `load_config()`(带缓存),各站点把自家的 `yaml.safe_load(open(CONFIG_PATH))` 换掉。
|
||||||
|
- **消重路径常量**:`compare.py` 自带那份 `BASE/DOWNLOADS/OUTPUT` 改成引用 `paths.py`。
|
||||||
|
|
||||||
|
### Tier 3 — 大文件拆分(手动测试闸门之后;具体决策推迟到闸门)
|
||||||
|
|
||||||
|
- **`anneng.py` → `sites/anneng/` 子包**(`cdp.py`/`nav.py`/`expected.py`/`actual.py`):接缝分层清晰,但共享可变状态多(`CDP_PORT` 全局被 `set_cdp_port` 改、各种 URL hint、僵尸 tab 逻辑),CLAUDE.md 标注为脚gun。**倾向不拆**(1375 行虽大但是内聚的 CDP 驱动,强拆无测试网兜底风险高)——最终在闸门后定。
|
||||||
|
- **抽 `sites/base.py`**:`with_retry` / `_remove_if_exists` 在 5 站逐字重复;原作者在 CLAUDE.md 写"现阶段刻意不优化结构"。**抽不抽,闸门后定。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 待决策(推迟到手动测试闸门)
|
||||||
|
|
||||||
|
| 决策 | 默认倾向 | 何时定 |
|
||||||
|
|---|---|---|
|
||||||
|
| `anneng.py` 拆不拆子包 | **不拆** | 手动测试通过后 |
|
||||||
|
| `with_retry`/`_remove_if_exists` 抽 `base.py` | 待定 | 手动测试通过后 |
|
||||||
|
| `requirements.txt` 留还是删 | **留静态镜像**(注明 pyproject 为准) | 随时可改 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. 文档同步(Tier 1 必做)
|
||||||
|
|
||||||
|
- **README.md**:第二节目录树、第三节环境准备(加 `pip install -e .`)、第五节运行(改新命令)。
|
||||||
|
- **CLAUDE.md**:常用命令段全部改 `python -m inbound_verify…` / `inbound-verify…`,补 `pip install -e .` 与 `playwright install chromium`、`black`/`py_compile` 的包内路径写法。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. 范围外
|
||||||
|
|
||||||
|
- **不做** src/ 布局。
|
||||||
|
- **不做** 给站点流程加 mock 测试。
|
||||||
|
- **不自动** commit/push(等用户明确指示)。
|
||||||
|
- 父仓库 `LogisticsHubIPA` 的子模块指针更新,是父仓库的单独一步,不在本 spec 范围。
|
||||||
1
inbound_verify/__init__.py
Normal file
1
inbound_verify/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
# inbound_verify package
|
||||||
1
inbound_verify/cli/__init__.py
Normal file
1
inbound_verify/cli/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
# inbound_verify.cli — 入口（交互菜单 / 服务）
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# main_router.py
|
# inbound_verify/cli/router.py
|
||||||
#
|
#
|
||||||
# 交互菜单模式入口（调试 / 人工操作）。核心 Playwright 管理、任务派发、心跳
|
# 交互菜单模式入口（调试 / 人工操作）。核心 Playwright 管理、任务派发、心跳
|
||||||
# 已抽到 runtime.py 共享；本文件只保留交互菜单与自动化测试。
|
# 已抽到 runtime.py 共享；本文件只保留交互菜单与自动化测试。
|
||||||
# 服务模式（常驻 + FastAPI 接收指令）见 server.py。
|
# 服务模式（常驻 + FastAPI 接收指令）见 cli/server.py。
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import queue
|
import queue
|
||||||
@@ -11,8 +11,8 @@ import time
|
|||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
from paths import CONFIG_PATH
|
from inbound_verify.paths import CONFIG_PATH
|
||||||
from runtime import (
|
from inbound_verify.runtime import (
|
||||||
APP_SITES,
|
APP_SITES,
|
||||||
HEARTBEAT_INTERVAL,
|
HEARTBEAT_INTERVAL,
|
||||||
dispatch_task,
|
dispatch_task,
|
||||||
@@ -20,22 +20,18 @@ from runtime import (
|
|||||||
run_heartbeat,
|
run_heartbeat,
|
||||||
)
|
)
|
||||||
|
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
|
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
|
||||||
import site_shunxin
|
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
|
||||||
import site_baishi
|
from inbound_verify import compare
|
||||||
import site_zto
|
|
||||||
import site_yunda
|
|
||||||
import site_anneng
|
|
||||||
import expected_undelivered
|
|
||||||
|
|
||||||
|
|
||||||
def run_undelivered_compare():
|
def run_undelivered_compare():
|
||||||
"""应到未到比对（全站点）：调用 expected_undelivered，读 downloads/ 下的应到/实到
|
"""应到未到比对（全站点）：调用 compare，读 downloads/ 下的应到/实到
|
||||||
数据，生成 output/应到未到数据.xlsx（汇总报表 + 各站明细）。"""
|
数据，生成 output/应到未到数据.xlsx（汇总报表 + 各站明细）。"""
|
||||||
print("\n▶ 开始执行【应到未到比对（全站点）】任务 ...")
|
print("\n▶ 开始执行【应到未到比对（全站点）】任务 ...")
|
||||||
expected_undelivered.main()
|
compare.main()
|
||||||
|
|
||||||
|
|
||||||
# ====================================================================
|
# ====================================================================
|
||||||
@@ -57,20 +53,20 @@ def run_automation_test(pages_map):
|
|||||||
# 不被模块内部"失败→重置→重试"机制掩盖。
|
# 不被模块内部"失败→重置→重试"机制掩盖。
|
||||||
flow_table = {
|
flow_table = {
|
||||||
"顺心": {
|
"顺心": {
|
||||||
"expected": ("应到", site_shunxin.shunxin_expected_download_impl),
|
"expected": ("应到", shunxin.shunxin_expected_download_impl),
|
||||||
"actual": ("实到", site_shunxin.shunxin_actual_download_impl),
|
"actual": ("实到", shunxin.shunxin_actual_download_impl),
|
||||||
},
|
},
|
||||||
"中通": {
|
"中通": {
|
||||||
"expected": ("应到", site_zto.zto_expected_download_impl),
|
"expected": ("应到", zto.zto_expected_download_impl),
|
||||||
"actual": ("实到", site_zto.zto_actual_download_impl),
|
"actual": ("实到", zto.zto_actual_download_impl),
|
||||||
},
|
},
|
||||||
"韵达": {
|
"韵达": {
|
||||||
"expected": ("应到", site_yunda.yunda_expected_download_impl),
|
"expected": ("应到", yunda.yunda_expected_download_impl),
|
||||||
"actual": ("实到", site_yunda.yunda_actual_download_impl),
|
"actual": ("实到", yunda.yunda_actual_download_impl),
|
||||||
},
|
},
|
||||||
"安能": {
|
"安能": {
|
||||||
"expected": ("应到", site_anneng.anneng_expected_download_impl),
|
"expected": ("应到", anneng.anneng_expected_download_impl),
|
||||||
"actual": ("实到", site_anneng.anneng_actual_download_impl),
|
"actual": ("实到", anneng.anneng_actual_download_impl),
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -83,7 +79,7 @@ def run_automation_test(pages_map):
|
|||||||
continue
|
continue
|
||||||
for sx_idx, sx_page in enumerate(pages_map["顺心"], start=1):
|
for sx_idx, sx_page in enumerate(pages_map["顺心"], start=1):
|
||||||
try:
|
try:
|
||||||
tag = site_shunxin.shunxin_belonging(sx_page)
|
tag = shunxin.shunxin_belonging(sx_page)
|
||||||
except Exception:
|
except Exception:
|
||||||
tag = f"账号{sx_idx}" # 读不到归属地时用序号占位，不阻断测试
|
tag = f"账号{sx_idx}" # 读不到归属地时用序号占位，不阻断测试
|
||||||
for flow_key in CROSS_TEST_SEQUENCE:
|
for flow_key in CROSS_TEST_SEQUENCE:
|
||||||
@@ -102,7 +98,7 @@ def run_automation_test(pages_map):
|
|||||||
(
|
(
|
||||||
"百世",
|
"百世",
|
||||||
"应到未到",
|
"应到未到",
|
||||||
site_baishi.baishi_download_undelivered_data_impl,
|
baishi.baishi_download_undelivered_data_impl,
|
||||||
pages_map["百世"],
|
pages_map["百世"],
|
||||||
"",
|
"",
|
||||||
)
|
)
|
||||||
@@ -343,5 +339,10 @@ def run_multi_site_daemon():
|
|||||||
print("程序已退出。")
|
print("程序已退出。")
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
def main():
|
||||||
|
"""交互菜单模式入口。"""
|
||||||
run_multi_site_daemon()
|
run_multi_site_daemon()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -26,9 +26,9 @@ from fastapi import FastAPI, HTTPException
|
|||||||
from fastapi.responses import FileResponse
|
from fastapi.responses import FileResponse
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, OUTPUT_DIR
|
from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
from runtime import (
|
from inbound_verify.runtime import (
|
||||||
HEARTBEAT_INTERVAL,
|
HEARTBEAT_INTERVAL,
|
||||||
TASK_HANDLERS,
|
TASK_HANDLERS,
|
||||||
dispatch_task,
|
dispatch_task,
|
||||||
@@ -64,7 +64,9 @@ def _worker_loop():
|
|||||||
# 【P1-2 重启自愈】worker 就绪后清理上轮遗留的 pending/running 僵尸任务
|
# 【P1-2 重启自愈】worker 就绪后清理上轮遗留的 pending/running 僵尸任务
|
||||||
cleaned = state_store.fail_stale_tasks()
|
cleaned = state_store.fail_stale_tasks()
|
||||||
if cleaned:
|
if cleaned:
|
||||||
print(f">> [worker] 自愈：清理 {cleaned} 条遗留任务（pending/running → failed）")
|
print(
|
||||||
|
f">> [worker] 自愈：清理 {cleaned} 条遗留任务（pending/running → failed）"
|
||||||
|
)
|
||||||
print(">> [worker] 各站就绪，开始接收任务 ...")
|
print(">> [worker] 各站就绪，开始接收任务 ...")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
worker_state["error"] = str(e)
|
worker_state["error"] = str(e)
|
||||||
@@ -161,7 +163,9 @@ def create_task(req: TaskRequest):
|
|||||||
"""提交任务 {site, kind} → 入队，返回 task_id。"""
|
"""提交任务 {site, kind} → 入队，返回 task_id。"""
|
||||||
# 【P0】后端未就绪时直接拒绝，避免任务在 worker 启动前入队卡死
|
# 【P0】后端未就绪时直接拒绝，避免任务在 worker 启动前入队卡死
|
||||||
if not worker_state["ready"]:
|
if not worker_state["ready"]:
|
||||||
raise HTTPException(status_code=409, detail="后端尚未就绪，请等待各站点登录完成后再操作")
|
raise HTTPException(
|
||||||
|
status_code=409, detail="后端尚未就绪，请等待各站点登录完成后再操作"
|
||||||
|
)
|
||||||
if (req.site, req.kind) not in TASK_HANDLERS:
|
if (req.site, req.kind) not in TASK_HANDLERS:
|
||||||
raise HTTPException(status_code=400, detail=f"无效任务: {req.site}/{req.kind}")
|
raise HTTPException(status_code=400, detail=f"无效任务: {req.site}/{req.kind}")
|
||||||
task_id = state_store.create_task(req.site, req.kind)
|
task_id = state_store.create_task(req.site, req.kind)
|
||||||
@@ -287,5 +291,10 @@ def download_data(filename: str):
|
|||||||
return FileResponse(path, filename=filename)
|
return FileResponse(path, filename=filename)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""服务模式入口。传字符串导入路径（规范写法；不开 reload/workers 时进程内 import，行为等价）。"""
|
||||||
|
uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
main()
|
||||||
@@ -40,95 +40,22 @@ from openpyxl.chart import BarChart, Reference
|
|||||||
from openpyxl.worksheet.page import PageMargins
|
from openpyxl.worksheet.page import PageMargins
|
||||||
from openpyxl.worksheet.properties import PageSetupProperties
|
from openpyxl.worksheet.properties import PageSetupProperties
|
||||||
|
|
||||||
BASE = os.path.dirname(os.path.abspath(__file__))
|
from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR
|
||||||
DOWNLOADS = os.path.join(BASE, "downloads")
|
from inbound_verify.domain import (
|
||||||
OUTPUT = os.path.join(BASE, "output")
|
ALL_REPORT_SITES,
|
||||||
OUTFILE = os.path.join(OUTPUT, "应到未到数据.xlsx")
|
BAISHI_COLUMNS,
|
||||||
|
BAISHI_FILE,
|
||||||
|
SITE_UNDELIVERED_FILE,
|
||||||
|
STATIONS,
|
||||||
|
_site_cfg,
|
||||||
|
arrived_pieces_by_cols,
|
||||||
|
arrived_pieces_zhongtong,
|
||||||
|
)
|
||||||
|
|
||||||
# 汇总报表覆盖的全部站点（4 站在前、百世在末；汇总页图表只取 4 站）
|
# 比对报表输出文件（路径锚定统一走 paths.py）。
|
||||||
ALL_REPORT_SITES = ["顺心", "中通", "韵达", "安能", "百世"]
|
OUTFILE = os.path.join(OUTPUT_DIR, "应到未到数据.xlsx")
|
||||||
# 4 站单站未到明细文件名（百世未到文件由站点直接产出，名为 BAISHI_FILE）
|
|
||||||
SITE_UNDELIVERED_FILE = "{name}-未到数据.xlsx"
|
|
||||||
BAISHI_FILE = "百世-应到未到货物数据.xlsx"
|
|
||||||
BAISHI_COLUMNS = ["类型", "子单号", "运单号", "最新扫描记录"]
|
|
||||||
|
|
||||||
|
# 站点 / 文件名 / 列映射配置（ALL_REPORT_SITES / STATIONS / _site_cfg / BAISHI_FILE 等）见 domain.py。
|
||||||
# ============================ 比对逻辑 ============================
|
|
||||||
|
|
||||||
|
|
||||||
def arrived_pieces_zhongtong(df):
|
|
||||||
"""中通：实到「运单号」为复合串（H + 运单号(12) + 总数(4) + 顺序(4)）。
|
|
||||||
基号 = v[:-8]（与应到表运单号对齐），单件 = 整串（每串即一件）。"""
|
|
||||||
res = defaultdict(set)
|
|
||||||
for v in df["运单号"]:
|
|
||||||
v = str(v).strip()
|
|
||||||
if len(v) > 8 and v[-4:].isdigit():
|
|
||||||
res[v[:-8]].add(v) # 以完整复合串作为“已到单号”存入
|
|
||||||
return res
|
|
||||||
|
|
||||||
|
|
||||||
def arrived_pieces_by_cols(wb_col, piece_col):
|
|
||||||
"""顺心 / 韵达 / 安能：按干净运单列分组，单件 = 子单号 / 扫描单号。
|
|
||||||
wb_col：实到表中与应到运单号对齐的干净列
|
|
||||||
（顺心=运单号 / 韵达=主单号 / 安能=所属单号）
|
|
||||||
piece_col：实到表中每件货物的单号列（子单号 / 扫描单号）"""
|
|
||||||
def parse(df):
|
|
||||||
res = defaultdict(set)
|
|
||||||
for m, s in zip(df[wb_col], df[piece_col]):
|
|
||||||
m, s = str(m).strip(), str(s).strip()
|
|
||||||
if m and s:
|
|
||||||
res[m].add(s)
|
|
||||||
return res
|
|
||||||
return parse
|
|
||||||
|
|
||||||
|
|
||||||
STATIONS = [
|
|
||||||
{
|
|
||||||
"name": "中通",
|
|
||||||
"exp": "中通-应到货物数据.xlsx",
|
|
||||||
"act": "中通-实到货物数据.xlsx",
|
|
||||||
"exp_qty": "交接件数", # 应到件数口径：交接件数（非录单件数）
|
|
||||||
"exp_wb": "运单号", # 应到表运单号列（兼作去重键）
|
|
||||||
"exp_jd": "交接单号", # 未到数据需展示的交接单号
|
|
||||||
"arrived_pieces": arrived_pieces_zhongtong,
|
|
||||||
"columns": ["交接单号", "运单号", "总件数"],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "顺心",
|
|
||||||
"exp": "顺心-应到货物数据.xlsx",
|
|
||||||
"act": "顺心-实到货物数据.xlsx",
|
|
||||||
"exp_qty": "交接件数",
|
|
||||||
"exp_wb": "运单号",
|
|
||||||
"exp_jd": "交接单号",
|
|
||||||
"arrived_pieces": arrived_pieces_by_cols("运单号", "子单号"),
|
|
||||||
"columns": ["交接单号", "运单号", "总件数"],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "韵达",
|
|
||||||
"exp": "韵达-应到货物数据.xlsx",
|
|
||||||
"act": "韵达-实到货物数据.xlsx",
|
|
||||||
"exp_qty": "交接件数",
|
|
||||||
"exp_wb": "运单号",
|
|
||||||
"exp_jd": "交接单号",
|
|
||||||
"arrived_pieces": arrived_pieces_by_cols("主单号", "子单号"),
|
|
||||||
"columns": ["交接单号", "运单号", "总件数"],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "安能",
|
|
||||||
"exp": "安能-应到货物数据.xlsx",
|
|
||||||
"act": "安能-实到货物数据.xlsx",
|
|
||||||
"exp_qty": "交接件数",
|
|
||||||
"exp_wb": "运单号",
|
|
||||||
"exp_jd": "交接单号",
|
|
||||||
"arrived_pieces": arrived_pieces_by_cols("所属单号", "扫描单号"),
|
|
||||||
"columns": ["交接单号", "运单号", "总件数"],
|
|
||||||
},
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _site_cfg(name):
|
|
||||||
"""按名称取 4 站配置（百世不在 STATIONS，返回 None）。"""
|
|
||||||
return next((c for c in STATIONS if c["name"] == name), None)
|
|
||||||
|
|
||||||
|
|
||||||
def process(name):
|
def process(name):
|
||||||
@@ -139,8 +66,8 @@ def process(name):
|
|||||||
cfg = _site_cfg(name)
|
cfg = _site_cfg(name)
|
||||||
if cfg is None:
|
if cfg is None:
|
||||||
return None
|
return None
|
||||||
exp_path = os.path.join(DOWNLOADS, cfg["exp"])
|
exp_path = os.path.join(DOWNLOAD_DIR, cfg["exp"])
|
||||||
act_path = os.path.join(DOWNLOADS, cfg["act"])
|
act_path = os.path.join(DOWNLOAD_DIR, cfg["act"])
|
||||||
if not os.path.exists(exp_path) or not os.path.exists(act_path):
|
if not os.path.exists(exp_path) or not os.path.exists(act_path):
|
||||||
print(f"[跳过] {cfg['name']}：downloads 下缺少 {cfg['exp']} 或 {cfg['act']}")
|
print(f"[跳过] {cfg['name']}：downloads 下缺少 {cfg['exp']} 或 {cfg['act']}")
|
||||||
return None
|
return None
|
||||||
@@ -291,7 +218,7 @@ def write_station(ws, columns, rows):
|
|||||||
def process_baishi():
|
def process_baishi():
|
||||||
"""百世：读站点直供的未到明细，返回 (columns, rows, stats)；文件缺失返回 None。
|
"""百世：读站点直供的未到明细，返回 (columns, rows, stats)；文件缺失返回 None。
|
||||||
百世文件本身即未到结果（无应到/已到基数），统计只能给出未到件数。"""
|
百世文件本身即未到结果（无应到/已到基数），统计只能给出未到件数。"""
|
||||||
path = os.path.join(DOWNLOADS, BAISHI_FILE)
|
path = os.path.join(DOWNLOAD_DIR, BAISHI_FILE)
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
return None
|
return None
|
||||||
df = pd.read_excel(path, dtype=str).fillna("")
|
df = pd.read_excel(path, dtype=str).fillna("")
|
||||||
@@ -301,7 +228,9 @@ def process_baishi():
|
|||||||
# 应到/实到基数取自「扫描综合查询」应扫/已扫（到/接件扫描→当日），
|
# 应到/实到基数取自「扫描综合查询」应扫/已扫（到/接件扫描→当日），
|
||||||
# 由 baishi_download_undelivered_data_impl 在同次导航里抓取并落 site_settings。
|
# 由 baishi_download_undelivered_data_impl 在同次导航里抓取并落 site_settings。
|
||||||
# 未抓取过则 get_setting 返回 "" → 视为无基数（报表显示「—」）。
|
# 未抓取过则 get_setting 返回 "" → 视为无基数（报表显示「—」）。
|
||||||
import state_store # 与 _read_business_dates 一致：比对模块纯离线，懒加载
|
from inbound_verify import (
|
||||||
|
state_store,
|
||||||
|
) # 与 _read_business_dates 一致：比对模块纯离线，懒加载
|
||||||
|
|
||||||
def _to_int(v):
|
def _to_int(v):
|
||||||
v = (v or "").strip().replace(",", "")
|
v = (v or "").strip().replace(",", "")
|
||||||
@@ -329,7 +258,7 @@ def process_baishi():
|
|||||||
def write_site_file(name):
|
def write_site_file(name):
|
||||||
"""4 站：把该站未到明细写到 downloads/<站>-未到数据.xlsx。
|
"""4 站：把该站未到明细写到 downloads/<站>-未到数据.xlsx。
|
||||||
应到/实到缺（process 返回 None）→ 删旧文件、返回 False；成功返回 True。"""
|
应到/实到缺（process 返回 None）→ 删旧文件、返回 False；成功返回 True。"""
|
||||||
path = os.path.join(DOWNLOADS, SITE_UNDELIVERED_FILE.format(name=name))
|
path = os.path.join(DOWNLOAD_DIR, SITE_UNDELIVERED_FILE.format(name=name))
|
||||||
out = process(name)
|
out = process(name)
|
||||||
if out is None:
|
if out is None:
|
||||||
if os.path.exists(path):
|
if os.path.exists(path):
|
||||||
@@ -348,7 +277,7 @@ def _read_business_dates(include):
|
|||||||
"""从状态库读各站业务日期（dispatch 下载成功时快照写入），供报告「数据日期」列。
|
"""从状态库读各站业务日期（dispatch 下载成功时快照写入），供报告「数据日期」列。
|
||||||
4 站取 expected_business_date（报告按应到口径）；百世取 undelivered_business_date。
|
4 站取 expected_business_date（报告按应到口径）；百世取 undelivered_business_date。
|
||||||
从未下过的站返回空串（诚实留空，不反推）。"""
|
从未下过的站返回空串（诚实留空，不反推）。"""
|
||||||
import state_store # lazy import：比对模块本身保持纯离线
|
from inbound_verify import state_store # lazy import：比对模块本身保持纯离线
|
||||||
|
|
||||||
status = state_store.get_all_status()
|
status = state_store.get_all_status()
|
||||||
dates = {}
|
dates = {}
|
||||||
@@ -365,7 +294,7 @@ def build_full_report(include, dates=None):
|
|||||||
"""生成全站汇总报表 output/应到未到数据.xlsx。
|
"""生成全站汇总报表 output/应到未到数据.xlsx。
|
||||||
include: 本次成功的站点集合；未成功站点在汇总里保留行、无数据（不影响他站）。
|
include: 本次成功的站点集合；未成功站点在汇总里保留行、无数据（不影响他站）。
|
||||||
返回 {站点: 未到件或None} 供日志。"""
|
返回 {站点: 未到件或None} 供日志。"""
|
||||||
os.makedirs(OUTPUT, exist_ok=True)
|
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||||
wb = Workbook()
|
wb = Workbook()
|
||||||
wb.remove(wb.active)
|
wb.remove(wb.active)
|
||||||
summary_ws = wb.create_sheet("汇总报表") # 首页占位
|
summary_ws = wb.create_sheet("汇总报表") # 首页占位
|
||||||
@@ -534,8 +463,16 @@ def build_summary(ws, results, generated_at, dates=None):
|
|||||||
# 百世：未到明细已知；若已抓取应到/实到基数（扫描综合查询应扫/已扫）则填真实值
|
# 百世：未到明细已知；若已抓取应到/实到基数（扫描综合查询应扫/已扫）则填真实值
|
||||||
if s["应到件"] is not None and s["已到件"] is not None:
|
if s["应到件"] is not None and s["已到件"] is not None:
|
||||||
srate = (s["未到件"] / s["应到件"]) if s["应到件"] else 0
|
srate = (s["未到件"] / s["应到件"]) if s["应到件"] else 0
|
||||||
vals = [name, s["运单数"], s["应到件"], s["已到件"],
|
vals = [
|
||||||
s["未到件"], srate, "—", "—"]
|
name,
|
||||||
|
s["运单数"],
|
||||||
|
s["应到件"],
|
||||||
|
s["已到件"],
|
||||||
|
s["未到件"],
|
||||||
|
srate,
|
||||||
|
"—",
|
||||||
|
"—",
|
||||||
|
]
|
||||||
else:
|
else:
|
||||||
vals = [name, s["运单数"], "—", "—", s["未到件"], "—", "—", "—"]
|
vals = [name, s["运单数"], "—", "—", s["未到件"], "—", "—", "—"]
|
||||||
else:
|
else:
|
||||||
@@ -564,7 +501,12 @@ def build_summary(ws, results, generated_at, dates=None):
|
|||||||
cell.fill = PatternFill("solid", fgColor=ZEBRA)
|
cell.fill = PatternFill("solid", fgColor=ZEBRA)
|
||||||
if isinstance(v, (int, float)):
|
if isinstance(v, (int, float)):
|
||||||
cell.number_format = "0.0%" if i == 5 else "#,##0"
|
cell.number_format = "0.0%" if i == 5 else "#,##0"
|
||||||
if i == 5 and s is not None and isinstance(v, (int, float)) and not isinstance(v, bool):
|
if (
|
||||||
|
i == 5
|
||||||
|
and s is not None
|
||||||
|
and isinstance(v, (int, float))
|
||||||
|
and not isinstance(v, bool)
|
||||||
|
):
|
||||||
cell.fill = PatternFill("solid", fgColor=heat(srate))
|
cell.fill = PatternFill("solid", fgColor=heat(srate))
|
||||||
ws.row_dimensions[r].height = 19
|
ws.row_dimensions[r].height = 19
|
||||||
r += 1
|
r += 1
|
||||||
@@ -647,14 +589,14 @@ def main():
|
|||||||
include = set()
|
include = set()
|
||||||
for name in ALL_REPORT_SITES:
|
for name in ALL_REPORT_SITES:
|
||||||
if name == "百世":
|
if name == "百世":
|
||||||
if os.path.exists(os.path.join(DOWNLOADS, BAISHI_FILE)):
|
if os.path.exists(os.path.join(DOWNLOAD_DIR, BAISHI_FILE)):
|
||||||
include.add(name)
|
include.add(name)
|
||||||
else:
|
else:
|
||||||
cfg = _site_cfg(name)
|
cfg = _site_cfg(name)
|
||||||
if (
|
if (
|
||||||
cfg
|
cfg
|
||||||
and os.path.exists(os.path.join(DOWNLOADS, cfg["exp"]))
|
and os.path.exists(os.path.join(DOWNLOAD_DIR, cfg["exp"]))
|
||||||
and os.path.exists(os.path.join(DOWNLOADS, cfg["act"]))
|
and os.path.exists(os.path.join(DOWNLOAD_DIR, cfg["act"]))
|
||||||
):
|
):
|
||||||
include.add(name)
|
include.add(name)
|
||||||
if not include:
|
if not include:
|
||||||
92
inbound_verify/domain.py
Normal file
92
inbound_verify/domain.py
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""domain.py — 站点 / 文件名 / 列映射的共享配置（单一来源）。
|
||||||
|
|
||||||
|
比对（compare）与入库（store）都依赖这套配置；抽出独立 leaf 模块，
|
||||||
|
让 store 不必为读配置而依赖整个比对引擎。纯数据，无 state_store / 文件 IO 依赖。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from collections import defaultdict
|
||||||
|
|
||||||
|
# 汇总报表覆盖的全部站点（4 站在前、百世在末；汇总页图表只取 4 站）
|
||||||
|
ALL_REPORT_SITES = ["顺心", "中通", "韵达", "安能", "百世"]
|
||||||
|
# 4 站单站未到明细文件名（百世未到文件由站点直接产出，名为 BAISHI_FILE）
|
||||||
|
SITE_UNDELIVERED_FILE = "{name}-未到数据.xlsx"
|
||||||
|
BAISHI_FILE = "百世-应到未到货物数据.xlsx"
|
||||||
|
BAISHI_COLUMNS = ["类型", "子单号", "运单号", "最新扫描记录"]
|
||||||
|
|
||||||
|
|
||||||
|
def arrived_pieces_zhongtong(df):
|
||||||
|
"""中通：实到「运单号」为复合串（H + 运单号(12) + 总数(4) + 顺序(4)）。
|
||||||
|
基号 = v[:-8]（与应到表运单号对齐），单件 = 整串（每串即一件）。"""
|
||||||
|
res = defaultdict(set)
|
||||||
|
for v in df["运单号"]:
|
||||||
|
v = str(v).strip()
|
||||||
|
if len(v) > 8 and v[-4:].isdigit():
|
||||||
|
res[v[:-8]].add(v) # 以完整复合串作为“已到单号”存入
|
||||||
|
return res
|
||||||
|
|
||||||
|
|
||||||
|
def arrived_pieces_by_cols(wb_col, piece_col):
|
||||||
|
"""顺心 / 韵达 / 安能：按干净运单列分组，单件 = 子单号 / 扫描单号。
|
||||||
|
wb_col：实到表中与应到运单号对齐的干净列
|
||||||
|
（顺心=运单号 / 韵达=主单号 / 安能=所属单号）
|
||||||
|
piece_col：实到表中每件货物的单号列（子单号 / 扫描单号）"""
|
||||||
|
|
||||||
|
def parse(df):
|
||||||
|
res = defaultdict(set)
|
||||||
|
for m, s in zip(df[wb_col], df[piece_col]):
|
||||||
|
m, s = str(m).strip(), str(s).strip()
|
||||||
|
if m and s:
|
||||||
|
res[m].add(s)
|
||||||
|
return res
|
||||||
|
|
||||||
|
return parse
|
||||||
|
|
||||||
|
|
||||||
|
STATIONS = [
|
||||||
|
{
|
||||||
|
"name": "中通",
|
||||||
|
"exp": "中通-应到货物数据.xlsx",
|
||||||
|
"act": "中通-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数", # 应到件数口径：交接件数（非录单件数）
|
||||||
|
"exp_wb": "运单号", # 应到表运单号列（兼作去重键）
|
||||||
|
"exp_jd": "交接单号", # 未到数据需展示的交接单号
|
||||||
|
"arrived_pieces": arrived_pieces_zhongtong,
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "顺心",
|
||||||
|
"exp": "顺心-应到货物数据.xlsx",
|
||||||
|
"act": "顺心-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("运单号", "子单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "韵达",
|
||||||
|
"exp": "韵达-应到货物数据.xlsx",
|
||||||
|
"act": "韵达-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("主单号", "子单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "安能",
|
||||||
|
"exp": "安能-应到货物数据.xlsx",
|
||||||
|
"act": "安能-实到货物数据.xlsx",
|
||||||
|
"exp_qty": "交接件数",
|
||||||
|
"exp_wb": "运单号",
|
||||||
|
"exp_jd": "交接单号",
|
||||||
|
"arrived_pieces": arrived_pieces_by_cols("所属单号", "扫描单号"),
|
||||||
|
"columns": ["交接单号", "运单号", "总件数"],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _site_cfg(name):
|
||||||
|
"""按名称取 4 站配置（百世不在 STATIONS，返回 None）。"""
|
||||||
|
return next((c for c in STATIONS if c["name"] == name), None)
|
||||||
@@ -6,7 +6,8 @@
|
|||||||
import os
|
import os
|
||||||
|
|
||||||
# 项目根目录（以本文件所在位置为基准，与从哪个目录启动脚本无关）
|
# 项目根目录（以本文件所在位置为基准，与从哪个目录启动脚本无关）
|
||||||
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
|
# __file__ = <root>/inbound_verify/paths.py → 上两级 = 项目根
|
||||||
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
# 统一的下载 / 输出目录
|
# 统一的下载 / 输出目录
|
||||||
DOWNLOAD_DIR = os.path.join(BASE_DIR, "downloads")
|
DOWNLOAD_DIR = os.path.join(BASE_DIR, "downloads")
|
||||||
@@ -2,8 +2,8 @@
|
|||||||
# 阶段1：服务化共享核心。把"启动浏览器 + 各站就绪 + 弹窗 + 心跳初值"
|
# 阶段1：服务化共享核心。把"启动浏览器 + 各站就绪 + 弹窗 + 心跳初值"
|
||||||
# (launch_and_prepare)、"执行一条任务"(dispatch_task)、"一轮心跳"(run_heartbeat)
|
# (launch_and_prepare)、"执行一条任务"(dispatch_task)、"一轮心跳"(run_heartbeat)
|
||||||
# 抽出来，供
|
# 抽出来，供
|
||||||
# - main_router.py（交互菜单模式：调试 / 人工操作）
|
# - cli/router.py（交互菜单模式：调试 / 人工操作）
|
||||||
# - server.py（FastAPI 服务模式：常驻 + 接收 API 指令）
|
# - cli/server.py（FastAPI 服务模式：常驻 + 接收 API 指令）
|
||||||
# 共同复用，避免两处重复维护。
|
# 共同复用，避免两处重复维护。
|
||||||
#
|
#
|
||||||
# 线程模型：launch_and_prepare 内 sync_playwright().start() 必须在"持有 Playwright 的
|
# 线程模型：launch_and_prepare 内 sync_playwright().start() 必须在"持有 Playwright 的
|
||||||
@@ -21,22 +21,19 @@ from datetime import datetime, timedelta
|
|||||||
import yaml
|
import yaml
|
||||||
from playwright.sync_api import sync_playwright
|
from playwright.sync_api import sync_playwright
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
|
from inbound_verify.domain import SITE_UNDELIVERED_FILE
|
||||||
|
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
import site_shunxin
|
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
|
||||||
import site_baishi
|
from inbound_verify import compare # dispatch 的 compare 任务用
|
||||||
import site_zto
|
|
||||||
import site_yunda
|
|
||||||
import site_anneng
|
|
||||||
import expected_undelivered # dispatch 的 compare 任务用
|
|
||||||
|
|
||||||
# 各网页站点首页 URL（单一来源，取自各站点模块 HOME_URL）
|
# 各网页站点首页 URL（单一来源，取自各站点模块 HOME_URL）
|
||||||
SITES_CONFIG = {
|
SITES_CONFIG = {
|
||||||
"顺心": site_shunxin.HOME_URL,
|
"顺心": shunxin.HOME_URL,
|
||||||
"百世": site_baishi.HOME_URL,
|
"百世": baishi.HOME_URL,
|
||||||
"中通": site_zto.HOME_URL,
|
"中通": zto.HOME_URL,
|
||||||
"韵达": site_yunda.HOME_URL,
|
"韵达": yunda.HOME_URL,
|
||||||
}
|
}
|
||||||
|
|
||||||
# 站点就绪特征：登录成功进入工作台后的标志性控件
|
# 站点就绪特征：登录成功进入工作台后的标志性控件
|
||||||
@@ -114,8 +111,10 @@ def launch_anneng(app_path):
|
|||||||
anneng_env.pop("NODE_OPTIONS", None)
|
anneng_env.pop("NODE_OPTIONS", None)
|
||||||
port = _find_free_port()
|
port = _find_free_port()
|
||||||
print(f">> 以调试模式启动【安能】应用（端口 {port}）：{app_path}")
|
print(f">> 以调试模式启动【安能】应用（端口 {port}）：{app_path}")
|
||||||
proc = subprocess.Popen([app_path, f"--remote-debugging-port={port}"], env=anneng_env)
|
proc = subprocess.Popen(
|
||||||
site_anneng.set_cdp_port(port)
|
[app_path, f"--remote-debugging-port={port}"], env=anneng_env
|
||||||
|
)
|
||||||
|
anneng.set_cdp_port(port)
|
||||||
if not _wait_cdp_up(port):
|
if not _wait_cdp_up(port):
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"安能应用调试端口 {port} 未就绪——可能应用已在运行（单实例），"
|
f"安能应用调试端口 {port} 未就绪——可能应用已在运行（单实例），"
|
||||||
@@ -136,7 +135,7 @@ def probe_site_login(site_name, pages_map):
|
|||||||
if site_name not in pages_map:
|
if site_name not in pages_map:
|
||||||
return False
|
return False
|
||||||
if site_name == "安能":
|
if site_name == "安能":
|
||||||
return site_anneng.anneng_ready()
|
return anneng.anneng_ready()
|
||||||
if site_name == "顺心":
|
if site_name == "顺心":
|
||||||
return all(
|
return all(
|
||||||
pg.locator(READY_SELECTORS["顺心"]).is_visible(timeout=500)
|
pg.locator(READY_SELECTORS["顺心"]).is_visible(timeout=500)
|
||||||
@@ -300,21 +299,39 @@ def launch_and_prepare(debug_mode=False, debug_target=""):
|
|||||||
print("\n====================================================")
|
print("\n====================================================")
|
||||||
print("【启动】正在打开各站点页面...")
|
print("【启动】正在打开各站点页面...")
|
||||||
print("====================================================")
|
print("====================================================")
|
||||||
|
|
||||||
|
def _open_page(label, attempts=3):
|
||||||
|
"""开一个页面并 goto(url, domcontentloaded)；容忍瞬时 DNS/超时抖动重试，全失败才抛。
|
||||||
|
domcontentloaded：DOM 就绪即返回，不等慢资源(广告/图片)的 load 事件，
|
||||||
|
避免某站 load 超 30s / 瞬时 DNS 失败导致整个 worker 启动失败。就绪轮询自会判登录态。"""
|
||||||
|
last = None
|
||||||
|
for i in range(1, attempts + 1):
|
||||||
|
pg = context.new_page()
|
||||||
|
try:
|
||||||
|
pg.goto(url, wait_until="domcontentloaded")
|
||||||
|
return pg
|
||||||
|
except Exception as e:
|
||||||
|
last = e
|
||||||
|
try:
|
||||||
|
pg.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
print(f"⚠️ 打开【{label}】第 {i}/{attempts} 次失败: {e}")
|
||||||
|
if i < attempts:
|
||||||
|
time.sleep(2)
|
||||||
|
raise RuntimeError(f"打开【{label}】连续 {attempts} 次失败: {last}")
|
||||||
|
|
||||||
for site_name, url in active_sites.items():
|
for site_name, url in active_sites.items():
|
||||||
if site_name == "顺心":
|
if site_name == "顺心":
|
||||||
# 顺心：两个归属地账号在同一窗口各开一个标签页
|
# 顺心：两个归属地账号在同一窗口各开一个标签页
|
||||||
sx_pages = []
|
sx_pages = []
|
||||||
for acct in range(1, 3):
|
for acct in range(1, 3):
|
||||||
print(f">> 正在启动【顺心】账号{acct}标签页: {url}")
|
print(f">> 正在启动【顺心】账号{acct}标签页: {url}")
|
||||||
sx_page = context.new_page()
|
sx_pages.append(_open_page(f"顺心账号{acct}"))
|
||||||
sx_page.goto(url)
|
|
||||||
sx_pages.append(sx_page)
|
|
||||||
pages_map["顺心"] = sx_pages
|
pages_map["顺心"] = sx_pages
|
||||||
else:
|
else:
|
||||||
print(f">> 正在启动【{site_name}】页面: {url}")
|
print(f">> 正在启动【{site_name}】页面: {url}")
|
||||||
page = context.new_page()
|
pages_map[site_name] = _open_page(site_name)
|
||||||
page.goto(url)
|
|
||||||
pages_map[site_name] = page
|
|
||||||
|
|
||||||
# 4. 安能 Electron
|
# 4. 安能 Electron
|
||||||
anneng_proc = None
|
anneng_proc = None
|
||||||
@@ -333,7 +350,7 @@ def launch_and_prepare(debug_mode=False, debug_target=""):
|
|||||||
if "韵达" in pages_map:
|
if "韵达" in pages_map:
|
||||||
try:
|
try:
|
||||||
pages_map["韵达"].bring_to_front()
|
pages_map["韵达"].bring_to_front()
|
||||||
site_yunda.yunda_login(pages_map["韵达"])
|
yunda.yunda_login(pages_map["韵达"])
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f" ⚠️ 韵达前置自动登录模块发生波动: {e}")
|
print(f" ⚠️ 韵达前置自动登录模块发生波动: {e}")
|
||||||
|
|
||||||
@@ -403,9 +420,9 @@ def _dismiss_initial_popups(pages_map):
|
|||||||
bs_page = pages_map["百世"]
|
bs_page = pages_map["百世"]
|
||||||
bs_page.bring_to_front()
|
bs_page.bring_to_front()
|
||||||
print(">> 正在处理【百世】初始弹窗（阅读消息 / 配置检查 / 优惠券广告）...")
|
print(">> 正在处理【百世】初始弹窗（阅读消息 / 配置检查 / 优惠券广告）...")
|
||||||
# 委托给 site_baishi 的专用清理：优惠券广告是全屏居中 modal，
|
# 委托给 baishi 的专用清理：优惠券广告是全屏居中 modal，
|
||||||
# 关闭键为 .ant-modal-close（纯图标无文字），必须点它才能真正关掉。
|
# 关闭键为 .ant-modal-close（纯图标无文字），必须点它才能真正关掉。
|
||||||
site_baishi.dismiss_baishi_popups(bs_page)
|
baishi.dismiss_baishi_popups(bs_page)
|
||||||
print(" ✅ 【百世】初始弹窗处理完成。")
|
print(" ✅ 【百世】初始弹窗处理完成。")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
@@ -414,7 +431,7 @@ def _dismiss_initial_popups(pages_map):
|
|||||||
yd_page = pages_map["韵达"]
|
yd_page = pages_map["韵达"]
|
||||||
yd_page.bring_to_front()
|
yd_page.bring_to_front()
|
||||||
print(">> 正在检查【韵达】音频设备授权提示...")
|
print(">> 正在检查【韵达】音频设备授权提示...")
|
||||||
site_yunda.dismiss_audio_prompt(yd_page)
|
yunda.dismiss_audio_prompt(yd_page)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -445,10 +462,8 @@ def _site_undelivered_handler(site):
|
|||||||
(TASK_HANDLERS[(site, "actual")](ctx) is not False) if exp_ok else False
|
(TASK_HANDLERS[(site, "actual")](ctx) is not False) if exp_ok else False
|
||||||
)
|
)
|
||||||
if exp_ok and act_ok:
|
if exp_ok and act_ok:
|
||||||
return expected_undelivered.write_site_file(site)
|
return compare.write_site_file(site)
|
||||||
stale = os.path.join(
|
stale = os.path.join(DOWNLOAD_DIR, SITE_UNDELIVERED_FILE.format(name=site))
|
||||||
DOWNLOAD_DIR, expected_undelivered.SITE_UNDELIVERED_FILE.format(name=site)
|
|
||||||
)
|
|
||||||
if os.path.exists(stale):
|
if os.path.exists(stale):
|
||||||
os.remove(stale)
|
os.remove(stale)
|
||||||
return False
|
return False
|
||||||
@@ -460,22 +475,22 @@ def _site_undelivered_handler(site):
|
|||||||
|
|
||||||
|
|
||||||
TASK_HANDLERS = {
|
TASK_HANDLERS = {
|
||||||
("顺心", "expected"): _web_handler("顺心", site_shunxin.shunxin_expected_download),
|
("顺心", "expected"): _web_handler("顺心", shunxin.shunxin_expected_download),
|
||||||
("顺心", "actual"): _web_handler("顺心", site_shunxin.shunxin_actual_download),
|
("顺心", "actual"): _web_handler("顺心", shunxin.shunxin_actual_download),
|
||||||
("顺心", "undelivered"): _site_undelivered_handler("顺心"),
|
("顺心", "undelivered"): _site_undelivered_handler("顺心"),
|
||||||
("百世", "undelivered"): _web_handler(
|
("百世", "undelivered"): _web_handler(
|
||||||
"百世", site_baishi.baishi_download_undelivered_data
|
"百世", baishi.baishi_download_undelivered_data
|
||||||
),
|
),
|
||||||
("中通", "expected"): _web_handler("中通", site_zto.zto_expected_download),
|
("中通", "expected"): _web_handler("中通", zto.zto_expected_download),
|
||||||
("中通", "actual"): _web_handler("中通", site_zto.zto_actual_download),
|
("中通", "actual"): _web_handler("中通", zto.zto_actual_download),
|
||||||
("中通", "undelivered"): _site_undelivered_handler("中通"),
|
("中通", "undelivered"): _site_undelivered_handler("中通"),
|
||||||
("韵达", "expected"): _web_handler("韵达", site_yunda.yunda_expected_download),
|
("韵达", "expected"): _web_handler("韵达", yunda.yunda_expected_download),
|
||||||
("韵达", "actual"): _web_handler("韵达", site_yunda.yunda_actual_download),
|
("韵达", "actual"): _web_handler("韵达", yunda.yunda_actual_download),
|
||||||
("韵达", "undelivered"): _site_undelivered_handler("韵达"),
|
("韵达", "undelivered"): _site_undelivered_handler("韵达"),
|
||||||
("安能", "expected"): lambda ctx: site_anneng.anneng_expected_download(),
|
("安能", "expected"): lambda ctx: anneng.anneng_expected_download(),
|
||||||
("安能", "actual"): lambda ctx: site_anneng.anneng_actual_download(),
|
("安能", "actual"): lambda ctx: anneng.anneng_actual_download(),
|
||||||
("安能", "undelivered"): _site_undelivered_handler("安能"),
|
("安能", "undelivered"): _site_undelivered_handler("安能"),
|
||||||
("__compare__", "compare"): lambda ctx: (expected_undelivered.main() or True),
|
("__compare__", "compare"): lambda ctx: (compare.main() or True),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
1
inbound_verify/sites/__init__.py
Normal file
1
inbound_verify/sites/__init__.py
Normal file
@@ -0,0 +1 @@
|
|||||||
|
# inbound_verify.sites — 各承运商站点驱动
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
# site_anneng.py
|
# sites/anneng.py
|
||||||
#
|
#
|
||||||
# 安能全网门户（Electron 应用）—— 应到货物数据下载。
|
# 安能全网门户（Electron 应用）—— 应到货物数据下载。
|
||||||
#
|
#
|
||||||
# 与其他站点（网页、由 main_router 用 Playwright 驱动）不同，安能是一个 Electron
|
# 与其他站点（网页、由 runtime 用 Playwright 驱动）不同，安能是一个 Electron
|
||||||
# 桌面应用：由 main_router 以调试模式启动（动态空闲端口，经 set_cdp_port 告知本模块），
|
# 桌面应用：由 runtime 以调试模式启动（动态空闲端口，经 set_cdp_port 告知本模块），
|
||||||
# 本模块通过该端口的 CDP 驱动它；「进站交接单查询」「导出下载」等右侧 tab 是
|
# 本模块通过该端口的 CDP 驱动它；「进站交接单查询」「导出下载」等右侧 tab 是
|
||||||
# **独立 webContents**（远程网页），在 /json 里是独立目标。
|
# **独立 webContents**（远程网页），在 /json 里是独立目标。
|
||||||
# 也可脱离 main_router 独立运行（此时用默认端口 9222，需已自行启动应用）：
|
# 也可脱离 runtime 独立运行（此时用默认端口 9222，需已自行启动应用）：
|
||||||
# .venv/Scripts/python.exe site_anneng.py
|
# python -m inbound_verify.sites.anneng expected # 或 actual
|
||||||
#
|
#
|
||||||
# 应到 = 运单信息导出（每张交接单下应到的运单明细）。整体流程：
|
# 应到 = 运单信息导出（每张交接单下应到的运单明细）。整体流程：
|
||||||
# 主页(重庆鱼洞镇) → 运营管理 → 进站管理 → 进站交接单查询(tab)
|
# 主页(重庆鱼洞镇) → 运营管理 → 进站管理 → 进站交接单查询(tab)
|
||||||
@@ -39,8 +39,8 @@ import yaml
|
|||||||
# Windows 控制台默认 GBK，打印中文/emoji 会崩，强制 UTF-8。
|
# Windows 控制台默认 GBK，打印中文/emoji 会崩，强制 UTF-8。
|
||||||
sys.stdout.reconfigure(encoding="utf-8")
|
sys.stdout.reconfigure(encoding="utf-8")
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
|
||||||
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||||
@@ -73,13 +73,13 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
CDP_PORT = 9222 # 默认端口（独立运行 site_anneng.py 时用）；main_router 启动时会用 set_cdp_port 覆盖
|
CDP_PORT = 9222 # 默认端口（独立运行（python -m inbound_verify.sites.anneng）时用）；runtime 启动时会用 set_cdp_port 覆盖
|
||||||
POLL_INTERVAL = 0.5
|
POLL_INTERVAL = 0.5
|
||||||
DEFAULT_TIMEOUT = 25.0
|
DEFAULT_TIMEOUT = 25.0
|
||||||
|
|
||||||
|
|
||||||
def set_cdp_port(port):
|
def set_cdp_port(port):
|
||||||
"""由 main_router 在启动安能应用后调用，告知本模块实际使用的调试端口。"""
|
"""由 runtime 在启动安能应用后调用，告知本模块实际使用的调试端口。"""
|
||||||
global CDP_PORT
|
global CDP_PORT
|
||||||
CDP_PORT = int(port)
|
CDP_PORT = int(port)
|
||||||
|
|
||||||
@@ -186,7 +186,7 @@ def find_main_page_cdp():
|
|||||||
|
|
||||||
|
|
||||||
def anneng_ready():
|
def anneng_ready():
|
||||||
"""安能主页是否就绪（供 main_router 的就绪轮询调用）。
|
"""安能主页是否就绪（供 runtime 的就绪轮询调用）。
|
||||||
|
|
||||||
判据：能连上 CDP 且主页出现站点名称控件（带 title+fontsizenum 的 div）。
|
判据：能连上 CDP 且主页出现站点名称控件（带 title+fontsizenum 的 div）。
|
||||||
连不上或未就绪一律返回 False，不抛异常（就绪轮询会反复调用）。
|
连不上或未就绪一律返回 False，不抛异常（就绪轮询会反复调用）。
|
||||||
@@ -973,6 +973,7 @@ def set_scan_date(cdp, placeholder, value, target_ymd=None):
|
|||||||
返回 True 表示写入并校验成功，否则 False。
|
返回 True 表示写入并校验成功，否则 False。
|
||||||
"""
|
"""
|
||||||
import re as _re
|
import re as _re
|
||||||
|
|
||||||
ph = json.dumps(placeholder)
|
ph = json.dumps(placeholder)
|
||||||
val_json = json.dumps(value)
|
val_json = json.dumps(value)
|
||||||
if target_ymd is None:
|
if target_ymd is None:
|
||||||
@@ -998,7 +999,9 @@ def set_scan_date(cdp, placeholder, value, target_ymd=None):
|
|||||||
"(() => {const inp=[...document.querySelectorAll('input')]"
|
"(() => {const inp=[...document.querySelectorAll('input')]"
|
||||||
f".find(i=>i.placeholder==={ph}); if(!inp) return false; inp.focus();"
|
f".find(i=>i.placeholder==={ph}); if(!inp) return false; inp.focus();"
|
||||||
"let sel=true; try{ sel=document.execCommand('selectAll'); }catch(e){ try{inp.select();}catch(_){ sel=false; } }"
|
"let sel=true; try{ sel=document.execCommand('selectAll'); }catch(e){ try{inp.select();}catch(_){ sel=false; } }"
|
||||||
"let done=false; try{ done=document.execCommand('insertText',false," + val_json + "); }catch(e){ done=false; }"
|
"let done=false; try{ done=document.execCommand('insertText',false,"
|
||||||
|
+ val_json
|
||||||
|
+ "); }catch(e){ done=false; }"
|
||||||
"if(!done){ const s=Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype,'value').set;"
|
"if(!done){ const s=Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype,'value').set;"
|
||||||
f" s.call(inp,{val_json}); inp.dispatchEvent(new Event('input',{{bubbles:true}})); }}"
|
f" s.call(inp,{val_json}); inp.dispatchEvent(new Event('input',{{bubbles:true}})); }}"
|
||||||
"return inp.value;})()"
|
"return inp.value;})()"
|
||||||
@@ -1016,7 +1019,9 @@ def set_scan_date(cdp, placeholder, value, target_ymd=None):
|
|||||||
return True
|
return True
|
||||||
time.sleep(0.4)
|
time.sleep(0.4)
|
||||||
# 全部尝试失败：告警，避免静默下成「今天」
|
# 全部尝试失败：告警，避免静默下成「今天」
|
||||||
print(f" ⚠ set_scan_date 未能将 {placeholder} 设为目标日期 {target_ymd}（请检查 DatePicker 是否就绪）")
|
print(
|
||||||
|
f" ⚠ set_scan_date 未能将 {placeholder} 设为目标日期 {target_ymd}（请检查 DatePicker 是否就绪）"
|
||||||
|
)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
@@ -1158,8 +1163,12 @@ def wait_scan_form_ready(cdp, timeout=20.0):
|
|||||||
f".find(i=>i.placeholder==={json.dumps(ph)}); if(!inp) return false; inp.focus(); return true;}})()"
|
f".find(i=>i.placeholder==={json.dumps(ph)}); if(!inp) return false; inp.focus(); return true;}})()"
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
wait_until(cdp, "!!document.querySelector('.ant-picker-panel')",
|
wait_until(
|
||||||
"预热面板打开", timeout=4.0)
|
cdp,
|
||||||
|
"!!document.querySelector('.ant-picker-panel')",
|
||||||
|
"预热面板打开",
|
||||||
|
timeout=4.0,
|
||||||
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
cdp.eval(
|
cdp.eval(
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
# site_baishi.py
|
# sites/baishi.py
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import yaml
|
import yaml
|
||||||
from playwright.sync_api import sync_playwright
|
from playwright.sync_api import sync_playwright
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
|
||||||
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||||
@@ -38,7 +38,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
# 站点首页 URL：异常兜底重置用，也供 main_router 的 SITES_CONFIG 引用（单一来源）
|
# 站点首页 URL：异常兜底重置用，也供 runtime 的 SITES_CONFIG 引用（单一来源）
|
||||||
HOME_URL = "https://v5.800best.com"
|
HOME_URL = "https://v5.800best.com"
|
||||||
|
|
||||||
|
|
||||||
@@ -181,8 +181,12 @@ def baishi_download_undelivered_data_impl(page):
|
|||||||
# 10(应扫) 11(已扫) 12(未扫) 13(率)。（与下方 nth(12) 未扫同源）
|
# 10(应扫) 11(已扫) 12(未扫) 13(率)。（与下方 nth(12) 未扫同源）
|
||||||
try:
|
try:
|
||||||
_first_row = page.locator(".ant-table-tbody > tr").first
|
_first_row = page.locator(".ant-table-tbody > tr").first
|
||||||
_exp_txt = _first_row.locator("td").nth(10).inner_text().strip().replace(",", "")
|
_exp_txt = (
|
||||||
_arr_txt = _first_row.locator("td").nth(11).inner_text().strip().replace(",", "")
|
_first_row.locator("td").nth(10).inner_text().strip().replace(",", "")
|
||||||
|
)
|
||||||
|
_arr_txt = (
|
||||||
|
_first_row.locator("td").nth(11).inner_text().strip().replace(",", "")
|
||||||
|
)
|
||||||
|
|
||||||
def _to_int(v):
|
def _to_int(v):
|
||||||
try:
|
try:
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# site_shunxin.py
|
# sites/shunxin.py
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -7,8 +7,8 @@ import yaml
|
|||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
|
||||||
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||||
@@ -41,7 +41,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
# 站点首页 URL：异常兜底重置用，也供 main_router 的 SITES_CONFIG 引用（单一来源）
|
# 站点首页 URL：异常兜底重置用，也供 runtime 的 SITES_CONFIG 引用（单一来源）
|
||||||
HOME_URL = "https://sxne.sxjdfreight.com"
|
HOME_URL = "https://sxne.sxjdfreight.com"
|
||||||
|
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# site_yunda.py
|
# sites/yunda.py
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -7,8 +7,8 @@ import yaml
|
|||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
|
||||||
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||||
@@ -41,7 +41,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
# 站点首页 URL：异常兜底重置用，也供 main_router 的 SITES_CONFIG 引用（单一来源）
|
# 站点首页 URL：异常兜底重置用，也供 runtime 的 SITES_CONFIG 引用（单一来源）
|
||||||
HOME_URL = "https://ky-sso.yunda56.com"
|
HOME_URL = "https://ky-sso.yunda56.com"
|
||||||
|
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# site_zto.py
|
# sites/zto.py
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -7,8 +7,8 @@ import yaml
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
from paths import DOWNLOAD_DIR, CONFIG_PATH
|
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
|
||||||
import state_store
|
from inbound_verify import state_store
|
||||||
|
|
||||||
|
|
||||||
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||||
@@ -41,7 +41,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
# 站点首页 URL：异常兜底重置用，也供 main_router 的 SITES_CONFIG 引用（单一来源）
|
# 站点首页 URL：异常兜底重置用，也供 runtime 的 SITES_CONFIG 引用（单一来源）
|
||||||
HOME_URL = "https://ws.zto56.com/"
|
HOME_URL = "https://ws.zto56.com/"
|
||||||
|
|
||||||
|
|
||||||
@@ -9,7 +9,7 @@ import os
|
|||||||
import sqlite3
|
import sqlite3
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from paths import STATE_DB_PATH
|
from inbound_verify.paths import STATE_DB_PATH
|
||||||
|
|
||||||
# 登录态枚举
|
# 登录态枚举
|
||||||
LOGIN_UNKNOWN = "unknown" # 尚未探测过
|
LOGIN_UNKNOWN = "unknown" # 尚未探测过
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
"""
|
"""
|
||||||
db_store.py — 到货核销数据持久化（PostgreSQL）
|
store.py — 到货核销数据持久化（PostgreSQL）
|
||||||
|
|
||||||
职责：把 downloads/ 下各站点下载的应到 / 实到 / 未到 Excel 解析后，幂等写入 PostgreSQL。
|
职责：把 downloads/ 下各站点下载的应到 / 实到 / 未到 Excel 解析后，幂等写入 PostgreSQL。
|
||||||
与下载流程解耦：本模块只读 downloads/ 现有文件入库，不关心谁触发下载、下载了几次。
|
与下载流程解耦：本模块只读 downloads/ 现有文件入库，不关心谁触发下载、下载了几次。
|
||||||
@@ -13,10 +13,10 @@ db_store.py — 到货核销数据持久化（PostgreSQL）
|
|||||||
- 单号一律按文本读写（dtype=str），防长数字被科学计数 / 精度丢失
|
- 单号一律按文本读写（dtype=str），防长数字被科学计数 / 精度丢失
|
||||||
|
|
||||||
命令行：
|
命令行：
|
||||||
python db_store.py createdb 创建数据库（幂等）
|
python -m inbound_verify.store createdb 创建数据库（幂等）
|
||||||
python db_store.py init 建表（幂等 CREATE TABLE IF NOT EXISTS）
|
python -m inbound_verify.store init 建表（幂等 CREATE TABLE IF NOT EXISTS）
|
||||||
python db_store.py ingest [site] 入库全站或单站（幂等 UPSERT）
|
python -m inbound_verify.store ingest [site] 入库全站或单站（幂等 UPSERT）
|
||||||
python db_store.py all createdb → init → 全站 ingest 一条龙
|
python -m inbound_verify.store all createdb → init → 全站 ingest 一条龙
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
@@ -28,9 +28,13 @@ import psycopg
|
|||||||
import yaml
|
import yaml
|
||||||
from psycopg.types.json import Jsonb
|
from psycopg.types.json import Jsonb
|
||||||
|
|
||||||
from paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
|
from inbound_verify.paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
|
||||||
|
from inbound_verify.domain import (
|
||||||
|
BAISHI_FILE,
|
||||||
|
_site_cfg,
|
||||||
|
) # 站点 / 文件名配置（单一来源）
|
||||||
|
|
||||||
import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源）
|
from inbound_verify import compare # _read_business_dates（比对侧业务日期读取）
|
||||||
|
|
||||||
SCHEMA_PATH = os.path.join(BASE_DIR, "schema.sql")
|
SCHEMA_PATH = os.path.join(BASE_DIR, "schema.sql")
|
||||||
|
|
||||||
@@ -105,7 +109,7 @@ def init_schema():
|
|||||||
|
|
||||||
# ============================== 解析辅助 ==============================
|
# ============================== 解析辅助 ==============================
|
||||||
|
|
||||||
# 实到单号列 / 基号列映射（口径取自 expected_undelivered）：
|
# 实到单号列 / 基号列映射（口径取自 domain，与 STATIONS 对齐）：
|
||||||
# piece = 实到表里「每件」的单号列（扫描单号 / 子单号 / 复合串）
|
# piece = 实到表里「每件」的单号列（扫描单号 / 子单号 / 复合串）
|
||||||
# waybill = 与应到运单号对齐的干净列（中通无干净列，由 piece 复合串 v[:-8] 推导）
|
# waybill = 与应到运单号对齐的干净列（中通无干净列，由 piece 复合串 v[:-8] 推导）
|
||||||
# scan_time = 扫描时间列（缺失则不填，原始值仍在 raw）
|
# scan_time = 扫描时间列（缺失则不填，原始值仍在 raw）
|
||||||
@@ -208,7 +212,7 @@ def _raw_row(row):
|
|||||||
def _read_business_dates():
|
def _read_business_dates():
|
||||||
"""从状态库读各站本次业务日期（与报告口径一致；读不到返回空 dict）。"""
|
"""从状态库读各站本次业务日期（与报告口径一致；读不到返回空 dict）。"""
|
||||||
try:
|
try:
|
||||||
return eu._read_business_dates(ALL_SITES + ["百世"]) or {}
|
return compare._read_business_dates(ALL_SITES + ["百世"]) or {}
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f">> [warn] 读取业务日期失败（不影响入库）: {e}")
|
print(f">> [warn] 读取业务日期失败（不影响入库）: {e}")
|
||||||
return {}
|
return {}
|
||||||
@@ -259,7 +263,7 @@ _SQL_UNDELIVERED = """
|
|||||||
|
|
||||||
def _ingest_expected(cur, site, business_date):
|
def _ingest_expected(cur, site, business_date):
|
||||||
"""入库单站应到（运单级，按 waybill_no 去重 keep-first 后 UPSERT）。"""
|
"""入库单站应到（运单级，按 waybill_no 去重 keep-first 后 UPSERT）。"""
|
||||||
cfg = eu._site_cfg(site)
|
cfg = _site_cfg(site)
|
||||||
path = os.path.join(DOWNLOAD_DIR, cfg["exp"])
|
path = os.path.join(DOWNLOAD_DIR, cfg["exp"])
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
print(f" [跳过] {site} 应到：文件不存在 {cfg['exp']}")
|
print(f" [跳过] {site} 应到：文件不存在 {cfg['exp']}")
|
||||||
@@ -291,7 +295,7 @@ def _ingest_expected(cur, site, business_date):
|
|||||||
|
|
||||||
def _ingest_actual(cur, site):
|
def _ingest_actual(cur, site):
|
||||||
"""入库单站实到（扫描件级，按 piece_no UPSERT）。"""
|
"""入库单站实到（扫描件级，按 piece_no UPSERT）。"""
|
||||||
cfg = eu._site_cfg(site)
|
cfg = _site_cfg(site)
|
||||||
path = os.path.join(DOWNLOAD_DIR, cfg["act"])
|
path = os.path.join(DOWNLOAD_DIR, cfg["act"])
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
print(f" [跳过] {site} 实到：文件不存在 {cfg['act']}")
|
print(f" [跳过] {site} 实到：文件不存在 {cfg['act']}")
|
||||||
@@ -330,9 +334,9 @@ def _ingest_actual(cur, site):
|
|||||||
|
|
||||||
def _ingest_undelivered_baishi(cur):
|
def _ingest_undelivered_baishi(cur):
|
||||||
"""入库百世应到未到明细（子单级，按 (site, piece_no) UPSERT）。"""
|
"""入库百世应到未到明细（子单级，按 (site, piece_no) UPSERT）。"""
|
||||||
path = os.path.join(DOWNLOAD_DIR, eu.BAISHI_FILE)
|
path = os.path.join(DOWNLOAD_DIR, BAISHI_FILE)
|
||||||
if not os.path.exists(path):
|
if not os.path.exists(path):
|
||||||
print(f" [跳过] 百世 未到：文件不存在 {eu.BAISHI_FILE}")
|
print(f" [跳过] 百世 未到：文件不存在 {BAISHI_FILE}")
|
||||||
return 0
|
return 0
|
||||||
df = pd.read_excel(path, dtype=str).fillna("")
|
df = pd.read_excel(path, dtype=str).fillna("")
|
||||||
rows = []
|
rows = []
|
||||||
@@ -376,7 +380,7 @@ def ingest(site=None):
|
|||||||
# ============================== 命令行 ==============================
|
# ============================== 命令行 ==============================
|
||||||
|
|
||||||
|
|
||||||
def _cli():
|
def main():
|
||||||
cmd = sys.argv[1] if len(sys.argv) > 1 else "all"
|
cmd = sys.argv[1] if len(sys.argv) > 1 else "all"
|
||||||
site = sys.argv[2] if len(sys.argv) > 2 else None
|
site = sys.argv[2] if len(sys.argv) > 2 else None
|
||||||
if cmd == "createdb":
|
if cmd == "createdb":
|
||||||
@@ -395,4 +399,4 @@ def _cli():
|
|||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
_cli()
|
main()
|
||||||
28
pyproject.toml
Normal file
28
pyproject.toml
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "inbound-verify"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "物流到货数据自动下载与应到未到核对工具"
|
||||||
|
requires-python = ">=3.10"
|
||||||
|
dependencies = [
|
||||||
|
"pandas>=2.0.0",
|
||||||
|
"playwright>=1.40.0",
|
||||||
|
"openpyxl>=3.1.0",
|
||||||
|
"PyYAML>=6.0",
|
||||||
|
"websocket-client>=1.0.0",
|
||||||
|
"fastapi>=0.110.0",
|
||||||
|
"uvicorn>=0.27.0",
|
||||||
|
"apscheduler>=3.10.0",
|
||||||
|
"psycopg[binary]>=3.1",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
inbound-verify = "inbound_verify.cli.router:main"
|
||||||
|
inbound-verify-server = "inbound_verify.cli.server:main"
|
||||||
|
inbound-verify-db = "inbound_verify.store:main"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
include = ["inbound_verify*"]
|
||||||
Reference in New Issue
Block a user