diff --git a/CLAUDE.md b/CLAUDE.md index 8566885..0c405ef 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,73 +8,82 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co 异常运单并汇总成 Excel。4 个网页站点 + 1 个 Electron 应用（安能）。 **两种运行模式**（阶段1 起）： -- **交互模式** `main_router.py`：人工调试 / 操作，交互菜单（登录、触发下载、[12] 状态盘）。 -- **服务模式** `server.py`：常驻 + FastAPI，客户端经 HTTP 触发任务、查状态、下载数据（API 文档 `/docs`）。 -- 两者共享 `runtime.py`（启动 / 就绪 / 任务派发 / 心跳）与 `state_store.py`（SQLite 状态持久化）。 +- **交互模式** `inbound_verify.cli.router`：人工调试 / 操作，交互菜单（登录、触发下载、[12] 状态盘）。 +- **服务模式** `inbound_verify.cli.server`：常驻 + FastAPI，客户端经 HTTP 触发任务、查状态、下载数据（API 文档 `/docs`）。 +- 两者共享 `inbound_verify.runtime`（启动 / 就绪 / 任务派发 / 心跳）与 `inbound_verify.state_store`（SQLite 状态持久化）。 ## 常用命令 所有 Python 一律在项目虚拟环境 `.venv` 中运行（Windows 下可直接用 -`.venv/Scripts/python.exe`，无需激活）。 +`.venv/Scripts/python.exe`，无需激活）。首次 / 拉取新代码后需 +`.venv/Scripts/python.exe -m pip install -e .`（以可编辑模式注册 +`inbound-verify` 等命令）。 ```bash -# 安装依赖（含安能 CDP 驱动所需的 websocket-client） +# 安装依赖（含安能 CDP 驱动所需的 websocket-client）+ 以可编辑模式注册命令 pip install -r requirements.txt +pip install -e . playwright install chromium -# 运行主程序（交互式菜单，详见 main_router 的 run_multi_site_daemon） -.venv/Scripts/python.exe main_router.py +# 运行主程序（交互式菜单，详见 inbound_verify.cli.router 的 run_multi_site_daemon） +.venv/Scripts/python.exe -m inbound_verify.cli.router +# 装包后也可直接用命令：inbound-verify # 服务模式（常驻 + FastAPI，客户端经 HTTP 触发；默认 :8000，API 文档见 /docs） -.venv/Scripts/python.exe server.py +.venv/Scripts/python.exe -m inbound_verify.cli.server +# 或：inbound-verify-server + +# DB CLI：建库 / 初始化 / 灌数据 / 全流程 +.venv/Scripts/python.exe -m inbound_verify.store createdb # 或 init | ingest | all +# 或：inbound-verify-db createdb|init|ingest|all # 单站点联调：在 config.yaml 设 debug.enabled=true + debug.target_site=顺心|百世|中通|韵达|安能 # 网页站：只挂载该站；安能：只启动 Electron 应用。 # 安能独立运行（需先以 --remote-debugging-port=9222 启动「安能全网门户.exe」并手动登录） -.venv/Scripts/python.exe site_anneng.py expected # 或 actual +.venv/Scripts/python.exe -m inbound_verify.sites.anneng expected # 或 actual # 格式化（全局规范：改完 Python 必须 Black） -.venv/Scripts/python.exe -m black +.venv/Scripts/python.exe -m black inbound_verify # 语法自检 -.venv/Scripts/python.exe -m py_compile +.venv/Scripts/python.exe -m py_compile inbound_verify ``` -**没有 pytest 测试套件。** "测试"指 `main_router` 菜单 **[8] 自动化测试** +**没有 pytest 测试套件。** "测试"指 `inbound_verify.cli.router` 菜单 **[8] 自动化测试** （`run_automation_test`，按 `CROSS_TEST_SEQUENCE` 交叉跑通各站点流程）。 ## 架构（big picture） ### 运行模式与共享核心（阶段0/1 重构） -- **`runtime.py`**：两种模式共享的核心——`launch_and_prepare`（启动 Playwright + 各站就绪 + 弹窗 + 心跳初值，阻塞至就绪）、`dispatch_task(ctx, {site,kind})`（派发任务，掉登录直接判 failed）、`run_heartbeat`、`probe_site_login/probe_data_file`、`RuntimeContext.stop()`。常量 `SITES_CONFIG/READY_SELECTORS/APP_SITES/HEARTBEAT_INTERVAL/DATA_FILENAMES` 在此。 -- **`state_store.py`**：SQLite 状态持久化（`state/state.db`）。`site_status`（登录态 + 数据态 + 时间戳，心跳刷新）、`task_history`（任务记录）。重启不丢。 -- **`main_router.py`**：交互模式，菜单循环（input 后台线程 + `_await_command` + `dispatch_task` + 心跳）。 -- **`server.py`**：服务模式。**FastAPI（主线程）+ Playwright worker（独立线程）**——主线程处理 HTTP（绝不碰 Playwright），worker 独占 page 操作，经 `task_queue` + `state_store` 通信。API：`POST/GET /tasks`、`GET /status`、`GET /data/{file}`。 +- **`inbound_verify.runtime`**：两种模式共享的核心——`launch_and_prepare`（启动 Playwright + 各站就绪 + 弹窗 + 心跳初值，阻塞至就绪）、`dispatch_task(ctx, {site,kind})`（派发任务，掉登录直接判 failed）、`run_heartbeat`、`probe_site_login/probe_data_file`、`RuntimeContext.stop()`。常量 `SITES_CONFIG/READY_SELECTORS/APP_SITES/HEARTBEAT_INTERVAL/DATA_FILENAMES` 在此。 +- **`inbound_verify.state_store`**：SQLite 状态持久化（`state/state.db`）。`site_status`（登录态 + 数据态 + 时间戳，心跳刷新）、`task_history`（任务记录）。重启不丢。 +- **`inbound_verify.cli.router`**：交互模式，菜单循环（input 后台线程 + `_await_command` + `dispatch_task` + 心跳）。 +- **`inbound_verify.cli.server`**：服务模式。**FastAPI（主线程）+ Playwright worker（独立线程）**——主线程处理 HTTP（绝不碰 Playwright），worker 独占 page 操作，经 `task_queue` + `state_store` 通信。API：`POST/GET /tasks`、`GET /status`、`GET /data/{file}`。 - **关键线程约束**：Playwright sync 对象绑定创建它的线程；`launch_and_prepare`（含 `sync_playwright().start()`）必须在持有 Playwright 的线程调用（交互=主线程，服务=worker 线程）。FastAPI 路由绝不访问 page。 -- 站点模块 `site_*.py` 的 `with_retry` 返回 `True/False`（成功 / 放弃），供 `dispatch_task` 判成败。 +- 站点模块 `inbound_verify.sites.*` 的 `with_retry` 返回 `True/False`（成功 / 放弃），供 `dispatch_task` 判成败。 ### 两套驱动模态 —— 这是理解全局的关键 -- **网页 4 站**（顺心/百世/中通/韵达）：`main_router` 用 Playwright 开 chromium， +- **网页 4 站**（顺心/百世/中通/韵达）：`inbound_verify.cli.router` 用 Playwright 开 chromium， 每站一个 `page`，流程函数签名为 `xxx_download_impl(page)`。**例外：顺心是双账号** ——同一窗口开两个标签页（两个归属地账号），`pages_map["顺心"]` 存为 page **列表**， `shunxin_download(pages)` 接收列表（详见下文「顺心双账号」）。 -- **安能**：Electron 桌面应用，**不走 Playwright**。`main_router` 以 +- **安能**：Electron 桌面应用，**不走 Playwright**。`inbound_verify.cli.router` 以 `--remote-debugging-port=<动态空闲端口>` 启动 exe（`launch_anneng`）， - 通过 `site_anneng.set_cdp_port` 告知模块；`site_anneng.py` 用裸 CDP（websocket） + 通过 `inbound_verify.sites.anneng.set_cdp_port` 告知模块；`inbound_verify.sites.anneng` 用裸 CDP（websocket） 驱动，业务 tab 是独立 webContents。这也是 `playwright-cli` 接管不了安能的原因 （Electron 19 / Chrome 102 不支持 Playwright 要的 setDownloadBehavior）。 ### 分层：路由纯调度，站点模块自洽 -- `main_router.py` **只调度**：启动浏览器/安能、就绪轮询、登录检测、菜单分发。 - 菜单项直接调 `site_xxx.xxx_download(page)`（顺心传 page 列表），**不关心**重试/重置。 -- 每个 `site_xxx.py` 对外只暴露"把任务做了"的入口，内部自洽： +- `inbound_verify.cli.router` **只调度**：启动浏览器/安能、就绪轮询、登录检测、菜单分发。 + 菜单项直接调 `sites.xxx_download(page)`（顺心传 page 列表），**不关心**重试/重置。 +- 每个 `inbound_verify.sites.*` 模块对外只暴露"把任务做了"的入口，内部自洽： - `xxx_download(...)` —— 公开入口，= `with_retry(站点, 标签, xxx_download_impl, xxx_reset)` - `xxx_download_impl(...)` —— 单次执行、**无重试**（自动化测试刻意调它以探测原始失败） - `xxx_reset(...)` —— 重置回初始态（网页 = `page.goto(HOME_URL)`；安能 = 关业务 tab + 收菜单） - `with_retry(...)` —— 重试逻辑**内联在每个站点模块**（不抽公共组件，现阶段刻意不优化结构）； 失败→重置→重试，最多 3 次（含首次），每次失败都重置（含最终放弃那次清场） - - `HOME_URL` —— 站点首页 URL，`main_router.SITES_CONFIG` 引用它（单一来源） + - `HOME_URL` —— 站点首页 URL，`runtime.SITES_CONFIG` 引用它（单一来源） ### 导出任务队列模式（顺心/中通/韵达/安能-应到 共用） 这些站的下载是异步的：提交导出（记 `export_times` 时间戳）→ 跳"导出任务管理"页轮询 → @@ -88,7 +97,7 @@ playwright install chromium ### 顺心双账号（双归属地） 顺心业务上要同时处理**两个归属地网点**（两个账号）。程序在同一窗口开两个标签页， 人工分别登录两个账号（顺心站点支持同浏览器双账号并存，无需独立 context/窗口）。 -- `main_router` 启动时为顺心开 2 个 `context.new_page()`，`pages_map["顺心"]` 为列表； +- `runtime` 启动时为顺心开 2 个 `context.new_page()`，`pages_map["顺心"]` 为列表； 就绪轮询要求**两个标签页都进主页**才算就绪；初始弹窗对两个标签页各处理一遍。 - `shunxin_expected_download(pages)` / `shunxin_actual_download(pages)` 接收 page 列表： 先用 `shunxin_belonging(page)` 读各账号归属地（首页「切换网点」控件 `.site___3o7nH`）， @@ -100,16 +109,16 @@ playwright install chromium 配合每账号独立 `export_times` + ≤40s 容差，B 不会误匹配 A 的任务。 ### 比对 -`expected_undelivered.py`（菜单 [9]）纯离线：读 `downloads/` 下各站应到/实到 xlsx， +`inbound_verify.expected_undelivered`（菜单 [9]）纯离线：读 `downloads/` 下各站应到/实到 xlsx， 比对生成 `output/应到未到数据.xlsx`（汇总 + 各站明细）。 ### 路径 -`paths.py` 把 `DOWNLOAD_DIR` / `CONFIG_PATH` / `BASE_DIR` 全部锚定到项目目录， +`inbound_verify.paths` 把 `DOWNLOAD_DIR` / `CONFIG_PATH` / `BASE_DIR` 全部锚定到项目目录， **不依赖运行时 cwd**——别用相对路径或 `os.getcwd()`。 ## 重要约定 / 易踩坑 -- **登录是手动的**：`main_router` 启动后会停在就绪轮询（`READY_SELECTORS` / `anneng_ready`）， +- **登录是手动的**：`inbound_verify.cli.router` 启动后会停在就绪轮询（`READY_SELECTORS` / `anneng_ready`）， 直到检测到所有站点进入工作台才进菜单。仅韵达支持凭 `config.yaml` 凭据自动登录。 **顺心需登录两个账号**：同一窗口的两个标签页分别登录两个不同归属地账号，两个标签页 都进主页后才算就绪（顺心站点支持同浏览器双账号并存，故用同 context 标签页而非独立窗口）。 diff --git a/README.md b/README.md index 4dbcde3..e3573e0 100644 --- a/README.md +++ b/README.md @@ -32,20 +32,30 @@ ``` InboundVerify/ -├── main_router.py # 主入口 / 调度层（菜单、启动浏览器与安能、就绪轮询、登录检测） -├── site_shunxin.py # 顺心站点模块（流程 + 重置 + 重试，自洽） -├── site_baishi.py # 百世站点模块 -├── site_zto.py # 中通站点模块 -├── site_yunda.py # 韵达站点模块（含自动登录） -├── site_anneng.py # 安能站点模块（Electron + CDP 驱动） -├── expected_undelivered.py # 全站点应到未到离线比对，输出 output/应到未到数据.xlsx -├── paths.py # 统一路径锚点（以本目录为基准，不依赖 cwd） +├── pyproject.toml # 打包 + 依赖 + console_scripts（inbound-verify 等） +├── inbound_verify/ # 源码包 +│ ├── paths.py # 统一路径锚点（以项目目录为基准，不依赖 cwd） +│ ├── runtime.py # 两种模式共享核心（启动 / 就绪 / 任务派发 / 心跳） +│ ├── state_store.py # SQLite 状态持久化（state/state.db） +│ ├── expected_undelivered.py# 全站点应到未到离线比对，输出 output/应到未到数据.xlsx +│ ├── store.py # DB CLI 入口（createdb|init|ingest|all） +│ ├── sites/ # 各站点模块（流程 + 重置 + 重试，自洽） +│ │ ├── shunxin.py # 顺心（含双账号） +│ │ ├── baishi.py # 百世 +│ │ ├── zto.py # 中通 +│ │ ├── yunda.py # 韵达（含自动登录） +│ │ └── anneng.py # 安能（Electron + CDP 驱动） +│ └── cli/ # 命令行入口 +│ ├── router.py # 交互菜单（调度层：启动 / 就绪轮询 / 登录检测 / 菜单分发） +│ └── server.py # FastAPI 服务模式（常驻 + HTTP 触发） ├── config.example.yaml # 配置模板 -├── config.yaml # 真实配置（自行创建，已被 .gitignore 忽略） -├── requirements.txt -├── downloads/ # 各站点下载的原始数据 -├── output/ # 比对报表输出 -└── docs/ # 说明文档 +├── config.yaml # 真实配置（自行创建，已被 .gitignore 忽略） +├── schema.sql # 数据库表结构（store.py createdb / init 使用） +├── requirements.txt # pyproject 依赖的静态镜像 +├── downloads/ # 各站点下载的原始数据 +├── output/ # 比对报表输出 +├── state/ # 运行状态持久化（state.db） +└── docs/ # 说明文档 ``` --- @@ -63,10 +73,13 @@ python -m venv .venv # 2. 安装依赖（含安能 CDP 驱动所需的 websocket-client） pip install -r requirements.txt -# 3. 安装 Playwright 浏览器内核（网页站点用） +# 3. 以可编辑模式安装本包（注册 inbound-verify 等命令） +pip install -e . + +# 4. 安装 Playwright 浏览器内核（网页站点用） playwright install chromium -# 4. 由模板创建本地配置并填入真实凭据 +# 5. 由模板创建本地配置并填入真实凭据 cp config.example.yaml config.yaml ``` @@ -96,7 +109,10 @@ cp config.example.yaml config.yaml ## 五、运行 ```bash -python main_router.py +# 交互菜单（任选其一） +python -m inbound_verify.cli.router +# 或装包后直接用命令： +inbound-verify ``` 程序会： @@ -126,10 +142,10 @@ python main_router.py **分层原则：路由层只调度，站点模块自洽。** -- **`main_router.py`（调度层）**：负责启动浏览器 / 安能、就绪轮询、登录检测、 +- **`inbound_verify.cli.router`（调度层）**：负责启动浏览器 / 安能、就绪轮询、登录检测、 菜单分发。**不关心**"任务能否完成、失败怎么办"——只调 - `site_xxx.xxx_download(page)` 然后等结果。 -- **各 `site_xxx.py`（站点模块）**：每个模块对外只暴露一个"把任务做了"的入口 + `inbound_verify.sites.xxx_download(page)` 然后等结果。 +- **各 `inbound_verify.sites.*`（站点模块）**：每个模块对外只暴露一个"把任务做了"的入口 `xxx_download(...)`，内部自行处理一切： - `HOME_URL`：站点首页 URL（也供路由层 `SITES_CONFIG` 引用，单一来源）； - `xxx_reset(...)`：异常兜底的重置（网页 = 跳首页 URL；安能 = 关业务 tab + 收菜单）； @@ -162,4 +178,4 @@ python main_router.py - 首次运行需手动登录各站点（程序会停在就绪轮询，直到检测到所有站点进入工作台）； - 安能为单实例 Electron 应用：启动前请先关闭已打开的安能窗口； -- 所有下载/输出路径以项目目录为基准（见 `paths.py`），与从哪个目录启动无关。 +- 所有下载/输出路径以项目目录为基准（见 `inbound_verify.paths`），与从哪个目录启动无关。 diff --git a/docs/superpowers/plans/2026-07-23-tier1-package-move.md b/docs/superpowers/plans/2026-07-23-tier1-package-move.md new file mode 100644 index 0000000..b2098f6 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-tier1-package-move.md @@ -0,0 +1,579 @@ +# Tier 1: Package Move Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Move all 12 flat root-level Python modules into an installable `inbound_verify/` package (sites/, cli/ subpackages), rewrite every internal import to package-qualified form, and expose three console_script entry points — with **zero behavior change**. + +**Architecture:** Pure mechanical relocation (git mv preserves history) + import rewrite + paths.py anchor fix + entry-point `main()` wrappers + `pyproject.toml` packaging. No logic changes. Module names kept for heavily-referenced modules (`state_store`, `expected_undelivered`, `runtime`, `paths`) to avoid call-site churn; only leaf entries (`db_store→store`, `main_router→cli/router`, `server→cli/server`) and site files (drop `site_` prefix) are renamed. + +**Tech Stack:** Python ≥3.10, setuptools (PEP 517/621), pip editable install, psycopg3, FastAPI/uvicorn, Playwright. + +## Global Constraints + +- **Python ≥ 3.10** (`requires-python = ">=3.10"` in pyproject). +- **Package** import name `inbound_verify`; **distribution** name `inbound-verify`. +- **Dependency floors** (verbatim from spec): `pandas>=2.0.0`, `playwright>=1.40.0`, `openpyxl>=3.1.0`, `PyYAML>=6.0`, `websocket-client>=1.0.0`, `fastapi>=0.110.0`, `uvicorn>=0.27.0`, `apscheduler>=3.10.0`, `psycopg[binary]>=3.1`. +- **No test suite** (user decision). Verification = `compileall` + import smoke + grep-for-stale-refs + DB connectivity. No pytest. +- **Behavior must not change** in Tier 1 — pure move. +- **No auto-commit/push.** Every commit step below runs ONLY after the user explicitly says "提交/commit". Commit messages in English. +- **Black-format** every changed `.py` (global rule). +- All changes are **inside the `InboundVerify` git submodule**; the parent repo pointer bump is a separate parent-repo step, out of scope. +- All commands run from the `InboundVerify/` directory using the venv interpreter `.venv/Scripts/python.exe` (Windows; no activation needed). + +**Reference spec:** `docs/superpowers/specs/2026-07-23-package-restructure-design.md` (§3 mapping table, §4 paths anchor, §5 import rules, §6 entry/packaging, §7 verification gate). + +--- + +## File Structure (what each file becomes responsible for) + +``` +inbound_verify/ +├── __init__.py # empty (package marker) +├── paths.py # path anchors → PROJECT ROOT (one dir above package) +├── runtime.py # orchestration core (unchanged logic) +├── state_store.py # SQLite state (unchanged; name kept) +├── expected_undelivered.py# offline compare (unchanged; name kept — Tier 2 renames to compare) +├── store.py # PostgreSQL persist + main() (was db_store.py) +├── sites/ +│ ├── __init__.py # empty +│ ├── shunxin.py # (was site_shunxin.py) +│ ├── baishi.py # (was site_baishi.py) +│ ├── zto.py # (was site_zto.py) +│ ├── yunda.py # (was site_yunda.py) +│ └── anneng.py # (was site_anneng.py) +└── cli/ + ├── __init__.py # empty + ├── router.py # interactive menu + main() (was main_router.py) + └── server.py # FastAPI service + main() (was server.py) +``` + +Root keeps: `pyproject.toml` (new), `config.yaml`, `config.example.yaml`, `schema.sql`, `requirements.txt`, `README.md`, `CLAUDE.md`, `docs/`, `downloads/`, `output/`, `state/`. + +--- + +## Task 1: Package scaffold + pyproject + editable install + +**Files:** +- Create: `inbound_verify/__init__.py`, `inbound_verify/sites/__init__.py`, `inbound_verify/cli/__init__.py` +- Create: `pyproject.toml` + +**Interfaces:** +- Produces: an importable (near-empty) `inbound_verify` package + console_script registration. The flat root scripts remain 100% functional after this task (untouched). + +- [ ] **Step 1: Create package marker files** + +Create three empty files: +- `inbound_verify/__init__.py` +- `inbound_verify/sites/__init__.py` +- `inbound_verify/cli/__init__.py` + +Each is a single comment line: +```python +# inbound_verify package +``` + +- [ ] **Step 2: Write pyproject.toml** + +Create `pyproject.toml`: +```toml +[build-system] +requires = ["setuptools>=68"] +build-backend = "setuptools.build_meta" + +[project] +name = "inbound-verify" +version = "0.1.0" +description = "物流到货数据自动下载与应到未到核对工具" +requires-python = ">=3.10" +dependencies = [ + "pandas>=2.0.0", + "playwright>=1.40.0", + "openpyxl>=3.1.0", + "PyYAML>=6.0", + "websocket-client>=1.0.0", + "fastapi>=0.110.0", + "uvicorn>=0.27.0", + "apscheduler>=3.10.0", + "psycopg[binary]>=3.1", +] + +[project.scripts] +inbound-verify = "inbound_verify.cli.router:main" +inbound-verify-server = "inbound_verify.cli.server:main" +inbound-verify-db = "inbound_verify.store:main" + +[tool.setuptools.packages.find] +include = ["inbound_verify*"] +``` + +- [ ] **Step 3: Editable-install into the existing venv** + +Run: +```bash +.venv/Scripts/python.exe -m pip install -e . +``` +Expected: `Successfully installed inbound-verify-0.1.0` (deps already satisfied from earlier install — no network needed). + +- [ ] **Step 4: Verify package imports** + +Run: +```bash +.venv/Scripts/python.exe -c "import inbound_verify, inbound_verify.sites, inbound_verify.cli; print('package OK')" +``` +Expected output: `package OK` + +- [ ] **Step 5: Commit (only after user confirms)** + +```bash +git add inbound_verify/__init__.py inbound_verify/sites/__init__.py inbound_verify/cli/__init__.py pyproject.toml +git commit -m "chore: scaffold inbound_verify package and pyproject + +Co-Authored-By: Claude " +``` +> Do NOT commit until the user says to. + +--- + +## Task 2: Atomic move — relocate, rewrite imports, fix anchor, wire mains, verify + +**Files:** +- Move (git mv): all 12 root `.py` modules → package locations (see Step 1) +- Modify: `inbound_verify/paths.py` (anchor), and import lines + call sites in every moved module +- Modify: entry `main()` in `cli/router.py`, `cli/server.py`, `store.py` + +**Interfaces:** +- Consumes: the scaffold from Task 1 (package importable + editable install active). +- Produces: a fully functional `inbound_verify` package invokable via `python -m inbound_verify.cli.router`, `python -m inbound_verify.cli.server`, `python -m inbound_verify.store`, or the three console_scripts. Old root `.py` files are gone. The flat root scripts no longer exist — invocation switches to package form. + +> **Why this is one task:** in a flat-import codebase, moving `paths.py` (imported by everyone) immediately breaks every importer until ALL moves + rewrites are complete. There is no intermediate state that imports cleanly, so the whole move is one atomic unit verified by the gate at the end. Each file-edit step below is followed by `py_compile` of that file to catch syntax errors as we go. + +- [ ] **Step 1: Relocate all 12 modules with git mv (history preserved)** + +From the `InboundVerify/` directory: +```bash +git mv paths.py inbound_verify/paths.py +git mv runtime.py inbound_verify/runtime.py +git mv state_store.py inbound_verify/state_store.py +git mv expected_undelivered.py inbound_verify/expected_undelivered.py +git mv db_store.py inbound_verify/store.py +git mv main_router.py inbound_verify/cli/router.py +git mv server.py inbound_verify/cli/server.py +git mv site_shunxin.py inbound_verify/sites/shunxin.py +git mv site_baishi.py inbound_verify/sites/baishi.py +git mv site_zto.py inbound_verify/sites/zto.py +git mv site_yunda.py inbound_verify/sites/yunda.py +git mv site_anneng.py inbound_verify/sites/anneng.py +``` +After this the tree is temporarily broken (imports unresolved) — expected. Continue. + +- [ ] **Step 2: Fix paths.py anchor to point at project root** + +In `inbound_verify/paths.py`, replace: +```python +BASE_DIR = os.path.dirname(os.path.abspath(__file__)) +``` +with: +```python +# __file__ = /inbound_verify/paths.py → 上两级 = 项目根 +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +``` +(`DOWNLOAD_DIR`/`OUTPUT_DIR`/`CONFIG_PATH`/`STATE_DB_PATH` lines stay unchanged — they derive from BASE_DIR.) + +Verify syntax: +```bash +.venv/Scripts/python.exe -m py_compile inbound_verify/paths.py +``` +Expected: no output (success). + +- [ ] **Step 3: Rewrite imports in state_store.py** + +In `inbound_verify/state_store.py`, replace: +```python +from paths import STATE_DB_PATH +``` +with: +```python +from inbound_verify.paths import STATE_DB_PATH +``` +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/state_store.py` → no output. + +- [ ] **Step 4: Rewrite imports + call sites in runtime.py** + +In `inbound_verify/runtime.py`, replace the import block: +```python +from paths import DOWNLOAD_DIR, CONFIG_PATH +import state_store +import site_shunxin +import site_baishi +import site_zto +import site_yunda +import site_anneng +import expected_undelivered # dispatch 的 compare 任务用 +``` +with: +```python +from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH +from inbound_verify import state_store +from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng +from inbound_verify import expected_undelivered # dispatch 的 compare 任务用 +``` + +Then **drop the `site_` prefix at every call site** (28 references — 18 in this file). Apply these 5 replacements (all occurrences): +- `site_shunxin.` → `shunxin.` +- `site_baishi.` → `baishi.` +- `site_zto.` → `zto.` +- `site_yunda.` → `yunda.` +- `site_anneng.` → `anneng.` + +Affected lines in runtime.py (for reference, all must change): 36, 37, 38, 39, 118, 139, 336, 408, 417, 463, 464, 467, 469, 470, 472, 473, 475, 476. + +`state_store.` and `expected_undelivered.` call sites stay UNCHANGED (names kept). Confirm the key handler block now reads: +```python +TASK_HANDLERS = { + ("顺心", "expected"): _web_handler("顺心", shunxin.shunxin_expected_download), + ("顺心", "actual"): _web_handler("顺心", shunxin.shunxin_actual_download), + ("顺心", "undelivered"): _site_undelivered_handler("顺心"), + ("百世", "undelivered"): _web_handler( + "百世", baishi.baishi_download_undelivered_data + ), + ("中通", "expected"): _web_handler("中通", zto.zto_expected_download), + ("中通", "actual"): _web_handler("中通", zto.zto_actual_download), + ("中通", "undelivered"): _site_undelivered_handler("中通"), + ("韵达", "expected"): _web_handler("韵达", yunda.yunda_expected_download), + ("韵达", "actual"): _web_handler("韵达", yunda.yunda_actual_download), + ("韵达", "undelivered"): _site_undelivered_handler("韵达"), + ("安能", "expected"): lambda ctx: anneng.anneng_expected_download(), + ("安能", "actual"): lambda ctx: anneng.anneng_actual_download(), + ("安能", "undelivered"): _site_undelivered_handler("安能"), + ("__compare__", "compare"): lambda ctx: (expected_undelivered.main() or True), +} +``` +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/runtime.py` → no output. + +- [ ] **Step 5: Rewrite imports in expected_undelivered.py** + +This module has TWO lazy `import state_store` statements (inside functions). Replace each occurrence of: +```python +import state_store +``` +with: +```python +from inbound_verify import state_store +``` +(There is no `from paths import` here — the module has its own `BASE/DOWNLOADS/OUTPUT` constants; that dedup is Tier 2, not now.) +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/expected_undelivered.py` → no output. + +- [ ] **Step 6: Rewrite imports + rename _cli→main in store.py** + +In `inbound_verify/store.py`, replace: +```python +from paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR +``` +with: +```python +from inbound_verify.paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR +``` +Replace: +```python +import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源） +``` +with: +```python +from inbound_verify import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源） +``` +(`eu.` call sites stay unchanged.) + +Rename the CLI entry: replace the function definition: +```python +def _cli(): +``` +with: +```python +def main(): +``` +And at the bottom replace: +```python +if __name__ == "__main__": + _cli() +``` +with: +```python +if __name__ == "__main__": + main() +``` +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/store.py` → no output. + +- [ ] **Step 7: Rewrite imports in all 5 site modules** + +In each of `inbound_verify/sites/{shunxin,baishi,zto,yunda,anneng}.py`, replace: +```python +from paths import DOWNLOAD_DIR, CONFIG_PATH +import state_store +``` +with: +```python +from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH +from inbound_verify import state_store +``` +(Each site module has exactly these two internal imports; `state_store.` call sites unchanged.) +Verify all five: +```bash +.venv/Scripts/python.exe -m py_compile inbound_verify/sites/shunxin.py inbound_verify/sites/baishi.py inbound_verify/sites/zto.py inbound_verify/sites/yunda.py inbound_verify/sites/anneng.py +``` +Expected: no output. + +- [ ] **Step 8: Rewrite imports + call sites + add main() in cli/router.py** + +In `inbound_verify/cli/router.py`, replace the import block (lines ~14–31): +```python +from paths import CONFIG_PATH +from runtime import ( + APP_SITES, + HEARTBEAT_INTERVAL, + dispatch_task, + launch_and_prepare, + run_heartbeat, +) + +import state_store + +# 各站点模块（自动化测试 + 比对用；任务派发在 runtime） +import site_shunxin +import site_baishi +import site_zto +import site_yunda +import site_anneng +import expected_undelivered +``` +with: +```python +from inbound_verify.paths import CONFIG_PATH +from inbound_verify.runtime import ( + APP_SITES, + HEARTBEAT_INTERVAL, + dispatch_task, + launch_and_prepare, + run_heartbeat, +) + +from inbound_verify import state_store + +# 各站点模块（自动化测试 + 比对用；任务派发在 runtime） +from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng +from inbound_verify import expected_undelivered +``` + +Drop the `site_` prefix at the 10 call sites (apply the same 5 replacements as Step 4). Affected lines: 60, 61, 64, 65, 68, 69, 72, 73, 86, 105. (`expected_undelivered.main()` at line 38 stays unchanged.) + +Add an entry function and update the `__main__` guard. Replace: +```python +if __name__ == "__main__": + run_multi_site_daemon() +``` +with: +```python +def main(): + """交互菜单模式入口。""" + run_multi_site_daemon() + + +if __name__ == "__main__": + main() +``` +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/router.py` → no output. + +- [ ] **Step 9: Rewrite imports + add main() in cli/server.py** + +In `inbound_verify/cli/server.py`, replace: +```python +from paths import DOWNLOAD_DIR, OUTPUT_DIR +import state_store +from runtime import ( + HEARTBEAT_INTERVAL, + TASK_HANDLERS, + dispatch_task, + launch_and_prepare, + run_heartbeat, +) +``` +with: +```python +from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR +from inbound_verify import state_store +from inbound_verify.runtime import ( + HEARTBEAT_INTERVAL, + TASK_HANDLERS, + dispatch_task, + launch_and_prepare, + run_heartbeat, +) +``` + +Replace the bottom entry block: +```python +if __name__ == "__main__": + uvicorn.run(app, host="0.0.0.0", port=8000) +``` +with: +```python +def main(): + """服务模式入口。传字符串导入路径（规范写法；不开 reload/workers 时进程内 import，行为等价）。""" + uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000) + + +if __name__ == "__main__": + main() +``` +Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/server.py` → no output. + +- [ ] **Step 10: Clean up stale filename comments** + +Cosmetic but keeps grep clean (Step 12 depends on it). In each `inbound_verify/sites/*.py`, update the line-1 header `# site_xxx.py` → `# sites/xxx.py`. In `inbound_verify/sites/anneng.py`, update the standalone-run comment near the top: +```python +# .venv/Scripts/python.exe site_anneng.py +``` +→ +```python +# python -m inbound_verify.sites.anneng expected # 或 actual +``` +And the comment at the `CDP_PORT` line referencing "独立运行 site_anneng.py" — update to "独立运行（python -m inbound_verify.sites.anneng）". + +- [ ] **Step 11: Black-format all changed files** + +```bash +.venv/Scripts/python.exe -m black inbound_verify +``` +Expected: `reformatted ...` / `left unchanged` lines, exit 0. + +- [ ] **Step 12: VERIFICATION GATE — run all five checks** + +**12a. compileall (syntax across whole package):** +```bash +.venv/Scripts/python.exe -m compileall inbound_verify +``` +Expected: no errors. + +**12b. Import smoke (catches every wrong import path / missed rewrite):** +```bash +.venv/Scripts/python.exe -c "import inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, inbound_verify.state_store; print('import smoke OK')" +``` +Expected: `import smoke OK`. (The three entries transitively import sites + expected_undelivered.) + +**12c. No stale `site_` references:** +```bash +grep -rn "site_shunxin\|site_baishi\|site_zto\|site_yunda\|site_anneng" inbound_verify || echo "no stale site_ refs OK" +``` +Expected: `no stale site_ refs OK`. + +**12d. No stale bare flat imports:** +```bash +grep -rnE "^from paths import|^from runtime import|^import site_|^import state_store$|^import expected_undelivered$" inbound_verify || echo "no stale flat imports OK" +``` +Expected: `no stale flat imports OK`. + +**12e. paths anchor points at project root + DB still connects:** +```bash +.venv/Scripts/python.exe -c "from inbound_verify.paths import BASE_DIR; print('BASE_DIR', BASE_DIR)" +.venv/Scripts/python.exe -c "from inbound_verify.store import _connect, _load_pg_config; c=_load_pg_config(); conn=_connect(c['dbname']); print('DB OK', conn.info.server_version); conn.close()" +``` +Expected: `BASE_DIR` prints the `InboundVerify` project root (the dir containing `config.yaml`); `DB OK `. + +> If 12b fails with ModuleNotFoundError for a site module, run `.venv/Scripts/python.exe -m pip install -e .` again (editable finder refresh) and retry. If 12d still shows a line, that import was missed — rewrite it per Step 4/8 rules. + +- [ ] **Step 13: Commit (only after user confirms)** + +```bash +git add -A inbound_verify +git commit -m "refactor: move flat modules into inbound_verify package (Tier 1, behavior-identical) + +- relocate 12 root .py into inbound_verify/ (sites/, cli/ subpackages) +- rewrite all internal imports to package-qualified +- fix paths.py BASE_DIR to anchor at project root +- add main() entry wrappers; register console_scripts +- drop site_ prefix on site modules; keep state_store/expected_undelivered names + +Co-Authored-By: Claude " +``` +> Do NOT commit until the user says to. This commit is the **safety baseline**; the manual end-to-end test (spec §7 gate) runs against this state before any Tier 2/Tier 3 work. + +--- + +## Task 3: Docs sync (README + CLAUDE.md) + +**Files:** +- Modify: `README.md` (§二 directory tree, §三 env prep, §五 run) +- Modify: `CLAUDE.md` (常用命令 section) + +**Interfaces:** +- Consumes: the completed package from Task 2 (docs must describe the real new layout/commands). +- Produces: documentation matching the new invocation model. No code impact. + +- [ ] **Step 1: Update README §二 directory tree** + +Replace the tree block (README lines ~33–49) with the actual new layout: +``` +InboundVerify/ +├── pyproject.toml # 打包 + 依赖 + console_scripts +├── inbound_verify/ # 源码包 +│ ├── paths.py runtime.py state_store.py expected_undelivered.py store.py +│ ├── sites/ shunxin / baishi / zto / yunda / anneng +│ └── cli/ router（交互菜单）/ server（FastAPI 服务） +├── config.example.yaml / config.yaml +├── schema.sql +├── requirements.txt # pyproject 的静态镜像 +├── downloads/ output/ state/ +└── docs/ +``` + +- [ ] **Step 2: Update README §三 env prep — add editable install** + +In the env-prep command block (README lines ~58–71), after `pip install -r requirements.txt`, add: +```bash +# 4. 以可编辑模式安装本包（注册 inbound-verify 等命令） +pip install -e . +``` +(renumber the subsequent `cp config.example.yaml config.yaml` step). + +- [ ] **Step 3: Update README §五 run — new commands** + +Replace `python main_router.py` with: +```bash +# 交互菜单（任选其一） +python -m inbound_verify.cli.router +# 或装包后：inbound-verify +``` + +- [ ] **Step 4: Update CLAUDE.md 常用命令** + +In the 常用命令 section, change every `.venv/Scripts/python.exe .py` to the package form: +- `main_router.py` → `python -m inbound_verify.cli.router` (or `inbound-verify`) +- `server.py` → `python -m inbound_verify.cli.server` (or `inbound-verify-server`) +- `db_store.py createdb|init|ingest|all` → `python -m inbound_verify.store createdb|init|ingest|all` (or `inbound-verify-db ...`) +- `site_anneng.py expected|actual` → `python -m inbound_verify.sites.anneng expected|actual` +- `black`/`py_compile` targets → package paths (e.g. `-m black inbound_verify`) + +Add a one-liner near the top of that section: `首次/拉取新代码后需 .venv/Scripts/python.exe -m pip install -e .`。 + +- [ ] **Step 5: Verify docs render + commands are real** + +```bash +grep -n "main_router.py\|python server.py\|python db_store.py\|site_anneng.py\|site_shunxin" README.md CLAUDE.md || echo "no stale old-path commands OK" +``` +Expected: `no stale old-path commands OK` (every old invocation updated). + +- [ ] **Step 6: Commit (only after user confirms)** + +```bash +git add README.md CLAUDE.md +git commit -m "docs: update README and CLAUDE.md for package layout and console_scripts + +Co-Authored-By: Claude " +``` + +--- + +## Self-Review (completed) + +- **Spec coverage:** spec §3 (layout) → Task 1+2; §4 (paths anchor) → Task 2 Step 2; §5 (import rules) → Task 2 Steps 3–9; §6 (entry/packaging) → Task 1 Step 2 + Task 2 Steps 6/8/9; §7 (verification gate) → Task 2 Step 12; §10 (docs) → Task 3. All covered. +- **Placeholder scan:** none — every step has exact code or an exact command with expected output. The 28 call-site rewrites are given as a deterministic prefix-drop rule + enumerated line numbers + verification grep (complete, not a placeholder). +- **Type/name consistency:** kept-module names (`state_store`, `expected_undelivered`, `runtime`, `paths`) used consistently across all import rewrites and call sites; renamed entries (`store`, `cli/router`, `cli/server`) consistent with pyproject `[project.scripts]`. `main()` signature consistent across router/server/store and console_scripts. diff --git a/docs/superpowers/specs/2026-07-23-package-restructure-design.md b/docs/superpowers/specs/2026-07-23-package-restructure-design.md new file mode 100644 index 0000000..587bb90 --- /dev/null +++ b/docs/superpowers/specs/2026-07-23-package-restructure-design.md @@ -0,0 +1,274 @@ +# InboundVerify 包化重构设计 + +- **日期**:2026-07-23 +- **方案**:C(规范)—— `pyproject.toml` + `[project.scripts]` + 根级包 `inbound_verify/`,根目录不留 `.py` 薄壳 +- **力度**:彻底(建包 + 定点小改进 + 大文件拆分),但**大文件拆分(Tier 3)前置手动测试闸门** +- **状态**:已与用户对齐,待 spec 评审 + +--- + +## 1. 背景与目标 + +当前 13 个 Python 模块(共约 6500 行)全部平铺在仓库根目录,随脚本量增长结构混乱。本设计将其重组为一个规范的、可 `pip install -e .` 安装的 Python 包。 + +**目标** + +1. 扁平脚本 → `inbound_verify/` 包(`sites/`、`cli/` 两个子包,其余平铺包根,避免一层只放一两个文件的过度嵌套)。 +2. 全部内部 import 改为包内绝对引用。 +3. `pyproject.toml` 打包,`[project.scripts]` 暴露命令入口;**根目录不留 `.py` 薄壳**(规范要求)。 +4. 顺手抽离共享配置、消重(定点小改进,Tier 2)。 +5. 大文件拆分作为**最后、可选、风险隔离**的阶段(Tier 3),且必须先过手动测试闸门。 + +**已确认约束** + +- InboundVerify 是 git **子模块**;改动在子模块内提交,父仓库 `LogisticsHubIPA` 仅跟踪子模块指针。 +- **不加测试套件**(用户决定);验证靠 `compileall` + 导入冒烟 + grep 查残留引用 + 手动端到端测试。 +- 入口走规范:无根薄壳;一次性 `pip install -e .` 后用命令或 `python -m` 启动。 +- **不做 src/ 布局**(内部工具、不发 PyPI、无测试,边际价值有限)。 +- 本仓库约定:**不自动提交 / 不自动推送**;改动等用户明确说"提交"再 commit/push(本约定覆盖 brainstorming 默认的"写完即提交")。 + +--- + +## 2. 现状分析 + +### 2.1 依赖分层(自底向上) + +``` +paths ← 万物之基(被所有模块 import) +state_store ← SQLite 状态持久化 +expected_undelivered ← 离线比对(兼职存站点/文件名/列映射共享配置,被 db_store 复用 —— 耦合点) +site_*.py(×5) ← 各依赖 paths + state_store +runtime ← 编排核心(启动/派发/心跳,依赖上面全部) +main_router / server / db_store ← 三入口(均有 __main__/CLI) +``` + +### 2.2 关键发现 + +1. **`paths.py` 是地雷**:`BASE_DIR = dirname(abspath(__file__))`——paths.py 在哪,根就在哪。搬进子目录后必须改为上跳一级,否则 `downloads/`、`output/`、`config.yaml`、`state/state.db`、`schema.sql` 全部跑偏。 +2. **import 全是扁平顶层**(`import site_shunxin`、`from paths import …`),且**没有任何地方按字符串名引用模块**(`TASK_HANDLERS` 用函数引用、`dispatch_task` 用 site/kind 字典),改写纯机械。 +3. **重复代码**:`with_retry`、`_remove_if_exists` 在 5 个站点逐字重复(CLAUDE.md 注明"现阶段刻意不优化结构")。 +4. **路径常量重复**:`expected_undelivered.py` 自带 `BASE/DOWNLOADS/OUTPUT`,与 `paths.py` 重复。 +5. **耦合**:`db_store` 为读站点配置而 `import expected_undelivered`(为了配置而依赖整个比对引擎)。 + +--- + +## 3. 目标目录结构 + +``` +InboundVerify/ +├── pyproject.toml # 新增:打包 + 依赖 + console_scripts +├── config.example.yaml +├── config.yaml # 留根(gitignored) +├── schema.sql # 留根(store 按绝对路径读) +├── requirements.txt # 保留为静态镜像(pyproject 为准) +├── README.md / CLAUDE.md / .gitignore +├── downloads/ output/ state/ # 运行时数据,留根 +├── docs/ +└── inbound_verify/ # ← 包 + ├── __init__.py + ├── __main__.py # 可选:python -m inbound_verify → 交互菜单 + ├── paths.py # 锚点改为指向项目根(§4) + ├── config.py # 新增(Tier2):集中 load_config() + ├── domain.py # 新增(Tier2):从 expected_undelivered 抽出的共享站点/文件/列映射 + ├── runtime.py # ← runtime.py(编排核心,整体保留不拆) + ├── state_store.py # ← state_store.py(保留名,仅搬运) + ├── expected_undelivered.py # ← 搬运;Tier2 改名 compare.py + 抽 domain.py + ├── store.py # ← db_store.py(叶子,改名无 churn) + ├── sites/ + │ ├── __init__.py + │ ├── shunxin.py baishi.py zto.py yunda.py # ← site_*.py(去 site_ 前缀) + │ └── anneng.py # ← site_anneng.py(Tier3 可选再拆成子包) + └── cli/ + ├── __init__.py + ├── router.py # ← main_router.py(叶子,改名无 churn) + └── server.py # ← server.py(叶子) +``` + +### 3.1 搬运映射表(全部 `git mv` 保历史) + +| 现在 | Tier 1 后 | 调用点改动 | +|---|---|---| +| `paths.py` | `inbound_verify/paths.py` | 无(仅改 import 行 + 锚点) | +| `runtime.py` | `inbound_verify/runtime.py` | 无(保留名) | +| `state_store.py` | `inbound_verify/state_store.py` | 无(**保留名**,仅改 import 行) | +| `expected_undelivered.py` | `inbound_verify/expected_undelivered.py` | 无(Tier1 保留名;Tier2 改名 compare) | +| `db_store.py` | `inbound_verify/store.py` | 无(叶子,无人 import) | +| `main_router.py` | `inbound_verify/cli/router.py` | 无(叶子) | +| `server.py` | `inbound_verify/cli/server.py` | 无(叶子) | +| `site_{shunxin,baishi,zto,yunda,anneng}.py` | `inbound_verify/sites/{…}.py`(去 `site_` 前缀) | 改 `runtime` + `main_router` 调用点;**grep 查残留引用兜底** | + +> **命名策略(降低无测试下的风险)**:Tier 1 只对**被多处裸名引用**的模块(`state_store`、`expected_undelivered`、`runtime`、`paths`)**保留原名**,做到"仅改 import 行、零调用点改动";只对**叶子入口**(`db_store`/`main_router`/`server`)和**站点文件**(去 `site_` 前缀)做改名。`expected_undelivered` 的改名(`→ compare.py`)推迟到 Tier 2——那时本就要为抽 `domain.py` 重做该文件及其调用方,把改名 churn 并入一个已经在改的批次。这样 Tier 1 的纯搬运可被 `compileall` + 导入冒烟 + grep 充分验证,不依赖手测。 + +--- + +## 4. `paths.py` 锚点修正(唯一地雷,必须改对) + +包搬进 `inbound_verify/` 后,`__file__` 多降一级。要把 `BASE_DIR` 继续指回项目根: + +```python +# inbound_verify/paths.py +import os +# __file__ = .../InboundVerify/inbound_verify/paths.py +# 上两级 = .../InboundVerify (= 项目根,config.yaml/downloads/schema.sql 所在) +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +DOWNLOAD_DIR = os.path.join(BASE_DIR, "downloads") +OUTPUT_DIR = os.path.join(BASE_DIR, "output") +CONFIG_PATH = os.path.join(BASE_DIR, "config.yaml") +STATE_DB_PATH = os.path.join(BASE_DIR, "state", "state.db") +``` + +editable 安装不会移动文件,`__file__` 仍指向源码树,两级上跳稳定指向项目根。`store.py` 的 `SCHEMA_PATH = join(BASE_DIR, "schema.sql")` 自动跟着对。 + +--- + +## 5. import 改写规则 + +```python +from paths import ... → from inbound_verify.paths import ... +import state_store → from inbound_verify import state_store # 保留名,调用点 state_store.X 不变 +from runtime import (...) → from inbound_verify.runtime import (...) +import site_shunxin → from inbound_verify.sites import shunxin # 5 站同理,调用点改 shunxin.X +import expected_undelivered → from inbound_verify import expected_undelivered # Tier1 保留名 +import expected_undelivered as eu → from inbound_verify import expected_undelivered as eu # eu.X 不变 +``` + +(Tier 2 后,`expected_undelivered` 改名 `compare`,`db_store` 的共享配置改 `from inbound_verify import domain`。) + +--- + +## 6. 入口与打包 + +### 6.1 `main()` 包装(根目录不留薄壳) + +```python +# inbound_verify/cli/server.py 末尾 +def main(): + uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000) +if __name__ == "__main__": + main() +``` + +> `uvicorn.run` 改传**字符串** `"inbound_verify.cli.server:app"`(规范写法;不开 `reload`/`workers` 时仍在当前进程 import,行为等价)。`router.py` 套 `main()` 调 `run_multi_site_daemon()`;`store.py` 已有 `_cli`,改名为 `main`。 + +### 6.2 `pyproject.toml`(骨架) + +```toml +[build-system] +requires = ["setuptools>=68"] +build-backend = "setuptools.build_meta" + +[project] +name = "inbound-verify" +version = "0.1.0" +requires-python = ">=3.10" +dependencies = ["pandas>=2.0.0", "playwright>=1.40.0", "openpyxl>=3.1.0", + "PyYAML>=6.0", "websocket-client>=1.0.0", "fastapi>=0.110.0", + "uvicorn>=0.27.0", "apscheduler>=3.10.0", "psycopg[binary]>=3.1"] + +[project.scripts] +inbound-verify = "inbound_verify.cli.router:main" +inbound-verify-server = "inbound_verify.cli.server:main" +inbound-verify-db = "inbound_verify.store:main" + +[tool.setuptools.packages.find] +include = ["inbound_verify*"] +``` + +### 6.3 装包后三种启动方式(都汇到同一个 `main()`) + +```bash +pip install -e . # 一次性 +inbound-verify-server # 命令 +python -m inbound_verify.cli.server # 兜底 +uvicorn inbound_verify.cli.server:app # 生产最标准(端口/worker 命令行控) +``` + +--- + +## 7. 执行顺序与验证闸门(为"无测试"量身) + +**安全原则**:Tier 1 必须先独立完成并验证为**行为等价**,再做任何动逻辑的改动。每段后必验证。验证职责分清——自动化部分我跑,端到端部分需你跑(真实登录/凭据/安能 Electron 只有你能提供)。 + +``` +Tier 1 纯搬运(行为零改变) + ├─ 我的自动验证: ① compileall 全过 ② 导入冒烟 ③ grep 查残留 site_ 引用 ④ DB 连通 + └─ [等用户说"提交"] commit ← 安全基线 +Tier 2 domain 抽取 / config 集中 / 路径消重 / expected_undelivered 改名 compare + ├─ 我的自动验证: 同上 + └─ [等用户说"提交"] commit +═══════ 手动测试闸门(你跑)═══════ + 全链路端到端: + 起 inbound-verify → 登录 5 站(顺心双账号)→ 各站下载 → 比对(菜单 9) + → inbound-verify-db ingest → 核对 output/应到未到数据.xlsx 与 PostgreSQL 三张表 + 通过? 否 → 修到通过 + 是 ↓ +Tier 3 此时再定:anneng 拆不拆 / with_retry 抽不抽 base.py + └─ 每项后重跑我的自动验证,你按需复测 +``` + +### 7.1 自动验证命令清单(我每个 Tier 后都跑) + +```bash +.venv/Scripts/python.exe -m compileall inbound_verify # ① 语法 +.venv/Scripts/python.exe -c "import inbound_verify.cli.router, \ + inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, \ + inbound_verify.state_store" # ② 导入冒烟(三个入口会传递导入 sites/expected_undelivered 等; + # Tier2 改名后 expected_undelivered → compare,无需单独显式导入) +grep -rn "site_shunxin\|site_baishi\|site_zto\|site_yunda\|site_anneng" \ + inbound_verify || echo "无残留 site_ 引用 ✓" # ③ 改名残留 +.venv/Scripts/python.exe -c "from inbound_verify.store import _connect, _load_pg_config; \ + c=_load_pg_config(); conn=_connect(c['dbname']); print('DB OK', conn.info.server_version)" # ④ DB +``` + +--- + +## 8. 各 Tier 内容 + +### Tier 1 — 纯搬运(行为零改变) + +1. 建包骨架 + 空白 `__init__.py`(`sites/`、`cli/`)。 +2. `git mv` §3.1 表中所有文件到新位置。 +3. 按 §5 规则机械改写全部 import;站点改名后改 `runtime` + `main_router` 调用点(`shunxin.shunxin_expected_download(...)` 等)。 +4. 改 `paths.py` 锚点(§4)。 +5. 三个入口加 `main()`(`store` 的 `_cli` 改名 `main`)。 +6. 写 `pyproject.toml`,`.venv` 内 `pip install -e .`。 +7. 跑 §7.1 四项自动验证。 +8. 等用户说"提交"→ commit(子模块内)。 + +### Tier 2 — 定点小改进(每项后跑 §7.1) + +- **抽 `domain.py`**:把 `expected_undelivered.py` 里的 `STATIONS`/`_site_cfg`/`ALL_REPORT_SITES`/`SITE_UNDELIVERED_FILE`/`BAISHI_FILE`/`BAISHI_COLUMNS`/`arrived_pieces_*` 移到 `inbound_verify/domain.py`;`store.py` 从依赖整个比对引擎改为 `from inbound_verify import domain`。**接缝最干净、收益明确,推荐做。** +- **`expected_undelivered.py` → `compare.py`** 改名,更新调用方(`store` 的 `as eu`、`runtime`、`cli/router`)。 +- **加 `config.py`**:集中 `load_config()`(带缓存),各站点把自家的 `yaml.safe_load(open(CONFIG_PATH))` 换掉。 +- **消重路径常量**:`compare.py` 自带那份 `BASE/DOWNLOADS/OUTPUT` 改成引用 `paths.py`。 + +### Tier 3 — 大文件拆分(手动测试闸门之后;具体决策推迟到闸门) + +- **`anneng.py` → `sites/anneng/` 子包**(`cdp.py`/`nav.py`/`expected.py`/`actual.py`):接缝分层清晰,但共享可变状态多(`CDP_PORT` 全局被 `set_cdp_port` 改、各种 URL hint、僵尸 tab 逻辑),CLAUDE.md 标注为脚gun。**倾向不拆**(1375 行虽大但是内聚的 CDP 驱动,强拆无测试网兜底风险高)——最终在闸门后定。 +- **抽 `sites/base.py`**:`with_retry` / `_remove_if_exists` 在 5 站逐字重复;原作者在 CLAUDE.md 写"现阶段刻意不优化结构"。**抽不抽,闸门后定。** + +--- + +## 9. 待决策(推迟到手动测试闸门) + +| 决策 | 默认倾向 | 何时定 | +|---|---|---| +| `anneng.py` 拆不拆子包 | **不拆** | 手动测试通过后 | +| `with_retry`/`_remove_if_exists` 抽 `base.py` | 待定 | 手动测试通过后 | +| `requirements.txt` 留还是删 | **留静态镜像**(注明 pyproject 为准) | 随时可改 | + +--- + +## 10. 文档同步(Tier 1 必做) + +- **README.md**:第二节目录树、第三节环境准备(加 `pip install -e .`)、第五节运行(改新命令)。 +- **CLAUDE.md**:常用命令段全部改 `python -m inbound_verify…` / `inbound-verify…`,补 `pip install -e .` 与 `playwright install chromium`、`black`/`py_compile` 的包内路径写法。 + +--- + +## 11. 范围外 + +- **不做** src/ 布局。 +- **不做** 给站点流程加 mock 测试。 +- **不自动** commit/push(等用户明确指示)。 +- 父仓库 `LogisticsHubIPA` 的子模块指针更新,是父仓库的单独一步,不在本 spec 范围。