LogisticsHubIPA/InboundVerify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update README/CLAUDE.md for package layout; add Tier 1 spec and plan

Browse Source 
Rewrite run commands to python -m inbound_verify.* (and console_script aliases); add pip install -e . to env prep; refresh the directory tree. Also commit the design spec and Tier 1 implementation plan under docs/superpowers/.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Misaka_Company
2026-07-23 12:21:45 +08:00
parent 7065b88269
commit a07b9b435b
4 changed files with 926 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

579
docs/superpowers/plans/2026-07-23-tier1-package-move.md Normal file
View File

@@ -0,0 +1,579 @@
# Tier 1: Package Move Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move all 12 flat root-level Python modules into an installable `inbound_verify/` package (sites/, cli/ subpackages), rewrite every internal import to package-qualified form, and expose three console_script entry points — with **zero behavior change**.
**Architecture:** Pure mechanical relocation (git mv preserves history) + import rewrite + paths.py anchor fix + entry-point `main()` wrappers + `pyproject.toml` packaging. No logic changes. Module names kept for heavily-referenced modules (`state_store`, `expected_undelivered`, `runtime`, `paths`) to avoid call-site churn; only leaf entries (`db_store→store`, `main_router→cli/router`, `server→cli/server`) and site files (drop `site_` prefix) are renamed.
**Tech Stack:** Python ≥3.10, setuptools (PEP 517/621), pip editable install, psycopg3, FastAPI/uvicorn, Playwright.
## Global Constraints
- **Python ≥ 3.10** (`requires-python = ">=3.10"` in pyproject).
- **Package** import name `inbound_verify`; **distribution** name `inbound-verify`.
- **Dependency floors** (verbatim from spec): `pandas>=2.0.0`, `playwright>=1.40.0`, `openpyxl>=3.1.0`, `PyYAML>=6.0`, `websocket-client>=1.0.0`, `fastapi>=0.110.0`, `uvicorn>=0.27.0`, `apscheduler>=3.10.0`, `psycopg[binary]>=3.1`.
- **No test suite** (user decision). Verification = `compileall` + import smoke + grep-for-stale-refs + DB connectivity. No pytest.
- **Behavior must not change** in Tier 1 — pure move.
- **No auto-commit/push.** Every commit step below runs ONLY after the user explicitly says "提交/commit". Commit messages in English.
- **Black-format** every changed `.py` (global rule).
- All changes are **inside the `InboundVerify` git submodule**; the parent repo pointer bump is a separate parent-repo step, out of scope.
- All commands run from the `InboundVerify/` directory using the venv interpreter `.venv/Scripts/python.exe` (Windows; no activation needed).
**Reference spec:** `docs/superpowers/specs/2026-07-23-package-restructure-design.md` (§3 mapping table, §4 paths anchor, §5 import rules, §6 entry/packaging, §7 verification gate).
---
## File Structure (what each file becomes responsible for)
```
inbound_verify/
├── __init__.py # empty (package marker)
├── paths.py # path anchors → PROJECT ROOT (one dir above package)
├── runtime.py # orchestration core (unchanged logic)
├── state_store.py # SQLite state (unchanged; name kept)
├── expected_undelivered.py# offline compare (unchanged; name kept — Tier 2 renames to compare)
├── store.py # PostgreSQL persist + main() (was db_store.py)
├── sites/
│ ├── __init__.py # empty
│ ├── shunxin.py # (was site_shunxin.py)
│ ├── baishi.py # (was site_baishi.py)
│ ├── zto.py # (was site_zto.py)
│ ├── yunda.py # (was site_yunda.py)
│ └── anneng.py # (was site_anneng.py)
└── cli/
├── __init__.py # empty
├── router.py # interactive menu + main() (was main_router.py)
└── server.py # FastAPI service + main() (was server.py)
```
Root keeps: `pyproject.toml` (new), `config.yaml`, `config.example.yaml`, `schema.sql`, `requirements.txt`, `README.md`, `CLAUDE.md`, `docs/`, `downloads/`, `output/`, `state/`.
---
## Task 1: Package scaffold + pyproject + editable install
**Files:**
- Create: `inbound_verify/__init__.py`, `inbound_verify/sites/__init__.py`, `inbound_verify/cli/__init__.py`
- Create: `pyproject.toml`
**Interfaces:**
- Produces: an importable (near-empty) `inbound_verify` package + console_script registration. The flat root scripts remain 100% functional after this task (untouched).
- [ ] **Step 1: Create package marker files**
Create three empty files:
- `inbound_verify/__init__.py`
- `inbound_verify/sites/__init__.py`
- `inbound_verify/cli/__init__.py`
Each is a single comment line:
```python
# inbound_verify package
```
- [ ] **Step 2: Write pyproject.toml**
Create `pyproject.toml`:
```toml
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "inbound-verify"
version = "0.1.0"
description = "物流到货数据自动下载与应到未到核对工具"
requires-python = ">=3.10"
dependencies = [
"pandas>=2.0.0",
"playwright>=1.40.0",
"openpyxl>=3.1.0",
"PyYAML>=6.0",
"websocket-client>=1.0.0",
"fastapi>=0.110.0",
"uvicorn>=0.27.0",
"apscheduler>=3.10.0",
"psycopg[binary]>=3.1",
]
[project.scripts]
inbound-verify = "inbound_verify.cli.router:main"
inbound-verify-server = "inbound_verify.cli.server:main"
inbound-verify-db = "inbound_verify.store:main"
[tool.setuptools.packages.find]
include = ["inbound_verify*"]
```
- [ ] **Step 3: Editable-install into the existing venv**
Run:
```bash
.venv/Scripts/python.exe -m pip install -e .
```
Expected: `Successfully installed inbound-verify-0.1.0` (deps already satisfied from earlier install — no network needed).
- [ ] **Step 4: Verify package imports**
Run:
```bash
.venv/Scripts/python.exe -c "import inbound_verify, inbound_verify.sites, inbound_verify.cli; print('package OK')"
```
Expected output: `package OK`
- [ ] **Step 5: Commit (only after user confirms)**
```bash
git add inbound_verify/__init__.py inbound_verify/sites/__init__.py inbound_verify/cli/__init__.py pyproject.toml
git commit -m "chore: scaffold inbound_verify package and pyproject
Co-Authored-By: Claude <noreply@anthropic.com>"
```
> Do NOT commit until the user says to.
---
## Task 2: Atomic move — relocate, rewrite imports, fix anchor, wire mains, verify
**Files:**
- Move (git mv): all 12 root `.py` modules → package locations (see Step 1)
- Modify: `inbound_verify/paths.py` (anchor), and import lines + call sites in every moved module
- Modify: entry `main()` in `cli/router.py`, `cli/server.py`, `store.py`
**Interfaces:**
- Consumes: the scaffold from Task 1 (package importable + editable install active).
- Produces: a fully functional `inbound_verify` package invokable via `python -m inbound_verify.cli.router`, `python -m inbound_verify.cli.server`, `python -m inbound_verify.store`, or the three console_scripts. Old root `.py` files are gone. The flat root scripts no longer exist — invocation switches to package form.
> **Why this is one task:** in a flat-import codebase, moving `paths.py` (imported by everyone) immediately breaks every importer until ALL moves + rewrites are complete. There is no intermediate state that imports cleanly, so the whole move is one atomic unit verified by the gate at the end. Each file-edit step below is followed by `py_compile` of that file to catch syntax errors as we go.
- [ ] **Step 1: Relocate all 12 modules with git mv (history preserved)**
From the `InboundVerify/` directory:
```bash
git mv paths.py inbound_verify/paths.py
git mv runtime.py inbound_verify/runtime.py
git mv state_store.py inbound_verify/state_store.py
git mv expected_undelivered.py inbound_verify/expected_undelivered.py
git mv db_store.py inbound_verify/store.py
git mv main_router.py inbound_verify/cli/router.py
git mv server.py inbound_verify/cli/server.py
git mv site_shunxin.py inbound_verify/sites/shunxin.py
git mv site_baishi.py inbound_verify/sites/baishi.py
git mv site_zto.py inbound_verify/sites/zto.py
git mv site_yunda.py inbound_verify/sites/yunda.py
git mv site_anneng.py inbound_verify/sites/anneng.py
```
After this the tree is temporarily broken (imports unresolved) — expected. Continue.
- [ ] **Step 2: Fix paths.py anchor to point at project root**
In `inbound_verify/paths.py`, replace:
```python
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
```
with:
```python
# __file__ = <root>/inbound_verify/paths.py → 上两级 = 项目根
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
```
(`DOWNLOAD_DIR`/`OUTPUT_DIR`/`CONFIG_PATH`/`STATE_DB_PATH` lines stay unchanged — they derive from BASE_DIR.)
Verify syntax:
```bash
.venv/Scripts/python.exe -m py_compile inbound_verify/paths.py
```
Expected: no output (success).
- [ ] **Step 3: Rewrite imports in state_store.py**
In `inbound_verify/state_store.py`, replace:
```python
from paths import STATE_DB_PATH
```
with:
```python
from inbound_verify.paths import STATE_DB_PATH
```
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/state_store.py` → no output.
- [ ] **Step 4: Rewrite imports + call sites in runtime.py**
In `inbound_verify/runtime.py`, replace the import block:
```python
from paths import DOWNLOAD_DIR, CONFIG_PATH
import state_store
import site_shunxin
import site_baishi
import site_zto
import site_yunda
import site_anneng
import expected_undelivered # dispatch 的 compare 任务用
```
with:
```python
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
from inbound_verify import state_store
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
from inbound_verify import expected_undelivered # dispatch 的 compare 任务用
```
Then **drop the `site_` prefix at every call site** (28 references — 18 in this file). Apply these 5 replacements (all occurrences):
- `site_shunxin.` → `shunxin.`
- `site_baishi.` → `baishi.`
- `site_zto.` → `zto.`
- `site_yunda.` → `yunda.`
- `site_anneng.` → `anneng.`
Affected lines in runtime.py (for reference, all must change): 36, 37, 38, 39, 118, 139, 336, 408, 417, 463, 464, 467, 469, 470, 472, 473, 475, 476.
`state_store.` and `expected_undelivered.` call sites stay UNCHANGED (names kept). Confirm the key handler block now reads:
```python
TASK_HANDLERS = {
("顺心", "expected"): _web_handler("顺心", shunxin.shunxin_expected_download),
("顺心", "actual"): _web_handler("顺心", shunxin.shunxin_actual_download),
("顺心", "undelivered"): _site_undelivered_handler("顺心"),
("百世", "undelivered"): _web_handler(
"百世", baishi.baishi_download_undelivered_data
),
("中通", "expected"): _web_handler("中通", zto.zto_expected_download),
("中通", "actual"): _web_handler("中通", zto.zto_actual_download),
("中通", "undelivered"): _site_undelivered_handler("中通"),
("韵达", "expected"): _web_handler("韵达", yunda.yunda_expected_download),
("韵达", "actual"): _web_handler("韵达", yunda.yunda_actual_download),
("韵达", "undelivered"): _site_undelivered_handler("韵达"),
("安能", "expected"): lambda ctx: anneng.anneng_expected_download(),
("安能", "actual"): lambda ctx: anneng.anneng_actual_download(),
("安能", "undelivered"): _site_undelivered_handler("安能"),
("__compare__", "compare"): lambda ctx: (expected_undelivered.main() or True),
}
```
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/runtime.py` → no output.
- [ ] **Step 5: Rewrite imports in expected_undelivered.py**
This module has TWO lazy `import state_store` statements (inside functions). Replace each occurrence of:
```python
import state_store
```
with:
```python
from inbound_verify import state_store
```
(There is no `from paths import` here — the module has its own `BASE/DOWNLOADS/OUTPUT` constants; that dedup is Tier 2, not now.)
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/expected_undelivered.py` → no output.
- [ ] **Step 6: Rewrite imports + rename _cli→main in store.py**
In `inbound_verify/store.py`, replace:
```python
from paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
```
with:
```python
from inbound_verify.paths import BASE_DIR, CONFIG_PATH, DOWNLOAD_DIR
```
Replace:
```python
import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源）
```
with:
```python
from inbound_verify import expected_undelivered as eu # 复用站点 / 文件名 / 列映射 / 基号口径（单一来源）
```
(`eu.` call sites stay unchanged.)
Rename the CLI entry: replace the function definition:
```python
def _cli():
```
with:
```python
def main():
```
And at the bottom replace:
```python
if __name__ == "__main__":
_cli()
```
with:
```python
if __name__ == "__main__":
main()
```
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/store.py` → no output.
- [ ] **Step 7: Rewrite imports in all 5 site modules**
In each of `inbound_verify/sites/{shunxin,baishi,zto,yunda,anneng}.py`, replace:
```python
from paths import DOWNLOAD_DIR, CONFIG_PATH
import state_store
```
with:
```python
from inbound_verify.paths import DOWNLOAD_DIR, CONFIG_PATH
from inbound_verify import state_store
```
(Each site module has exactly these two internal imports; `state_store.` call sites unchanged.)
Verify all five:
```bash
.venv/Scripts/python.exe -m py_compile inbound_verify/sites/shunxin.py inbound_verify/sites/baishi.py inbound_verify/sites/zto.py inbound_verify/sites/yunda.py inbound_verify/sites/anneng.py
```
Expected: no output.
- [ ] **Step 8: Rewrite imports + call sites + add main() in cli/router.py**
In `inbound_verify/cli/router.py`, replace the import block (lines ~14–31):
```python
from paths import CONFIG_PATH
from runtime import (
APP_SITES,
HEARTBEAT_INTERVAL,
dispatch_task,
launch_and_prepare,
run_heartbeat,
)
import state_store
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
import site_shunxin
import site_baishi
import site_zto
import site_yunda
import site_anneng
import expected_undelivered
```
with:
```python
from inbound_verify.paths import CONFIG_PATH
from inbound_verify.runtime import (
APP_SITES,
HEARTBEAT_INTERVAL,
dispatch_task,
launch_and_prepare,
run_heartbeat,
)
from inbound_verify import state_store
# 各站点模块（自动化测试 + 比对用；任务派发在 runtime）
from inbound_verify.sites import shunxin, baishi, zto, yunda, anneng
from inbound_verify import expected_undelivered
```
Drop the `site_` prefix at the 10 call sites (apply the same 5 replacements as Step 4). Affected lines: 60, 61, 64, 65, 68, 69, 72, 73, 86, 105. (`expected_undelivered.main()` at line 38 stays unchanged.)
Add an entry function and update the `__main__` guard. Replace:
```python
if __name__ == "__main__":
run_multi_site_daemon()
```
with:
```python
def main():
"""交互菜单模式入口。"""
run_multi_site_daemon()
if __name__ == "__main__":
main()
```
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/router.py` → no output.
- [ ] **Step 9: Rewrite imports + add main() in cli/server.py**
In `inbound_verify/cli/server.py`, replace:
```python
from paths import DOWNLOAD_DIR, OUTPUT_DIR
import state_store
from runtime import (
HEARTBEAT_INTERVAL,
TASK_HANDLERS,
dispatch_task,
launch_and_prepare,
run_heartbeat,
)
```
with:
```python
from inbound_verify.paths import DOWNLOAD_DIR, OUTPUT_DIR
from inbound_verify import state_store
from inbound_verify.runtime import (
HEARTBEAT_INTERVAL,
TASK_HANDLERS,
dispatch_task,
launch_and_prepare,
run_heartbeat,
)
```
Replace the bottom entry block:
```python
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)
```
with:
```python
def main():
"""服务模式入口。传字符串导入路径（规范写法；不开 reload/workers 时进程内 import，行为等价）。"""
uvicorn.run("inbound_verify.cli.server:app", host="0.0.0.0", port=8000)
if __name__ == "__main__":
main()
```
Verify: `.venv/Scripts/python.exe -m py_compile inbound_verify/cli/server.py` → no output.
- [ ] **Step 10: Clean up stale filename comments**
Cosmetic but keeps grep clean (Step 12 depends on it). In each `inbound_verify/sites/*.py`, update the line-1 header `# site_xxx.py` → `# sites/xxx.py`. In `inbound_verify/sites/anneng.py`, update the standalone-run comment near the top:
```python
# .venv/Scripts/python.exe site_anneng.py
```
→
```python
# python -m inbound_verify.sites.anneng expected # 或 actual
```
And the comment at the `CDP_PORT` line referencing "独立运行 site_anneng.py" — update to "独立运行（python -m inbound_verify.sites.anneng）".
- [ ] **Step 11: Black-format all changed files**
```bash
.venv/Scripts/python.exe -m black inbound_verify
```
Expected: `reformatted ...` / `left unchanged` lines, exit 0.
- [ ] **Step 12: VERIFICATION GATE — run all five checks**
**12a. compileall (syntax across whole package):**
```bash
.venv/Scripts/python.exe -m compileall inbound_verify
```
Expected: no errors.
**12b. Import smoke (catches every wrong import path / missed rewrite):**
```bash
.venv/Scripts/python.exe -c "import inbound_verify.cli.router, inbound_verify.cli.server, inbound_verify.store, inbound_verify.runtime, inbound_verify.state_store; print('import smoke OK')"
```
Expected: `import smoke OK`. (The three entries transitively import sites + expected_undelivered.)
**12c. No stale `site_` references:**
```bash
grep -rn "site_shunxin\|site_baishi\|site_zto\|site_yunda\|site_anneng" inbound_verify || echo "no stale site_ refs OK"
```
Expected: `no stale site_ refs OK`.
**12d. No stale bare flat imports:**
```bash
grep -rnE "^from paths import|^from runtime import|^import site_|^import state_store$|^import expected_undelivered$" inbound_verify || echo "no stale flat imports OK"
```
Expected: `no stale flat imports OK`.
**12e. paths anchor points at project root + DB still connects:**
```bash
.venv/Scripts/python.exe -c "from inbound_verify.paths import BASE_DIR; print('BASE_DIR', BASE_DIR)"
.venv/Scripts/python.exe -c "from inbound_verify.store import _connect, _load_pg_config; c=_load_pg_config(); conn=_connect(c['dbname']); print('DB OK', conn.info.server_version); conn.close()"
```
Expected: `BASE_DIR` prints the `InboundVerify` project root (the dir containing `config.yaml`); `DB OK <pg version>`.
> If 12b fails with ModuleNotFoundError for a site module, run `.venv/Scripts/python.exe -m pip install -e .` again (editable finder refresh) and retry. If 12d still shows a line, that import was missed — rewrite it per Step 4/8 rules.
- [ ] **Step 13: Commit (only after user confirms)**
```bash
git add -A inbound_verify
git commit -m "refactor: move flat modules into inbound_verify package (Tier 1, behavior-identical)
- relocate 12 root .py into inbound_verify/ (sites/, cli/ subpackages)
- rewrite all internal imports to package-qualified
- fix paths.py BASE_DIR to anchor at project root
- add main() entry wrappers; register console_scripts
- drop site_ prefix on site modules; keep state_store/expected_undelivered names
Co-Authored-By: Claude <noreply@anthropic.com>"
```
> Do NOT commit until the user says to. This commit is the **safety baseline**; the manual end-to-end test (spec §7 gate) runs against this state before any Tier 2/Tier 3 work.
---
## Task 3: Docs sync (README + CLAUDE.md)
**Files:**
- Modify: `README.md` (§二 directory tree, §三 env prep, §五 run)
- Modify: `CLAUDE.md` (常用命令 section)
**Interfaces:**
- Consumes: the completed package from Task 2 (docs must describe the real new layout/commands).
- Produces: documentation matching the new invocation model. No code impact.
- [ ] **Step 1: Update README §二 directory tree**
Replace the tree block (README lines ~33–49) with the actual new layout:
```
InboundVerify/
├── pyproject.toml # 打包 + 依赖 + console_scripts
├── inbound_verify/ # 源码包
│ ├── paths.py runtime.py state_store.py expected_undelivered.py store.py
│ ├── sites/ shunxin / baishi / zto / yunda / anneng
│ └── cli/ router（交互菜单）/ server（FastAPI 服务）
├── config.example.yaml / config.yaml
├── schema.sql
├── requirements.txt # pyproject 的静态镜像
├── downloads/ output/ state/
└── docs/
```
- [ ] **Step 2: Update README §三 env prep — add editable install**
In the env-prep command block (README lines ~58–71), after `pip install -r requirements.txt`, add:
```bash
# 4. 以可编辑模式安装本包（注册 inbound-verify 等命令）
pip install -e .
```
(renumber the subsequent `cp config.example.yaml config.yaml` step).
- [ ] **Step 3: Update README §五 run — new commands**
Replace `python main_router.py` with:
```bash
# 交互菜单（任选其一）
python -m inbound_verify.cli.router
# 或装包后：inbound-verify
```
- [ ] **Step 4: Update CLAUDE.md 常用命令**
In the 常用命令 section, change every `.venv/Scripts/python.exe <module>.py` to the package form:
- `main_router.py` → `python -m inbound_verify.cli.router` (or `inbound-verify`)
- `server.py` → `python -m inbound_verify.cli.server` (or `inbound-verify-server`)
- `db_store.py createdb|init|ingest|all` → `python -m inbound_verify.store createdb|init|ingest|all` (or `inbound-verify-db ...`)
- `site_anneng.py expected|actual` → `python -m inbound_verify.sites.anneng expected|actual`
- `black`/`py_compile` targets → package paths (e.g. `-m black inbound_verify`)
Add a one-liner near the top of that section: `首次/拉取新代码后需 .venv/Scripts/python.exe -m pip install -e .`。
- [ ] **Step 5: Verify docs render + commands are real**
```bash
grep -n "main_router.py\|python server.py\|python db_store.py\|site_anneng.py\|site_shunxin" README.md CLAUDE.md || echo "no stale old-path commands OK"
```
Expected: `no stale old-path commands OK` (every old invocation updated).
- [ ] **Step 6: Commit (only after user confirms)**
```bash
git add README.md CLAUDE.md
git commit -m "docs: update README and CLAUDE.md for package layout and console_scripts
Co-Authored-By: Claude <noreply@anthropic.com>"
```
---
## Self-Review (completed)
- **Spec coverage:** spec §3 (layout) → Task 1+2; §4 (paths anchor) → Task 2 Step 2; §5 (import rules) → Task 2 Steps 3–9; §6 (entry/packaging) → Task 1 Step 2 + Task 2 Steps 6/8/9; §7 (verification gate) → Task 2 Step 12; §10 (docs) → Task 3. All covered.
- **Placeholder scan:** none — every step has exact code or an exact command with expected output. The 28 call-site rewrites are given as a deterministic prefix-drop rule + enumerated line numbers + verification grep (complete, not a placeholder).
- **Type/name consistency:** kept-module names (`state_store`, `expected_undelivered`, `runtime`, `paths`) used consistently across all import rewrites and call sites; renamed entries (`store`, `cli/router`, `cli/server`) consistent with pyproject `[project.scripts]`. `main()` signature consistent across router/server/store and console_scripts.