Commit Graph

14 Commits

Author SHA1 Message Date
Misaka_Company
62a66467a9 feat(runtime): auto-ingest hook after successful download
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 11:04:04 +08:00
Misaka_Company
7211846375 feat(state_store): add ingest_state table + set/get helpers 2026-07-24 10:55:43 +08:00
Misaka_Company
1093256de6 fix(store): guard ingest_task against 百世 non-undelivered kinds 2026-07-24 10:46:51 +08:00
Misaka_Company
0c51a41cfc feat(store): add kind-level ingest_task + ingest-one CLI subcommand
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 10:40:51 +08:00
Misaka_Company
70f518a1c3 feat(store): add auto_ingest config + pg connect/statement timeouts
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 10:34:27 +08:00
Misaka_Company
79c84a5a0c fix: retry launch page.goto to absorb transient DNS/timeout
Third launch failure this session (after two 顺心 load-timeouts and a 中通 ERR_NAME_NOT_RESOLVED) — each a transient network blip that killed the whole worker because launch_and_prepare opened sites with no retry. Wrap each site open+goto in _open_page: up to 3 attempts (2s apart) on domcontentloaded, so transient DNS/timeout is absorbed; only persistent failure (all 3) aborts. All 5 sites + 安能 now launch clean.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 09:06:02 +08:00
Misaka_Company
62c44b262a fix: launch page.goto uses domcontentloaded to avoid slow-site timeout
launch_and_prepare opened each site with Playwright's default wait_until='load', which waits for every resource (ads/trackers/images). sxne.sxjdfreight.com (顺心) reliably takes >30s to fire load, so its goto timed out and — launch not being retried — killed the whole worker, leaving only one page open. Switch the two launch gotos to wait_until='domcontentloaded' (return as soon as the DOM is ready; the readiness polling still gates login). Behavior for already-fast sites unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 08:59:56 +08:00
Misaka_Company
23042c272b refactor: rename expected_undelivered to compare
git mv expected_undelivered.py -> compare.py; update the 3 importers (store/runtime/router) to import compare. All public names (main, write_site_file, _read_business_dates) unchanged. Also add the Tier 2 plan doc.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 08:40:04 +08:00
Misaka_Company
91ef8970b8 refactor: extract domain.py (shared site/file/colmap config)
Move ALL_REPORT_SITES / SITE_UNDELIVERED_FILE / BAISHI_FILE / BAISHI_COLUMNS / arrived_pieces_* / STATIONS / _site_cfg out of expected_undelivered into a new leaf module inbound_verify/domain.py. store.py and runtime.py now read site/file config from domain directly instead of through the compare engine (store keeps eu only for _read_business_dates). Behavior identical.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 08:38:18 +08:00
Misaka_Company
bd0df8909f refactor: expected_undelivered uses paths.DOWNLOAD_DIR/OUTPUT_DIR (Tier 2)
Remove the duplicate BASE/DOWNLOADS/OUTPUT self-anchor (the one that caused the Tier 1 hotfix bug when the file moved into the package). DOWNLOADS/OUTPUT now come from paths.py; OUTFILE derives from OUTPUT_DIR. Behavior identical.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 08:34:57 +08:00
Misaka_Company
17293bee79 fix: correct expected_undelivered.BASE anchor after package move
Tier 1 regression: expected_undelivered.py carries its own BASE=dirname(__file__) anchor (a duplicate of paths.py). The package move dropped the file one level deeper, so BASE resolved to inbound_verify/ and DOWNLOADS/OUTPUT pointed at non-existent inbound_verify/downloads|output — while site downloads write to the project-root dirs via paths.py. process()/write_site_file() thus skipped the compare with '[跳过] downloads 下缺少 ...', returned None/False, and every undelivered task for the 4 web/app sites reported failed (重试耗尽) despite the files downloading fine. Fix: anchor BASE two levels up (mirrors paths.py). Verified: write_site_file returns True; all 4 undelivered tasks now success with 未到 files generated.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 13:55:48 +08:00
Misaka_Company
5ac618c041 docs: update stale module-name references in comments and docstrings
Replace pre-rename references (main_router, db_store, server.py) in code comments and docstrings with their new locations (cli/router, store, cli/server, runtime). Comment-only; no behavior change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 12:26:34 +08:00
Misaka_Company
7065b88269 refactor: move flat modules into inbound_verify package (Tier 1, behavior-identical)
Relocate 12 root .py modules into inbound_verify/ (sites/, cli/ subpackages). Rewrite all internal imports to package-qualified; drop the site_ prefix on the 5 site modules and their 28 call sites. Fix paths.py BASE_DIR to anchor at the project root. Add main() entry wrappers (cli/router, cli/server, store). No behavior change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 12:21:31 +08:00
Misaka_Company
0174e64a04 chore: scaffold inbound_verify package and pyproject
Empty package markers (inbound_verify/, sites/, cli/) plus pyproject.toml with console_scripts entry points and dependency floors.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 12:20:54 +08:00