阶段1:服务化骨架(FastAPI + Playwright worker + 任务队列)
- 新增 runtime.py:抽离共享核心 launch_and_prepare / dispatch_task / run_heartbeat /
probe_* / launch_anneng / RuntimeContext,常量 SITES_CONFIG 等;main_router 复用
- 新增 server.py:FastAPI(主线程)+ Playwright worker(独立线程)+ 任务队列;
API:POST/GET /tasks、GET /status、GET /data/{file}(防路径穿越)
- state_store:加 task_history 表 + create/update/get/list 接口
- 5 站点 with_retry 改为返回 True/False,供 dispatch_task 判成败
- main_router:重写为复用 runtime 的交互模式(行为不变)
- requirements:加 fastapi、uvicorn
- CLAUDE.md:补充运行模式与共享核心架构说明
线程模型:主线程 FastAPI 不碰 Playwright,worker 线程独占 page,经 Queue + SQLite 通信。
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -47,6 +47,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||
|
||||
每次失败都重置(含最终放弃那一次):既是重试前的清场,也保证最终放弃时环境干净。
|
||||
flow 为零参可调用;返回 False 视为失败,其余视为成功。
|
||||
返回 True=最终成功,False=重试耗尽放弃(供调度层判断任务成败)。
|
||||
"""
|
||||
for attempt in range(1, max_attempts + 1):
|
||||
try:
|
||||
@@ -55,7 +56,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||
raise RuntimeError("流程返回失败状态")
|
||||
if attempt > 1:
|
||||
print(f">> 【{site_name}-{label}】第 {attempt} 次尝试成功 ✅")
|
||||
return
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"⚠️ 【{site_name}-{label}】第 {attempt}/{max_attempts} 次失败: {e}")
|
||||
print(f" → 重置【{site_name}】到初始态,清理环境 ...")
|
||||
@@ -68,6 +69,7 @@ def with_retry(site_name, label, flow, reset, max_attempts=3):
|
||||
print(
|
||||
f"❌ 【{site_name}-{label}】已达最大尝试次数 {max_attempts},放弃(环境已清理)。"
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
CDP_PORT = 9222 # 默认端口(独立运行 site_anneng.py 时用);main_router 启动时会用 set_cdp_port 覆盖
|
||||
|
||||
Reference in New Issue
Block a user