Misaka_Company 2ae5621090 feat(write_attachments): add --append mode, fix batch timing and SQL Server param limit ...

- Add --append flag: diff source-table 总排号 against existing
  Common.Attachment SN and classify/write only the missing ones. --limit
  caps the per-run append count, --order sets the direction. Backed by
  new fetch_existing_attachment_sns() in db.py.
- Fix batch elapsed-time accounting: summarize_results() summed each
  task's per-item elapsed_ms, which overcounts under ThreadPoolExecutor
  concurrency (cumulative work time, not real wall-clock — 100 tasks on
  8 workers reported ~5x the actual runtime). Callers now time
  classify_batch() via perf_counter and pass wall_clock_ms; both the
  write_attachments [汇总] line and main.py --summary report wall-clock
  separately from the cumulative sum.
- Format durations >=1s in seconds (88851.4 ms -> 88.85 s) in the
  human-readable [汇总] line; structured JSON --summary fields stay in ms.
- Chunk all IN (...) lists to 2000 items to respect SQL Server's 2100
  bind-parameter hard limit (previously --append --limit 5000 failed at
  the fetch step with "COUNT 字段不正确"). Applied to fetch_params_by_ids
  (sn / id paths) and to the DELETE inside upsert_attachments.

Co-Authored-By: Claude <noreply@anthropic.com>

2026-07-28 11:17:17 +08:00

9.3 KiB

Raw Permalink Blame History

View Raw