Commit Graph

3 Commits

Author SHA1 Message Date
Misaka_Company
ea48de290f fix(sync): track cleanup state to stop re-deleting log rows and bound SyncQueue
- access_reader.delete_log_ids returns the actual rows deleted (was None).

- sql_writer.mark_cleaned flips applied queue rows to 'cleaned' (sets CleanedAt) after their Access log rows are physically removed, so the same IDs are never deleted twice.

- sql_writer.purge_cleaned removes 'cleaned' rows older than a retention window (default 24h) so SyncQueue stops growing without bound.

- cleanup.cleanup_file marks rows cleaned after a successful delete and returns the real delete count, so the service log reports honest 'cleaned N' instead of a constant.

- service.cycle calls purge_cleaned once per pass; config adds cleaned_retention_hours (default 24).

- sql/01_sync_queue.sql adds CleanedAt column + IX_SyncQueue_Cleaned idempotently.

- tests: unit coverage for mark_cleaned/purge_cleaned/delete_log_ids return count; assert cycle purges each pass.
2026-07-14 16:32:47 +08:00
Misaka_Company
b1b118463d fix(sql): prevent stale Delete from outranking newer Insert in usp_SyncApply
Route both upsert and delete branches off a single ranked CTE (rn=1 per RecordID over all pending ops ordered by SourceLogID DESC). The previous design used two independent ranked CTEs, which let a stale Delete outrank a newer Insert for the same RecordID and silently drop the row. Also gitignore .claude/ and .workbuddy/ runtime dirs.
2026-07-14 15:32:03 +08:00
Misaka_Company
65939f5e85 feat: SyncQueue table and set-based apply stored procedure
- sql/01_sync_queue.sql: idempotent DDL for dbo.SyncQueue (PK + unique
  dedup index + pending lookup index), safe to re-run.
- sql/02_sync_apply.sql: dbo.usp_SyncApply (@MaxRetries INT=5). Per
  distinct (TargetSchema,TargetTable) it builds column projections from
  sys.columns (excludes ID key/computed/identity/rowversion) and runs a
  dynamic-SQL MERGE (last-write-wins via ROW_NUMBER over SourceLogID DESC)
  for Insert/Update plus a DELETE for the last op = Delete.
  SET IDENTITY_INSERT ON preserves Access PKs.
- tests/conftest.py: sql_conn fixture reads conn_str from gitignored
  config.yaml via load_config; skipped without RUN_INTEGRATION=1.
- tests/test_apply_proc.py: integration test covering IDENTITY-preserving
  INSERT, last-write-wins UPDATE, BIT conversion, and DELETE; cleans up.

Deviation from the brief's procedure (root-cause fix, design preserved):
every JSON path key is quoted ('$."col"') so non-ASCII column names
(e.g. Chinese 名字/数量) parse correctly. Without quoting, JSON_VALUE
raises "JSON path format is not correct" on Chinese columns, which is the
real target schema for this Access->SQL Server sync.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-14 12:12:21 +08:00