- Add sql/00_schema.sql: create dedicated ProductionDataBaseSync schema (idempotent)
- Move SyncQueue and usp_SyncApply from dbo into ProductionDataBaseSync
- Add sql/03_sync_log_archive.sql: permanent, append-only SyncLogArchive that
records both OriginalOperateType and ProcessedOperateType plus the Access log
OriginalTime, so pipeline divergences (e.g. Insert applied as Delete) stay
reconstructible forever (SyncQueue is transient and only keeps processed type)
- config.py: inject sync_queue_table / archive_table / apply_proc (default to the
new schema); SqlWriter takes these names instead of hardcoding dbo
- sql_writer.py: add ArchiveRow + insert_archive_row (dedup on source keys),
parametrize queue/archive/proc names throughout
- capture.py: archive every consumed log row before enqueue (preserves evidence
before cleanup deletes the Access log)
- service.py: pass the three names into SqlWriter
- tests: read queue/proc names from config instead of hardcoding dbo.SyncQueue
- access_reader.delete_log_ids returns the actual rows deleted (was None).
- sql_writer.mark_cleaned flips applied queue rows to 'cleaned' (sets CleanedAt) after their Access log rows are physically removed, so the same IDs are never deleted twice.
- sql_writer.purge_cleaned removes 'cleaned' rows older than a retention window (default 24h) so SyncQueue stops growing without bound.
- cleanup.cleanup_file marks rows cleaned after a successful delete and returns the real delete count, so the service log reports honest 'cleaned N' instead of a constant.
- service.cycle calls purge_cleaned once per pass; config adds cleaned_retention_hours (default 24).
- sql/01_sync_queue.sql adds CleanedAt column + IX_SyncQueue_Cleaned idempotently.
- tests: unit coverage for mark_cleaned/purge_cleaned/delete_log_ids return count; assert cycle purges each pass.
Adds SqlWriter: a pyodbc-backed writer that dedup-inserts into
dbo.SyncQueue (IF NOT EXISTS guarded by UX_SyncQueue_Dedup), invokes
dbo.usp_SyncApply, and reports applied SourceLogIDs.
Connection is opened with autocommit=True per the controller revision:
usp_SyncApply manages its own transaction internally (BEGIN/ROLLBACK),
and an outer pyodbc transaction would conflict on ROLLBACK (SQL error
266). The dedup IF NOT EXISTS...INSERT is a single atomic statement.
Integration test self-cleans via SourceFile='sqlw_test.accdb' marker;
conn_str comes from the gitignored config.yaml (no hardcoded creds).
Co-Authored-By: Claude <noreply@anthropic.com>
