Commit Graph

4 Commits

Author SHA1 Message Date
Misaka_Company
9c3cc38a44 fix(sync): retry cleanup on Access -1102 lock contention
ACE ODBC reports lock contention ("无法更新;当前被锁定。(-1102)")
as a generic pyodbc.Error (HY000), NOT pyodbc.OperationalError, so the
old except clause never caught it and gave up after the first collision
— making cleanup_lock_retries:3 a no-op for the 氩弧焊/高写入库 -1102 case.
Now catch pyodbc.Error and retry only on lock messages ("被锁定"/"-1102"),
re-raising other errors immediately instead of retrying them 3×.
2026-07-15 09:42:17 +08:00
Misaka_Company
a227aed0cb feat(sync): add one-shot full sync (TRUNCATE + bulk INSERT)
- New `sync.fullsync` CLI: `python -m sync.fullsync config.yaml [--db F] [--table T] [--clear-change-log]`
- AccessReader: list_user_tables / read_all_rows / read_all_log_ids
- SqlWriter: table_exists / truncate_target (TRUNCATE w/ DELETE fallback) / bulk_insert (SET IDENTITY_INSERT + chunked fast_executemany)
- Reuses FileMapping exclude/include rules (exclude beats include, same as capture)
- Preserves Access IDs via IDENTITY_INSERT; target schemas have no FKs so TRUNCATE is safe
- TableChangeLog NOT cleared by default (opt-in --clear-change-log)
- tests/test_fullsync.py covers resolve_tables exclude/include precedence
2026-07-14 18:06:39 +08:00
Misaka_Company
ea48de290f fix(sync): track cleanup state to stop re-deleting log rows and bound SyncQueue
- access_reader.delete_log_ids returns the actual rows deleted (was None).

- sql_writer.mark_cleaned flips applied queue rows to 'cleaned' (sets CleanedAt) after their Access log rows are physically removed, so the same IDs are never deleted twice.

- sql_writer.purge_cleaned removes 'cleaned' rows older than a retention window (default 24h) so SyncQueue stops growing without bound.

- cleanup.cleanup_file marks rows cleaned after a successful delete and returns the real delete count, so the service log reports honest 'cleaned N' instead of a constant.

- service.cycle calls purge_cleaned once per pass; config adds cleaned_retention_hours (default 24).

- sql/01_sync_queue.sql adds CleanedAt column + IX_SyncQueue_Cleaned idempotently.

- tests: unit coverage for mark_cleaned/purge_cleaned/delete_log_ids return count; assert cycle purges each pass.
2026-07-14 16:32:47 +08:00
Misaka_Company
1f7191421a feat: access reader and value serialization
Add to_jsonable() (Access/pyodbc value -> JSON-safe) with strict TDD
(5 tests: datetime iso, bool preserved before int, decimal->str,
none/numbers, dict round-trip). bool branch precedes int because bool
is a subclass of int in Python.

Add AccessReader: reads TableChangeLog (read_log), reads a full source
row by RecordID via to_jsonable (read_row, warns on >4000-char values
that JSON_VALUE would truncate), and deletes processed log IDs in
chunked, retried batches (delete_log_ids, no-op on empty list).
Connects via the ACE ODBC driver from config.access.driver, shared
autocommit mode to coexist with the live .laccdb client.

Integration test against the real 氩弧焊.accdb (read-only; delete_log_ids
only exercised with []) passes.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-14 12:22:48 +08:00