Commit Graph

150 Commits

Author SHA1 Message Date
Misaka
cf9976f605 fix: add AT and ZONE to PostgreSQL SQL keywords for prepareSql
The prepareSql function quotes any word not in SQL_KEYWORDS as an
identifier. Since AT and ZONE were missing from the set, the expression
(NOW() AT TIME ZONE 'UTC') was mangled into (NOW() "AT" TIME "ZONE"
'UTC'), causing INSERT failures on PostgreSQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 22:00:06 +08:00
Misaka
a6c2e2ccc2 fix: use explicit UTC timestamp in PostgreSQL dialect
PostgreSQL's CURRENT_TIMESTAMP returns session-local time, unlike
SYSUTCDATETIME() (SQL Server) and UTC_TIMESTAMP() (MySQL) which
explicitly return UTC. Switch to (NOW() AT TIME ZONE 'UTC') to
keep operation history timestamps consistent across all databases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 21:48:28 +08:00
Misaka_Company
21089e8b40 perf(import): bypass intermediate Excel file in extraction pipeline
Replace the Extract → Write Excel → Read Excel → Import DB flow with
direct record-to-database persistence. The extractor now builds
MaterialPlanRecord[] from parsed orders and imports them without the
round-trip through a merged Excel file.

Key changes:
- Add importFromRecords() to DataImportService for record-based import
- Add SQL Server OPENJSON batch insert and atomic replace operations
  in DiscreteMaterialPlanDAO for efficient bulk writes
- Extract common import logic into private importRecords() method
- Configure explicit request/connection timeouts for SQL Server
- Add unit tests for direct record import path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 17:11:30 +08:00
Misaka_Company
dbb8e4904e perf: optimize order resolution history writes 2026-04-28 13:39:08 +08:00
Misaka_Company
98865f5d7e fix(cleaner): preserve production IDs in operation history
The 总排号 field was always empty because getCleanerData() resolved
production IDs to order numbers before passing them to the cleaner,
losing the original inputs. Now originalInputs are carried through
the full chain so the resolver can properly set productionId.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 11:57:47 +08:00
Misaka_Company
5ff99cdd0f refactor: use dot notation for table name config (schema.table instead of schema_table)
Replace underscore-based table name splitting with dot-based splitting
to match the standard schema.tablename format, removing MySQL compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 11:10:03 +08:00
Misaka
664f26d63f fix(cleaner): guard missing session refresh config 2026-04-24 19:11:21 +08:00
Misaka_Company
ac43790127 Add cleaner session refresh and ERP diagnostics 2026-04-24 15:40:47 +08:00
Misaka_Company
f49f99fc0c perf(cleaner-history): parallelize batch fetching in searchBatches
Replace sequential for-loop with Promise.all so that matched batches
are fetched concurrently instead of one-by-one, reducing total query
latency from O(n) serial round-trips to a single parallel batch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 12:42:07 +08:00
Misaka_Company
3a30694684 feat(cleaner-history): add searchBatches DAO method
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 11:05:17 +08:00
Misaka
8fa4d6c16d fix: support postgres material upserts without unique constraints 2026-04-14 21:55:05 +08:00
Misaka
5b43d5a60c fix: restore cleaner history in postgresql 2026-04-14 20:49:29 +08:00
Misaka_Company
838783e384 fix(auth): clear cached silentLoginPromise on logout to allow re-authentication
After logout, the cached silentLoginPromise caused silentLogin() to return
a stale result instead of re-executing loginByComputerName(), leaving
sessionManager.currentUser as null and making subsequent switchUser() calls fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:17:34 +08:00
Misaka_Company
343cb24234 chore: run pretier format across project
- Format TypeScript source files
- Format documentation files
- Update eslint config formatting
2026-04-14 14:03:58 +08:00
Misaka_Company
6f49596467 feat(cleaner-history): record missing orders with production ID tracking
Record ALL input orders in history, including resolution failures (not_found)
and ERP query misses (erp_not_found). Add ProductionId column to track original
总排号 input. Add 总排号 column and new status styles to the history UI. Fix
empty result caching that prevented retry on transient query failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 10:16:49 +08:00
Misaka_Company
420f811488 fix(timezone): use UTC for database storage and local time for UI display
Backend changes:
- SQL Server dialect: GETDATE() → SYSUTCDATETIME()
- MySQL dialect: NOW() → UTC_TIMESTAMP()
- Ensures OperationTime and EndTime use consistent UTC timezone

Frontend changes:
- formatDateTime: display UTC timestamps in user's local timezone
- Uses getFullYear/getMonth/getDate/getHours (local) instead of UTC methods

Data migration:
- Executed migration script to fix historical OperationTime records
- All existing records now have correct UTC timestamps
- Execution duration now accurate (minutes, not hours)

Impact:
- New executions store UTC timestamps correctly
- UI displays times in user's local timezone (UTC+8 for CN users)
- Historical data corrected via migration
- Time difference between OperationTime and EndTime now accurate
2026-04-13 17:52:50 +08:00
Misaka_Company
151485caed feat(cleaner): track skipped materials and skip DB writes on dry run
Record materials not in the deletion list as "skipped" with reason
instead of just logging them. Skip inserting material details to
database during dry runs to avoid phantom records.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 15:58:00 +08:00
Misaka_Company
116539ff42 feat(cleaner): record all material operations in database, including successful deletions
Previously only skipped and failed materials were persisted. Now every
material (deleted, uncertain, skipped, failed) is recorded in
CleanerMaterialDetail for full audit traceability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 14:24:18 +08:00
Misaka_Company
0286df94dd fix(cleaner): cast BIT to INT for MAX() in getBatches query
SQL Server does not support MAX() on BIT columns, causing the
getBatches query to fail silently and return empty results.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 14:17:26 +08:00
Misaka_Company
32931cecad style: format changed files with prettier
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:35:31 +08:00
Misaka_Company
7dfa88c2a3 refactor(cleaner): remove Markdown report generator
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:06:19 +08:00
Misaka_Company
998edb4b84 refactor(cleaner): replace report generation with database persistence
Remove generateExecutionId(), generateAndUploadReport(), and all
executionId references from CleanerApplicationService. The service
now accepts batchId, historyDao, and appVersion from the IPC handler
and writes execution/order/material records to the database via
CleanerOperationHistoryDAO instead of generating Markdown reports.

All execution paths (success, failure, outer retry, retry-login-failure)
persist their results to the database with appropriate status tracking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 12:02:43 +08:00
Misaka_Company
a924e8a4e8 feat(cleaner): add CleanerOperationHistoryDAO for three-table persistence
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 11:29:54 +08:00
Misaka_Company
e1d55b8b39 feat(cleaner): add outer-level retry on fatal crash with execution ID
When CleanerService hits a fatal error (browser crash, timeout), the
outer catch now sets result.crashed=true. CleanerApplicationService
detects this, closes the dead browser session, re-logs into ERP, and
re-runs all orders once. An execution ID (CLN-yyyyMMddHHmmss-XXXX)
generated at startup ensures report files are deduplicated across
retries. Reports now display execution ID and app version.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 10:07:29 +08:00
Misaka
8b173890fa feat(cleaner): add multi-signal deletion verification with material-level retry
Replace fragile single-signal (row change only) deletion verification
with a robust multi-signal approach using row change + material count +
ERP message detection. Add material-level retry (up to 3 attempts) for
transient failures, with detailed tracking of failed/uncertain deletions.

- Add DeletionOutcome/DeletionErrorCategory enums and FailedMaterial type
- Add deleteWithVerification() core method with retry logic
- Add evaluateDeletionSignals() pure logic (unit tested, 9 cases)
- Add helper methods: readMaterialCount, checkErpMessages, handleConfirmDialog
- Extend CleanerResult/OrderCleanDetail with failed/uncertain tracking
- Update report generator with failed materials detail section
- Update ExecutionReportDialog to display failed/uncertain stats

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 21:55:47 +08:00
Misaka
3be7959067 feat(cleaner): support selectedManagers filtering for admin cleaner execution
Admin can now pass selectedManagers to getCleanerData so material codes
are queried from MaterialsToBeDeleted by ManagerName IN (selectedManagers).
When no managers are selected, fallback to DiscreteMaterialPlanData by
orderNumbers. User behavior is unchanged. Includes updated tests and
role-based flow documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 19:20:22 +08:00
Misaka
91f29a1167 refactor(audit): unify computerName source to cached os.hostname()
Export cachedHostname from audit-logger and use it in process-guards,
replacing process.env.COMPUTERNAME so all audit entries use the same value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 17:41:36 +08:00
Misaka
abad61758c refactor(audit): type-safe enums, expanded coverage, and crash-safe logging
Replace magic strings with AuditAction/AuditStatus enums across all consumers,
add logAuditWithCurrentUser() convenience wrapper, extend audit coverage to
data import, result export, app update, and ERP credentials operations, and
harden crash handlers with try/catch to prevent audit failures from cascading.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 17:40:30 +08:00
Misaka
4a7c220baa fix(extractor): resolve SQL syntax error from double-quoted table names 2026-04-05 13:51:18 +08:00
Misaka
f51cae0f6f fix(db): complete PostgreSQL integration in validation and cleaner services
OrderNumberResolver, validation, and cleaner services had incomplete
PostgreSQL support - they only handled SQL Server and MySQL, causing
PostgreSQL to fall through to MySQL code paths with invalid syntax
(backticks, ? placeholders) and missing schema.table name splitting.

Changes:
- Add PostgreSQL SQL generation ($N params, double-quoted identifiers)
  in OrderNumberResolver, validation-application-service,
  production-input-service, and validation-database
- Add PostgreSQL to database factory functions in validation-database
  and cleaner-application-service
- Add UPPER, LOWER, and 40+ common SQL functions to SQL_KEYWORDS to
  prevent prepareSql() from quoting them as identifiers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 13:46:37 +08:00
Misaka
7601b5f176 fix(db): PostgreSQL P0 fixes - SQL_KEYWORDS expansion, timeout config, and tests
- Expand SQL_KEYWORDS from ~120 to 226+ words covering:
  - Window functions (ROW_NUMBER, RANK, LAG, LEAD, etc.)
  - CTEs (WITH, RECURSIVE, MATERIALIZED, etc.)
  - Advanced grouping (ROLLUP, CUBE, GROUPING SETS)
  - JSON operations, types, table sampling
  - Transaction control and other PostgreSQL-specific keywords
- Add connection pool timeout configuration:
  - connectionTimeoutMillis: 10s
  - statement_timeout: 30s (PostgreSQL level)
  - idleTimeoutMillis: 30s (connection cleanup)
  - query_timeout: 60s (driver-level fallback)
- Add 12 comprehensive edge case tests covering:
  - Window functions, CTEs, advanced grouping
  - CASE expressions, set operations, JSON operators
- All 38 tests pass

Production-ready: prevents hung queries and supports complex SQL.
2026-04-05 13:09:01 +08:00
Misaka
e2669af870 fix: remove unused imports and fix logger test isolation
- Remove unused imports (run, trackDuration, PerformanceTracker,
  ConfigManager, disconnectDb) flagged by ESLint
- Remove unused isSlow variable in performance-monitor catch block
- Add eslint-disable for require() in Playwright JS script
- Fix logger-performance test flakiness by using vi.resetModules()
  with dynamic imports to prevent cached logger references across
  test files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 12:15:18 +08:00
Misaka
9791a84047 fix(db): auto-quote SQL identifiers for PostgreSQL case-sensitivity
Add prepareSql() to PostgreSqlService that quotes unquoted column names
before execution. PostgreSQL lowercases unquoted identifiers, but
SSMA-migrated tables have uppercase column names requiring double-quoting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 11:54:30 +08:00
Misaka
b5ba18b595 refactor(db): migrate BIPUsersDAO to use DatabaseFactory and SqlDialect
Replace hardcoded MySqlService/SqlServerService with DatabaseFactory,
enabling PostgreSQL support for user authentication and management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 11:20:17 +08:00
Misaka
13fb7bcf46 style: fix lint errors in dialect files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:37:32 +08:00
Misaka
0ca17a1807 fix(db): correct dialect import paths and extend bip-users-dao type
- Fix dialect files to use relative paths instead of @types alias
- Add 'postgresql' to BIPUsersDAO dbType union

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:34:03 +08:00
Misaka
54a3ac680a feat(db): integrate PostgreSQL into factory, config, and TypeORM data source
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:29:57 +08:00
Misaka
16b2882729 style: fix extra blank line after formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:26:29 +08:00
Misaka
e97ec63433 refactor(db): use SqlDialect in ExtractorOperationHistoryDAO
Replace all isSqlServer checks, buildPlaceholders, and hardcoded table names
with the SqlDialect abstraction. The dialect now handles parameter placeholders,
table name quoting, current timestamp functions, and pagination across all
supported database types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:25:38 +08:00
Misaka
9556891dea refactor(db): use SqlDialect in MaterialsTypeToBeDeletedDAO
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:20:04 +08:00
Misaka
fa57f9e564 refactor(db): use SqlDialect in MaterialsToBeDeletedDAO
Replace all isSqlServer/if-else branches with SqlDialect calls:
- Table name via dialect.quoteTableName()
- Placeholders via dialect.param() and dialect.params()
- UPSERT via dialect.upsert() in upsertMaterial(), upsertBatch(), updateManager()
- Remove buildPlaceholders(), TABLE_NAME_SQLSERVER, TABLE_NAME_MYSQL
- Re-export SqlDialect type from dialects barrel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:17:21 +08:00
Misaka
9300f3455f refactor(db): use SqlDialect in DiscreteMaterialPlanDAO
Replace all manual isSqlServer checks and inline SQL dialect logic with the
SqlDialect abstraction. Removes buildPlaceholders(), TABLE_NAME_SQLSERVER,
and TABLE_NAME_MYSQL in favor of dialect.params(), dialect.param(), and
dialect.quoteTableName(). Batch size logic now uses dialect.maxBatchRows().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:10:54 +08:00
Misaka
7e521da3f1 feat(db): add PostgreSqlService with pg driver
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:04:48 +08:00
Misaka
130e0602d1 feat(db): implement SqlDialect with MySQL, SQL Server, PostgreSQL dialects
Add three SqlDialect implementations with a factory function:
- MySqlDialect: positional ?, ON DUPLICATE KEY UPDATE, LIMIT/OFFSET
- SqlServerDialect: @pN params, MERGE USING, OFFSET/FETCH
- PostgreSqlDialect: $N (1-based), ON CONFLICT DO UPDATE, LIMIT/OFFSET

TDD approach: 43 tests written first, all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:01:24 +08:00
Misaka
4f3af2e9c3 feat(logging): enhance CleanerService logging granularity for better debugging
- Add detailed step-by-step logging in navigation phase with elapsed time tracking
- Enhance query interface setup with individual step logging and timing
- Improve order query and result collection with validation logging
- Add comprehensive processDetailPage logging with 8 tracked steps
- Detail material processing loop with decision tracking (delete/skip reasons)
- Enhance retry mechanism with per-attempt logging and success rate tracking
- Add performance monitoring with slow operation detection (isSlow flags)
- All logs use consistent Chinese labeling with [Phase] prefix format

Total: +437 lines of logging instrumentation across cleaner.ts
2026-04-04 18:09:50 +08:00
Misaka
7f38150d0a feat(logging): add ipAddress to default log metadata
Add getLocalIpAddress() that reliably resolves the primary LAN IPv4
address by collecting all non-loopback, non-APIPA addresses and
prioritizing RFC 1918 private ranges (192.168.x.x, 10.x.x.x,
172.16-31.x.x) over public IPs. Falls back to any non-internal
address or 'N/A'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 17:52:11 +08:00
Misaka
8be5a2763d fix(logging): register custom Winston levels so verbose captures debug
Winston's default npm levels assign debug=5 and verbose=4, so setting
level to 'verbose' (threshold 4) filtered out debug (5 > 4). Register
PROJECT_LEVELS { error:0, warn:1, info:2, debug:3, verbose:4 } so
Winston's <= threshold filter aligns with the project's intended
semantics where verbose is the most detailed level.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 17:20:10 +08:00
Misaka
1e0bb1de24 feat(logging): add Seq transport, global meta fields, and improve error handling
- Add Seq centralized logging transport with async ESM import
- Add appVersion and computerName to logger defaultMeta (all app logs)
- Add appVersion to audit log entries for version-level traceability
- Improve unhandledRejection to capture full stack traces for Error instances
- Add Seq config schema and template configuration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 16:33:03 +08:00
Misaka
fce8dbc37f feat(logging-p0): add screenshot capture and browser console diagnostics for ERP errors
Enhance ERP automation error diagnostics by capturing PNG screenshots
on every error and forwarding browser console warnings/errors to the
structured logger. Includes automatic cleanup of old screenshots
aligned with the configured log retention period.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 14:00:48 +08:00
Misaka
c8783a2cef feat(logging-p0): add full-step logging to ERP automation and unify capturePageContext
Add ~45 structured log calls across extractor-core, cleaner, and erp-auth
to cover all automation steps (navigation, query, download, material processing).
Enhance capturePageContext with a step parameter for precise failure localization,
and fix missing capturePageContext calls in error handlers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 13:19:41 +08:00