9 Commits
v1.14.1 ... dev

Author SHA1 Message Date
Misaka_Company
05539ff293 1.15.0 2026-08-04 09:44:05 +08:00
Misaka_Company
ab790a6be5 docs: add release notes for version 1.15.0 2026-08-04 09:43:59 +08:00
Misaka_Company
056bf3908c feat(cleaner): add configurable row number protection toggle 2026-08-04 09:43:31 +08:00
Misaka_Company
98567d69f0 docs: clarify project rename is planned only, not yet applied to code
- Update rename notice in all 6 doc files to state that the rename from
  ERPAuto to BIPMaterialManager is currently a plan only, with no actual
  code changes implemented
- Documentation names updated for forward compatibility; all code-level
  configs, paths, and artifact names remain unchanged

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-03 16:38:57 +08:00
Misaka
cf9976f605 fix: add AT and ZONE to PostgreSQL SQL keywords for prepareSql
The prepareSql function quotes any word not in SQL_KEYWORDS as an
identifier. Since AT and ZONE were missing from the set, the expression
(NOW() AT TIME ZONE 'UTC') was mangled into (NOW() "AT" TIME "ZONE"
'UTC'), causing INSERT failures on PostgreSQL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 22:00:06 +08:00
Misaka
a6c2e2ccc2 fix: use explicit UTC timestamp in PostgreSQL dialect
PostgreSQL's CURRENT_TIMESTAMP returns session-local time, unlike
SYSUTCDATETIME() (SQL Server) and UTC_TIMESTAMP() (MySQL) which
explicitly return UTC. Switch to (NOW() AT TIME ZONE 'UTC') to
keep operation history timestamps consistent across all databases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 21:48:28 +08:00
Misaka_Company
21089e8b40 perf(import): bypass intermediate Excel file in extraction pipeline
Replace the Extract → Write Excel → Read Excel → Import DB flow with
direct record-to-database persistence. The extractor now builds
MaterialPlanRecord[] from parsed orders and imports them without the
round-trip through a merged Excel file.

Key changes:
- Add importFromRecords() to DataImportService for record-based import
- Add SQL Server OPENJSON batch insert and atomic replace operations
  in DiscreteMaterialPlanDAO for efficient bulk writes
- Extract common import logic into private importRecords() method
- Configure explicit request/connection timeouts for SQL Server
- Add unit tests for direct record import path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 17:11:30 +08:00
Misaka_Company
f36c88aa89 fix(extractor): display operation time in local timezone
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 13:53:54 +08:00
Misaka_Company
dbb8e4904e perf: optimize order resolution history writes 2026-04-28 13:39:08 +08:00
33 changed files with 1036 additions and 177 deletions

View File

@@ -5,7 +5,9 @@
## 项目概览 ## 项目概览
ERPAuto 是一个基于 Electron 的桌面应用,用于自动化处理 ERP 系统中的数据提取、清理、校验和配置管理。 BIPMaterialManager 是一个基于 Electron 的桌面应用,用于自动化处理 BIP 系统中的物料数据提取、清理、校验和配置管理。
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
技术栈: 技术栈:

View File

@@ -1,6 +1,8 @@
# ERPAuto - ERP 数据自动化处理工具 # BIPMaterialManager - BIP 物料管理工具
一个基于 Electron 的桌面应用程序,用于自动化处理 ERP 系统中的数据提取和清理任务 > **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
一个基于 Electron 的桌面应用程序,用于自动化处理 BIP 系统中的物料数据提取和清理任务。
## 功能特性 ## 功能特性
@@ -22,7 +24,7 @@
```bash ```bash
# 克隆项目 # 克隆项目
git clone <repository-url> git clone <repository-url>
cd ERPAuto cd BIPMaterialManager
# 安装依赖 # 安装依赖
npm install npm install
@@ -143,7 +145,7 @@ const config = createMockConfigManager({ logging: { level: 'debug' } })
## 项目结构 ## 项目结构
``` ```
ERPAuto/ BIPMaterialManager/
├── src/ ├── src/
│ ├── main/ # 主进程代码 │ ├── main/ # 主进程代码
│ │ ├── services/ # 业务服务 │ │ ├── services/ # 业务服务
@@ -158,6 +160,8 @@ ERPAuto/
└── docs/ # 文档 └── docs/ # 文档
``` ```
> **注意**:由于项目正在进行重命名,代码层面的配置文件、路径和产物名称暂时仍使用 `erpauto`,将在后续阶段统一更新。
## 技术栈 ## 技术栈
- **框架**Electron 39 - **框架**Electron 39

View File

@@ -66,6 +66,7 @@ cleaner:
queryBatchSize: 100 queryBatchSize: 100
processConcurrency: 1 processConcurrency: 1
sessionRefreshOrderThreshold: 160 # 会在 batch 边界检查;达到或超过阈值后,在当前 batch 完成后重建浏览器会话 sessionRefreshOrderThreshold: 160 # 会在 batch 边界检查;达到或超过阈值后,在当前 batch 完成后重建浏览器会话
enableRowProtection: true # 行号保护:禁止删除行号 2000-7999 范围内的物料,关闭后不再检查行号范围
logging: logging:
level: info level: info

View File

@@ -1,6 +1,8 @@
# ERPAuto 文档指南 # BIPMaterialManager 文档指南
本文档是 ERPAuto 项目文档的**分类指南和编写规范**,用于: > **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
本文档是 BIPMaterialManager 项目文档的**分类指南和编写规范**,用于:
- 指导文档的分类和归档 - 指导文档的分类和归档
- 规范新文档的命名和格式 - 规范新文档的命名和格式

View File

@@ -1,6 +1,8 @@
# Playwright 部署说明 # Playwright 部署说明
本文档聚焦“如何让 ERPAuto 在目标机器上拥有可用的 Playwright Chromium 浏览器”,适合作为实际部署操作说明 > **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
本文档聚焦”如何让 BIPMaterialManager 在目标机器上拥有可用的 Playwright Chromium 浏览器”,适合作为实际部署操作说明。
如果你想看版本信息,请同时参考: 如果你想看版本信息,请同时参考:

View File

@@ -1,6 +1,8 @@
# 构建与发布流程 # 构建与发布流程
本文档说明 ERPAuto Windows 便携版的当前构建与发布方式,包括推荐的一键发布命令、分步命令,以及发布产物在对象存储中的结构 > **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理
本文档说明 BIPMaterialManager Windows 便携版的当前构建与发布方式,包括推荐的一键发布命令、分步命令,以及发布产物在对象存储中的结构。
## 概览 ## 概览

View File

@@ -1,4 +1,6 @@
# ERPAuto 便携版自动更新说明 # BIPMaterialManager 便携版自动更新说明
> **项目重命名说明**:本项目计划从 `ERPAuto` 重命名为 `BIPMaterialManager`。本次重命名仅停留在计划阶段,尚未落实到任何具体代码。目前仅对文档中的项目名称进行了更新,代码层面的配置、路径、产物名称等均保持原样,将在后续阶段统一处理。
## 概览 ## 概览

16
docs/releases/1.15.0.md Normal file
View File

@@ -0,0 +1,16 @@
# 1.15.0
## 核心功能
- 物料清理新增「行号保护」开关,可在执行设置中随时开启或关闭 2000-7999 行号范围的保护,关闭后不再限制该范围内物料的删除,灵活适配不同业务场景。
- 离散物料计划导入链路重构,提取结果直接写入数据库,不再经过中间 Excel 文件,大批量导入整体耗时显著下降。
## 改进
- 订单解析历史记录写入流程优化,批量场景下写入更高效,任务整体耗时更短。
## 问题修复
- 修复 ERP 操作历史中操作时间显示与本地时区不一致的问题,时间展示更直观。
- 修复 PostgreSQL 数据源中时间戳未显式按 UTC 写入的问题,避免跨时区数据偏差。
- 修复 PostgreSQL 预编译 SQL 对 AT、ZONE 等关键字处理错误的问题,相关查询可正常执行。

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "erpauto", "name": "erpauto",
"version": "1.14.1", "version": "1.15.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "erpauto", "name": "erpauto",
"version": "1.14.1", "version": "1.15.0",
"hasInstallScript": true, "hasInstallScript": true,
"dependencies": { "dependencies": {
"@aws-sdk/client-s3": "^3.929.0", "@aws-sdk/client-s3": "^3.929.0",

View File

@@ -1,6 +1,6 @@
{ {
"name": "erpauto", "name": "erpauto",
"version": "1.14.1", "version": "1.15.0",
"description": "An Electron application with React and TypeScript", "description": "An Electron application with React and TypeScript",
"main": "./out/main/index.js", "main": "./out/main/index.js",
"author": "example.com", "author": "example.com",

View File

@@ -126,7 +126,9 @@ export function registerExtractorHandlers(): void {
sendLog(sender, 'info', '正在解析订单号...') sendLog(sender, 'info', '正在解析订单号...')
const resolver = new OrderNumberResolver(dbService) const resolver = new OrderNumberResolver(dbService)
const resolutionStart = Date.now()
const mappings = await resolver.resolve(input.orderNumbers) const mappings = await resolver.resolve(input.orderNumbers)
const resolutionDurationMs = Date.now() - resolutionStart
// Get valid order numbers and warnings // Get valid order numbers and warnings
const validOrderNumbers = resolver.getValidOrderNumbers(mappings) const validOrderNumbers = resolver.getValidOrderNumbers(mappings)
@@ -146,7 +148,16 @@ export function registerExtractorHandlers(): void {
) )
} }
log.info('Resolved order numbers', { count: validOrderNumbers.length }) log.info('Resolved order numbers', {
inputCount: input.orderNumbers.length,
count: validOrderNumbers.length,
durationMs: resolutionDurationMs
})
sendLog(
sender,
'info',
`订单号解析完成:${validOrderNumbers.length}/${input.orderNumbers.length} 个有效,耗时 ${(resolutionDurationMs / 1000).toFixed(2)}`
)
// Initialize operation history recording // Initialize operation history recording
const currentUser = SessionManager.getInstance().getUserInfo() const currentUser = SessionManager.getInstance().getUserInfo()
@@ -155,6 +166,7 @@ export function registerExtractorHandlers(): void {
// Save order records to history (preserve productionId -> orderNumber mapping) // Save order records to history (preserve productionId -> orderNumber mapping)
if (currentUser) { if (currentUser) {
const historyInsertStart = Date.now()
const orderRecords = mappings.map((m) => ({ const orderRecords = mappings.map((m) => ({
productionId: m.productionId || null, productionId: m.productionId || null,
orderNumber: m.orderNumber || m.input orderNumber: m.orderNumber || m.input
@@ -165,9 +177,11 @@ export function registerExtractorHandlers(): void {
currentUser.username, currentUser.username,
orderRecords orderRecords
) )
const historyInsertDurationMs = Date.now() - historyInsertStart
log.info('Operation history batch created', { log.info('Operation history batch created', {
batchId, batchId,
recordCount: orderRecords.length recordCount: orderRecords.length,
durationMs: historyInsertDurationMs
}) })
} }

View File

@@ -100,7 +100,8 @@ const DEFAULT_CONFIG: FullConfig = {
cleaner: { cleaner: {
queryBatchSize: 100, queryBatchSize: 100,
processConcurrency: 1, processConcurrency: 1,
sessionRefreshOrderThreshold: 160 sessionRefreshOrderThreshold: 160,
enableRowProtection: true
}, },
orderResolution: { orderResolution: {
tableName: '', tableName: '',

View File

@@ -96,45 +96,12 @@ export class DataImportService {
// Step 1: Read Excel file // Step 1: Read Excel file
log.info('Reading Excel file...') log.info('Reading Excel file...')
const { records, sourceNumbers } = await this.readExcelFile(filePath) const { records, sourceNumbers } = await this.readExcelFile(filePath)
result.recordsRead = records.length
result.uniqueSourceNumbers = sourceNumbers.size
log.info('Excel read completed', { log.info('Excel read completed', {
recordsRead: result.recordsRead, recordsRead: records.length,
uniqueSourceNumbers: result.uniqueSourceNumbers uniqueSourceNumbers: sourceNumbers.size
}) })
if (records.length === 0) { await this.importRecords(records, batchSize, result)
result.success = true
result.errors.push('Excel file contains no data records')
return result
}
// Step 2: Delete existing records by SourceNumber
log.info('Deleting existing records...', {
sourceNumberCount: sourceNumbers.size
})
const sourceNumberArray = Array.from(sourceNumbers)
result.recordsDeleted = await this.dao.deleteBySourceNumbers(sourceNumberArray)
log.info('Existing records deleted', {
recordsDeleted: result.recordsDeleted
})
// Step 3: Batch insert new records
log.info('Inserting new records...', {
recordCount: records.length,
batchSize
})
result.recordsImported = await this.dao.batchInsert(records, batchSize)
log.info('Records imported successfully', {
recordsImported: result.recordsImported
})
result.success = true
} catch (error) { } catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error) const errorMsg = error instanceof Error ? error.message : String(error)
result.errors.push(`Import failed: ${errorMsg}`) result.errors.push(`Import failed: ${errorMsg}`)
@@ -167,6 +134,92 @@ export class DataImportService {
return result return result
} }
/**
* Import already parsed records to database.
*
* This is the preferred path for extraction: the downloader/parser already has
* structured rows, so database persistence should not require writing and
* reading an intermediate Excel file.
*/
async importFromRecords(records: MaterialPlanRecord[], batchSize = 1000): Promise<ImportResult> {
const result: ImportResult = {
success: false,
recordsRead: 0,
recordsDeleted: 0,
recordsImported: 0,
uniqueSourceNumbers: 0,
errors: []
}
try {
log.info('Starting import from parsed records', {
recordCount: records.length,
batchSize
})
return await this.importRecords(records, batchSize, result)
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
result.errors.push(`Import failed: ${errorMsg}`)
log.error('Import from records failed', { error: errorMsg })
return result
} finally {
try {
await this.dao.disconnect()
} catch (e) {
log.warn('Error disconnecting DAO', {
error: e instanceof Error ? e.message : String(e)
})
}
logAuditWithCurrentUser(
AuditAction.DATA_IMPORT,
'MATERIAL_PLAN',
result.success ? AuditStatus.SUCCESS : AuditStatus.FAILURE,
{
recordsRead: result.recordsRead,
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported,
uniqueSourceNumbers: result.uniqueSourceNumbers,
errorCount: result.errors.length
}
)
}
}
private async importRecords(
records: MaterialPlanRecord[],
batchSize: number,
result: ImportResult
): Promise<ImportResult> {
const sourceNumbers = new Set(records.map((record) => record.sourceNumber).filter(Boolean))
result.recordsRead = records.length
result.uniqueSourceNumbers = sourceNumbers.size
if (records.length === 0) {
result.success = true
result.errors.push('No data records to import')
return result
}
// Step 1: Replace existing records by SourceNumber
log.info('Replacing existing records...', {
sourceNumberCount: sourceNumbers.size
})
const replaceResult = await this.dao.replaceBySourceNumbers(records, batchSize)
result.recordsDeleted = replaceResult.deleted
result.recordsImported = replaceResult.inserted
log.info('Records replaced successfully', {
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported
})
result.success = true
return result
}
/** /**
* Read Excel file and extract records * Read Excel file and extract records
* @param filePath - Path to the Excel file * @param filePath - Path to the Excel file

View File

@@ -26,7 +26,7 @@ export class PostgreSqlDialect implements SqlDialect {
} }
currentTimestamp(): string { currentTimestamp(): string {
return 'CURRENT_TIMESTAMP' return "(NOW() AT TIME ZONE 'UTC')"
} }
upsert(params: { upsert(params: {

View File

@@ -13,6 +13,7 @@ import { createDialect, type SqlDialect } from './dialects'
import { createLogger, getRequestId, trackDuration } from '../logger' import { createLogger, getRequestId, trackDuration } from '../logger'
const log = createLogger('DiscreteMaterialPlanDAO') const log = createLogger('DiscreteMaterialPlanDAO')
const SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE = 25
/** /**
* Material plan record interface * Material plan record interface
@@ -562,9 +563,16 @@ export class DiscreteMaterialPlanDAO {
const tableName = this.getTableName() const tableName = this.getTableName()
const dialect = this.getDialect() const dialect = this.getDialect()
// SQL Server has a limit of 2100 parameters per query if (dbService.type === 'sqlserver') {
// Each record has 28 columns, so max rows per batch = 2100 / 28 = 75 return await this.batchInsertSqlServerJson(
// Leave some margin for query overhead dbService,
tableName,
records,
batchSize,
batchId
)
}
const columnsPerRow = 28 const columnsPerRow = 28
const effectiveBatchSize = Math.min(batchSize, dialect.maxBatchRows(columnsPerRow)) const effectiveBatchSize = Math.min(batchSize, dialect.maxBatchRows(columnsPerRow))
const totalBatches = Math.ceil(records.length / effectiveBatchSize) const totalBatches = Math.ceil(records.length / effectiveBatchSize)
@@ -626,6 +634,237 @@ export class DiscreteMaterialPlanDAO {
} }
} }
async replaceBySourceNumbers(
records: MaterialPlanRecord[],
batchSize = 1000
): Promise<{ deleted: number; inserted: number }> {
if (!records || records.length === 0) {
return { deleted: 0, inserted: 0 }
}
const dbService = await this.getDatabaseService()
const sourceNumbers = [...new Set(records.map((record) => record.sourceNumber).filter(Boolean))]
if (dbService.type === 'sqlserver') {
return await this.replaceSqlServerJson(dbService, records, sourceNumbers)
}
const deleted = await this.deleteBySourceNumbers(sourceNumbers)
const inserted = await this.batchInsert(records, batchSize)
return { deleted, inserted }
}
private async replaceSqlServerJson(
dbService: IDatabaseService,
records: MaterialPlanRecord[],
sourceNumbers: string[]
): Promise<{ deleted: number; inserted: number }> {
const tableName = this.getTableName()
const columns = this.getInsertColumns()
const withColumns = this.getSqlServerJsonWithColumns(columns)
const quotedColumns = columns.map((column) => `[${column}]`).join(', ')
const recordsBySourceNumber = this.groupRecordsBySourceNumber(records)
const totalBatches = Math.ceil(
sourceNumbers.length / SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
)
let totalDeleted = 0
let totalInserted = 0
log.info('SQL Server JSON replace started', {
tableName,
operationType: 'REPLACE',
totalSourceNumbers: sourceNumbers.length,
totalRecords: records.length,
sourceNumberBatchSize: SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE,
totalBatches
})
for (
let offset = 0;
offset < sourceNumbers.length;
offset += SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
) {
const sourceNumberBatch = sourceNumbers.slice(
offset,
offset + SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE
)
const batchNumber = Math.floor(offset / SQLSERVER_REPLACE_SOURCE_NUMBER_BATCH_SIZE) + 1
const recordBatch = sourceNumberBatch.flatMap(
(sourceNumber) => recordsBySourceNumber.get(sourceNumber) || []
)
const jsonRows = recordBatch.map((record) => this.buildJsonRow(record, columns))
const sqlString = `
DECLARE @deleted int = 0;
DECLARE @inserted int = 0;
BEGIN TRY
BEGIN TRANSACTION;
DELETE target
FROM ${tableName} AS target
INNER JOIN OPENJSON(@p0)
WITH (SourceNumber nvarchar(100) '$') AS source
ON target.SourceNumber = source.SourceNumber;
SET @deleted = @@ROWCOUNT;
INSERT INTO ${tableName} (${quotedColumns})
SELECT ${quotedColumns}
FROM OPENJSON(@p1)
WITH (
${withColumns}
);
SET @inserted = @@ROWCOUNT;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
IF @@TRANCOUNT > 0
ROLLBACK TRANSACTION;
THROW;
END CATCH;
SELECT @deleted AS deletedCount, @inserted AS insertedCount;
`
const result = await trackDuration(
async () =>
await dbService.query(sqlString, [
JSON.stringify(sourceNumberBatch),
JSON.stringify(jsonRows)
]),
{
operationName: 'DiscreteMaterialPlanDAO.replaceSqlServerJsonBatch',
context: {
tableName,
operationType: 'REPLACE',
batchNumber,
totalBatches,
sourceNumberCount: sourceNumberBatch.length,
recordCount: recordBatch.length
}
}
)
const stats = result.result.rows[0] || {}
totalDeleted += Number(stats.deletedCount || 0)
totalInserted += Number(stats.insertedCount || recordBatch.length)
log.debug('SQL Server JSON replace batch completed', {
tableName,
batchNumber,
totalBatches,
sourceNumberCount: sourceNumberBatch.length,
recordCount: recordBatch.length
})
}
log.info('SQL Server JSON replace completed', {
tableName,
operationType: 'REPLACE',
totalDeleted,
totalInserted,
totalBatches
})
return {
deleted: totalDeleted,
inserted: totalInserted
}
}
private groupRecordsBySourceNumber(
records: MaterialPlanRecord[]
): Map<string, MaterialPlanRecord[]> {
const groups = new Map<string, MaterialPlanRecord[]>()
for (const record of records) {
if (!record.sourceNumber) {
continue
}
const existing = groups.get(record.sourceNumber) || []
existing.push(record)
groups.set(record.sourceNumber, existing)
}
return groups
}
private async batchInsertSqlServerJson(
dbService: IDatabaseService,
tableName: string,
records: MaterialPlanRecord[],
batchSize: number,
batchId: string
): Promise<number> {
const columns = this.getInsertColumns()
const effectiveBatchSize = Math.max(1, batchSize)
const totalBatches = Math.ceil(records.length / effectiveBatchSize)
let totalInserted = 0
log.info('SQL Server JSON batch insert started', {
tableName,
operationType: 'INSERT',
requestId: batchId,
totalRecords: records.length,
effectiveBatchSize,
totalBatches
})
for (let i = 0; i < records.length; i += effectiveBatchSize) {
const batch = records.slice(i, i + effectiveBatchSize)
const batchNumber = Math.floor(i / effectiveBatchSize) + 1
const jsonRows = batch.map((record) => this.buildJsonRow(record, columns))
const withColumns = this.getSqlServerJsonWithColumns(columns)
const quotedColumns = columns.map((column) => `[${column}]`).join(', ')
const sqlString = `
INSERT INTO ${tableName} (${quotedColumns})
SELECT ${quotedColumns}
FROM OPENJSON(@p0)
WITH (
${withColumns}
)
`
const result = await trackDuration(
async () => await dbService.query(sqlString, [JSON.stringify(jsonRows)]),
{
operationName: 'DiscreteMaterialPlanDAO.insertBatchSqlServerJson',
context: {
tableName,
operationType: 'INSERT',
batchId,
batchNumber,
totalBatches,
recordCount: batch.length
}
}
)
totalInserted += result.result.rowCount || batch.length
log.debug('Inserted SQL Server JSON batch', {
batch: batchNumber,
totalBatches,
count: batch.length,
batchId
})
}
log.info('SQL Server JSON batch insert completed', {
tableName,
operationType: 'INSERT',
requestId: batchId,
totalInserted,
batchSize: effectiveBatchSize,
totalBatches
})
return totalInserted
}
/** /**
* Insert a single batch of records with tracking * Insert a single batch of records with tracking
*/ */
@@ -641,37 +880,7 @@ export class DiscreteMaterialPlanDAO {
return 0 return 0
} }
// Build column list (excluding id) const columns = this.getInsertColumns()
const columns = [
'Factory',
'MaterialStatus',
'PlanNumber',
'SourceNumber',
'MaterialType',
'ProductCode',
'ProductName',
'ProductUnit',
'ProductPlanQuantity',
'UseDepartment',
'Remark',
'Creator',
'CreateDate',
'Approver',
'ApproveDate',
'SequenceNumber',
'MaterialCode',
'MaterialName',
'Specification',
'Model',
'DrawingNumber',
'MaterialQuality',
'PlanQuantity',
'Unit',
'RequiredDate',
'Warehouse',
'UnitUsage',
'CumulativeOutputQuantity'
]
// Build parameterized insert // Build parameterized insert
const values: any[] = [] const values: any[] = []
@@ -730,6 +939,91 @@ export class DiscreteMaterialPlanDAO {
}) })
} }
private getInsertColumns(): string[] {
return [
'Factory',
'MaterialStatus',
'PlanNumber',
'SourceNumber',
'MaterialType',
'ProductCode',
'ProductName',
'ProductUnit',
'ProductPlanQuantity',
'UseDepartment',
'Remark',
'Creator',
'CreateDate',
'Approver',
'ApproveDate',
'SequenceNumber',
'MaterialCode',
'MaterialName',
'Specification',
'Model',
'DrawingNumber',
'MaterialQuality',
'PlanQuantity',
'Unit',
'RequiredDate',
'Warehouse',
'UnitUsage',
'CumulativeOutputQuantity'
]
}
private buildJsonRow(record: MaterialPlanRecord, columns: string[]): Record<string, unknown> {
const row: Record<string, unknown> = {}
for (const column of columns) {
const value = this.getColumnValue(record, column)
row[column] = value instanceof Date ? value.toISOString() : value
}
return row
}
private getSqlServerJsonWithColumns(columns: string[]): string {
return columns
.map((column) => `[${column}] ${this.getSqlServerJsonColumnType(column)} '$.${column}'`)
.join(',\n ')
}
private getSqlServerJsonColumnType(column: string): string {
const columnTypes: Record<string, string> = {
Factory: 'nvarchar(100)',
MaterialStatus: 'nvarchar(50)',
PlanNumber: 'nvarchar(100)',
SourceNumber: 'nvarchar(100)',
MaterialType: 'nvarchar(100)',
ProductCode: 'nvarchar(100)',
ProductName: 'nvarchar(255)',
ProductUnit: 'nvarchar(50)',
ProductPlanQuantity: 'decimal(18,4)',
UseDepartment: 'nvarchar(100)',
Remark: 'nvarchar(500)',
Creator: 'nvarchar(100)',
CreateDate: 'datetime2',
Approver: 'nvarchar(100)',
ApproveDate: 'datetime2',
SequenceNumber: 'int',
MaterialCode: 'nvarchar(100)',
MaterialName: 'nvarchar(255)',
Specification: 'nvarchar(255)',
Model: 'nvarchar(255)',
DrawingNumber: 'nvarchar(100)',
MaterialQuality: 'nvarchar(100)',
PlanQuantity: 'decimal(18,4)',
Unit: 'nvarchar(50)',
RequiredDate: 'datetime2',
Warehouse: 'nvarchar(100)',
UnitUsage: 'decimal(18,6)',
CumulativeOutputQuantity: 'decimal(18,4)'
}
return columnTypes[column] || 'nvarchar(max)'
}
/** /**
* Get the value for a specific column from the record * Get the value for a specific column from the record
*/ */
@@ -777,6 +1071,10 @@ export class DiscreteMaterialPlanDAO {
return null return null
} }
if (value instanceof Date && Number.isNaN(value.getTime())) {
return null
}
// Handle empty strings for string fields // Handle empty strings for string fields
if (typeof value === 'string' && value.trim() === '') { if (typeof value === 'string' && value.trim() === '') {
return null return null

View File

@@ -126,35 +126,47 @@ export class ExtractorOperationHistoryDAO {
recordCount: records.length recordCount: records.length
}) })
for (const record of records) { const columnsPerRecord = 5
const batchSize = Math.max(1, dialect.maxBatchRows(columnsPerRecord))
for (let offset = 0; offset < records.length; offset += batchSize) {
const batch = records.slice(offset, offset + batchSize)
try { try {
const valuesSql: string[] = []
const params: (string | number | null)[] = []
batch.forEach((record, index) => {
const paramOffset = index * columnsPerRecord
valuesSql.push(
`(${dialect.param(paramOffset)}, ${dialect.param(paramOffset + 1)}, ${dialect.param(paramOffset + 2)}, ${dialect.param(paramOffset + 3)}, ${dialect.param(paramOffset + 4)}, ${dialect.currentTimestamp()}, 'pending')`
)
params.push(batchId, userId, username, record.productionId || null, record.orderNumber)
})
const sqlString = ` const sqlString = `
INSERT INTO ${tableName} INSERT INTO ${tableName}
(BatchId, UserId, Username, ProductionId, OrderNumber, OperationTime, Status) (BatchId, UserId, Username, ProductionId, OrderNumber, OperationTime, Status)
VALUES VALUES
(${dialect.param(0)}, ${dialect.param(1)}, ${dialect.param(2)}, ${dialect.param(3)}, ${dialect.param(4)}, ${dialect.currentTimestamp()}, 'pending') ${valuesSql.join(',\n ')}
` `
await trackDuration( await trackDuration(async () => await dbService.query(sqlString, params), {
async () =>
await dbService.query(sqlString, [
batchId,
userId,
username,
record.productionId || null,
record.orderNumber
]),
{
operationName: 'ExtractorOperationHistoryDAO.insertBatchRecords', operationName: 'ExtractorOperationHistoryDAO.insertBatchRecords',
context: { tableName, operationType: 'INSERT', batchId } context: {
tableName,
operationType: 'INSERT',
batchId,
batchOffset: offset,
batchCount: batch.length
} }
) })
} catch (error) { } catch (error) {
log.error('Error inserting individual record', { log.error('Error inserting record batch', {
tableName, tableName,
operationType: 'INSERT', operationType: 'INSERT',
requestId, requestId,
batchId, batchId,
orderNumber: record.orderNumber, batchOffset: offset,
batchCount: batch.length,
error: error instanceof Error ? error.message : String(error) error: error instanceof Error ? error.message : String(error)
}) })
} }

View File

@@ -339,7 +339,11 @@ const SQL_KEYWORDS = new Set([
'IF', 'IF',
'CURRENT_TIMESTAMP', 'CURRENT_TIMESTAMP',
'NOW', 'NOW',
'GETDATE' 'GETDATE',
// ==================== Timezone Expression ====================
'AT',
'ZONE'
]) ])
/** /**

View File

@@ -38,6 +38,8 @@ export class SqlServerService implements IDatabaseService {
user: this.config.user, user: this.config.user,
password: this.config.password, password: this.config.password,
database: this.config.database, database: this.config.database,
requestTimeout: 60000,
connectionTimeout: 15000,
options: { options: {
encrypt: this.config.options?.encrypt ?? false, encrypt: this.config.options?.encrypt ?? false,
trustServerCertificate: this.config.options?.trustServerCertificate ?? false trustServerCertificate: this.config.options?.trustServerCertificate ?? false

View File

@@ -169,6 +169,7 @@ export interface ShouldDeleteParams {
pendingQty: string pendingQty: string
materialCode: string materialCode: string
deleteSet: Set<string> deleteSet: Set<string>
enableRowProtection: boolean
} }
function clampNumber( function clampNumber(
@@ -244,13 +245,13 @@ export class CleanerService {
* Determine if a material should be deleted * Determine if a material should be deleted
*/ */
shouldDeleteMaterial(params: ShouldDeleteParams): boolean { shouldDeleteMaterial(params: ShouldDeleteParams): boolean {
const { rowNumber, pendingQty, materialCode, deleteSet } = params const { rowNumber, pendingQty, materialCode, deleteSet, enableRowProtection } = params
if (!deleteSet.has(materialCode)) { if (!deleteSet.has(materialCode)) {
return false return false
} }
if (rowNumber >= 2000 && rowNumber < 8000) { if (enableRowProtection && rowNumber >= 2000 && rowNumber < 8000) {
return false return false
} }
@@ -262,12 +263,12 @@ export class CleanerService {
} }
getSkipReason(params: ShouldDeleteParams): string { getSkipReason(params: ShouldDeleteParams): string {
const { rowNumber, pendingQty, materialCode, deleteSet } = params const { rowNumber, pendingQty, materialCode, deleteSet, enableRowProtection } = params
if (!deleteSet.has(materialCode)) { if (!deleteSet.has(materialCode)) {
return '物料不在删除清单中' return '物料不在删除清单中'
} }
if (rowNumber >= 2000 && rowNumber < 8000) { if (enableRowProtection && rowNumber >= 2000 && rowNumber < 8000) {
return '行号在 2000-7999 范围内(受保护)' return '行号在 2000-7999 范围内(受保护)'
} }
if (pendingQty && pendingQty.trim() !== '') { if (pendingQty && pendingQty.trim() !== '') {
@@ -317,6 +318,7 @@ export class CleanerService {
input.sessionRefreshOrderThreshold && input.sessionRefreshOrderThreshold > 0 input.sessionRefreshOrderThreshold && input.sessionRefreshOrderThreshold > 0
? Math.trunc(input.sessionRefreshOrderThreshold) ? Math.trunc(input.sessionRefreshOrderThreshold)
: DEFAULT_SESSION_REFRESH_ORDER_THRESHOLD : DEFAULT_SESSION_REFRESH_ORDER_THRESHOLD
const enableRowProtection = input.enableRowProtection ?? true
log.info('Starting cleaner', { log.info('Starting cleaner', {
totalOrders, totalOrders,
@@ -446,6 +448,7 @@ export class CleanerService {
detailPage: openedDetailPage, detailPage: openedDetailPage,
deleteSet, deleteSet,
dryRun, dryRun,
enableRowProtection,
expectedOrderNumber: orderNumber, expectedOrderNumber: orderNumber,
progressState, progressState,
onProgress: input.onProgress onProgress: input.onProgress
@@ -567,6 +570,7 @@ export class CleanerService {
), ),
deleteSet, deleteSet,
dryRun, dryRun,
enableRowProtection,
onProgress: input.onProgress onProgress: input.onProgress
}) })
@@ -1149,6 +1153,7 @@ export class CleanerService {
detailPage: Page detailPage: Page
deleteSet: Set<string> deleteSet: Set<string>
dryRun: boolean dryRun: boolean
enableRowProtection: boolean
progressState: ProgressState progressState: ProgressState
expectedOrderNumber?: string expectedOrderNumber?: string
onProgress?: ( onProgress?: (
@@ -1157,7 +1162,7 @@ export class CleanerService {
extra?: Partial<import('../../types/cleaner.types').CleanerProgress> extra?: Partial<import('../../types/cleaner.types').CleanerProgress>
) => void ) => void
}): Promise<OrderCleanDetail> { }): Promise<OrderCleanDetail> {
const { detailPage, deleteSet, dryRun, progressState, expectedOrderNumber, onProgress } = params const { detailPage, deleteSet, dryRun, enableRowProtection, progressState, expectedOrderNumber, onProgress } = params
const processStartTime = Date.now() const processStartTime = Date.now()
log.info('[ORDER_START] 开始处理订单', { log.info('[ORDER_START] 开始处理订单', {
@@ -1482,7 +1487,8 @@ export class CleanerService {
rowNumber: rowNumInt, rowNumber: rowNumInt,
pendingQty, pendingQty,
materialCode, materialCode,
deleteSet deleteSet,
enableRowProtection
}) })
if (shouldDelete && !dryRun) { if (shouldDelete && !dryRun) {
@@ -1559,7 +1565,8 @@ export class CleanerService {
rowNumber: rowNumInt, rowNumber: rowNumInt,
pendingQty, pendingQty,
materialCode, materialCode,
deleteSet deleteSet,
enableRowProtection
}) })
log.debug('[物料跳过] 物料不满足删除条件', { log.debug('[物料跳过] 物料不满足删除条件', {
orderNumber, orderNumber,
@@ -2097,13 +2104,14 @@ export class CleanerService {
failedDetails: OrderCleanDetail[] failedDetails: OrderCleanDetail[]
deleteSet: Set<string> deleteSet: Set<string>
dryRun: boolean dryRun: boolean
enableRowProtection: boolean
onProgress?: ( onProgress?: (
message: string, message: string,
progress?: number, progress?: number,
extra?: Partial<import('../../types/cleaner.types').CleanerProgress> extra?: Partial<import('../../types/cleaner.types').CleanerProgress>
) => void ) => void
}): Promise<RetryResult> { }): Promise<RetryResult> {
const { workFrame, popupPage, failedDetails, deleteSet, dryRun, onProgress } = params const { workFrame, popupPage, failedDetails, deleteSet, dryRun, enableRowProtection, onProgress } = params
const result: RetryResult = { const result: RetryResult = {
retriedOrders: 0, retriedOrders: 0,
@@ -2204,6 +2212,7 @@ export class CleanerService {
detailPage, detailPage,
deleteSet, deleteSet,
dryRun, dryRun,
enableRowProtection,
expectedOrderNumber: orderNumber, expectedOrderNumber: orderNumber,
progressState: { progressState: {
ordersStarted: detailIndex, ordersStarted: detailIndex,

View File

@@ -10,6 +10,7 @@ import type {
LogLevel LogLevel
} from '../../types/extractor.types' } from '../../types/extractor.types'
import { DataImportService } from '../database/data-importer' import { DataImportService } from '../database/data-importer'
import type { MaterialPlanRecord } from '../database/discrete-material-plan-dao'
import { createLogger, withRequestContext, getRequestId } from '../logger' import { createLogger, withRequestContext, getRequestId } from '../logger'
import { trackDuration } from '../logger/performance-monitor' import { trackDuration } from '../logger/performance-monitor'
@@ -112,16 +113,18 @@ export class ExtractorService {
// Always clean up temporary files regardless of merge success // Always clean up temporary files regardless of merge success
await this.cleanupTempFiles(result.downloadedFiles, input.orderNumbers) await this.cleanupTempFiles(result.downloadedFiles, input.orderNumbers)
// Auto-import to database if merge was successful // Auto-import parsed records directly. The merged Excel file is an archive artifact,
if (result.mergedFile) { // not the source for persistence.
if (mergeResult.records.length > 0) {
const importProgress = (1 + totalBatches + 1) * progressPerPoint const importProgress = (1 + totalBatches + 1) * progressPerPoint
input.onProgress?.('正在写入数据库...', importProgress, { input.onProgress?.('正在写入数据库...', importProgress, {
phase: 'importing', phase: 'importing',
totalBatches totalBatches
}) })
const importResult = await this.importToDatabaseWithLogging( const importResult = await this.importRecordsToDatabaseWithLogging(
result.mergedFile, mergeResult.records,
input.onLog input.onLog,
result.mergedFile
) )
result.importResult = importResult result.importResult = importResult
@@ -168,9 +171,10 @@ export class ExtractorService {
recordCount: number recordCount: number
error?: string error?: string
orderRecordCounts: Array<{ orderNumber: string; recordCount: number }> orderRecordCounts: Array<{ orderNumber: string; recordCount: number }>
records: MaterialPlanRecord[]
}> { }> {
if (filePaths.length === 0) { if (filePaths.length === 0) {
return { mergedFile: null, recordCount: 0, orderRecordCounts: [] } return { mergedFile: null, recordCount: 0, orderRecordCounts: [], records: [] }
} }
log.info('Starting merge', { fileCount: filePaths.length, orderCount: orderNumbers.length }) log.info('Starting merge', { fileCount: filePaths.length, orderCount: orderNumbers.length })
@@ -219,10 +223,11 @@ export class ExtractorService {
} }
log.info('Merge summary', { orderCount: allOrders.length, recordCount }) log.info('Merge summary', { orderCount: allOrders.length, recordCount })
const records = this.buildMaterialPlanRecords(allOrders)
if (recordCount === 0) { if (recordCount === 0) {
log.warn('No records found in any downloaded files', { orderNumbers }) log.warn('No records found in any downloaded files', { orderNumbers })
return { mergedFile: null, recordCount: 0, orderRecordCounts } return { mergedFile: null, recordCount: 0, orderRecordCounts, records }
} }
// Generate output filename with timestamp // Generate output filename with timestamp
@@ -238,7 +243,7 @@ export class ExtractorService {
log.info('Saving merged file', { outputPath }) log.info('Saving merged file', { outputPath })
await this.saveMergedOrders(allOrders, outputPath) await this.saveMergedOrders(allOrders, outputPath)
log.info('Merged file saved successfully', { recordCount }) log.info('Merged file saved successfully', { recordCount })
return { mergedFile: outputPath, recordCount, orderRecordCounts } return { mergedFile: outputPath, recordCount, orderRecordCounts, records }
} catch (error) { } catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error) const errorMsg = error instanceof Error ? error.message : String(error)
const errorStack = error instanceof Error ? error.stack : '' const errorStack = error instanceof Error ? error.stack : ''
@@ -253,6 +258,7 @@ export class ExtractorService {
mergedFile: null, mergedFile: null,
recordCount, recordCount,
orderRecordCounts, orderRecordCounts,
records,
error: `保存合并文件失败:${errorMsg}` error: `保存合并文件失败:${errorMsg}`
} }
} }
@@ -370,6 +376,78 @@ export class ExtractorService {
log.debug('File saved successfully', { outputPath }) log.debug('File saved successfully', { outputPath })
} }
private buildMaterialPlanRecords(
orders: Array<{ orderInfo: any; materials: any[] }>
): MaterialPlanRecord[] {
const records: MaterialPlanRecord[] = []
for (const order of orders) {
const { orderInfo, materials } = order
for (const material of materials) {
records.push({
factory: this.toText(orderInfo.factory),
materialStatus: this.toText(orderInfo.materialStatus),
planNumber: this.toText(orderInfo.planNumber),
sourceNumber: this.toText(orderInfo.productionOrder),
materialType: this.toText(orderInfo.materialType),
productCode: this.toText(orderInfo.productCode),
productName: this.toText(orderInfo.productName),
productPlanQuantity: this.toNumber(orderInfo.plannedQuantity),
productUnit: this.toText(orderInfo.unit),
useDepartment: this.toText(orderInfo.department),
remark: this.toText(orderInfo.remark),
creator: this.toText(orderInfo.creator),
createDate: this.toDate(orderInfo.createDate),
approver: this.toText(orderInfo.approver),
approveDate: this.toDate(orderInfo.approveDate),
sequenceNumber: this.toNumber(material.sequence),
materialCode: this.toText(material.materialCode),
materialName: this.toText(material.materialName),
specification: this.toText(material.specification),
model: this.toText(material.model),
drawingNumber: this.toText(material.drawingNumber),
materialQuality: this.toText(material.material),
planQuantity: this.toNumber(material.quantity),
unit: this.toText(material.unit),
requiredDate: this.toDate(material.requiredDate),
warehouse: this.toText(material.warehouse),
unitUsage: this.toNumber(material.unitUsage),
cumulativeOutputQuantity: this.toNumber(material.cumulativeOutboundQty),
bomVersion: ''
})
}
}
return records
}
private toText(value: unknown): string {
if (value === null || value === undefined) {
return ''
}
return String(value).trim()
}
private toNumber(value: unknown): number {
if (value === null || value === undefined || value === '') {
return 0
}
const parsed = Number(value)
return Number.isFinite(parsed) ? parsed : 0
}
private toDate(value: unknown): Date {
if (value instanceof Date) {
return value
}
if (value === null || value === undefined || value === '') {
return new Date(NaN)
}
const parsed = new Date(String(value))
return parsed
}
/** /**
* Clean up temporary batch files after merging * Clean up temporary batch files after merging
* @param filePaths - Array of temporary file paths to delete * @param filePaths - Array of temporary file paths to delete
@@ -458,4 +536,70 @@ export class ExtractorService {
return trackedResult.result return trackedResult.result
} }
private async importRecordsToDatabaseWithLogging(
records: MaterialPlanRecord[],
onLog?: (level: LogLevel, message: string) => void,
archiveFilePath?: string | null
): Promise<ImportResult> {
log.info('Starting database import from parsed records', {
recordCount: records.length,
archiveFilePath
})
onLog?.('info', `开始导入数据到数据库...`)
const trackedResult = await trackDuration(
async () => {
const importService = new DataImportService()
try {
const result = await importService.importFromRecords(records, 1000)
log.info('Import completed', {
success: result.success,
recordsRead: result.recordsRead,
recordsDeleted: result.recordsDeleted,
recordsImported: result.recordsImported
})
if (result.success) {
onLog?.(
'success',
`导入完成:读取 ${result.recordsRead} 条,删除 ${result.recordsDeleted} 条,导入 ${result.recordsImported}`
)
} else if (result.errors.length > 0) {
result.errors.forEach((err) => onLog?.('error', err))
}
return result
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error)
log.error('Import failed', {
error: errorMsg,
archiveFilePath,
downloadDir: this.downloadDir
})
onLog?.('error', `导入失败:${errorMsg}`)
return {
success: false,
recordsRead: 0,
recordsDeleted: 0,
recordsImported: 0,
uniqueSourceNumbers: 0,
errors: [errorMsg]
}
}
},
{
operationName: 'Database Import',
context: {
recordCount: records.length,
archiveFilePath
}
}
)
return trackedResult.result
}
} }

View File

@@ -32,6 +32,8 @@ const PRODUCTION_ID_PATTERN = /^\d{2}[A-Z]\d{1,6}$/i
*/ */
const ORDER_NUMBER_PATTERN = /^SC\d{14}$/i const ORDER_NUMBER_PATTERN = /^SC\d{14}$/i
const RESOLUTION_QUERY_BATCH_SIZE = 1000
/** /**
* Database table and field names * Database table and field names
* Loaded from config.yaml via ConfigManager * Loaded from config.yaml via ConfigManager
@@ -56,6 +58,14 @@ export class OrderNumberResolver {
this.dbService = dbService this.dbService = dbService
} }
private chunk<T>(items: T[], size: number): T[][] {
const chunks: T[][] = []
for (let index = 0; index < items.length; index += size) {
chunks.push(items.slice(index, index + size))
}
return chunks
}
/** /**
* Get table name based on database type * Get table name based on database type
* Converts schema.tablename format to database-specific quoting: * Converts schema.tablename format to database-specific quoting:
@@ -156,30 +166,29 @@ export class OrderNumberResolver {
// P1: Deduplicate input productionIds to avoid redundant queries // P1: Deduplicate input productionIds to avoid redundant queries
const uniqueProductionIds = [...new Set(productionIds)] const uniqueProductionIds = [...new Set(productionIds)]
// Use parameterized query to prevent SQL injection const mappings = new Map<string, string>()
const placeholders = uniqueProductionIds.map((_, i) => `@p${i}`).join(', ') const batches = this.chunk(uniqueProductionIds, RESOLUTION_QUERY_BATCH_SIZE)
const params = uniqueProductionIds
for (const batch of batches) {
let sql: string let sql: string
const params = batch
if (this.dbService.type === 'sqlserver') { if (this.dbService.type === 'sqlserver') {
// P0: Use DISTINCT to prevent duplicates from one-to-many relationships const placeholders = batch.map((_, i) => `@p${i}`).join(', ')
// 使用 COLLATE 指定不区分大小写的排序规则 // P0: Use DISTINCT to prevent duplicates from one-to-many relationships.
sql = `SELECT DISTINCT [${dbConfig.FIELD_PRODUCTION_ID}], [${dbConfig.FIELD_ORDER_NUMBER}] FROM ${tableName} WHERE [${dbConfig.FIELD_PRODUCTION_ID}] COLLATE SQL_Latin1_General_CP1_CI_AS IN (${placeholders})` sql = `SELECT DISTINCT [${dbConfig.FIELD_PRODUCTION_ID}], [${dbConfig.FIELD_ORDER_NUMBER}] FROM ${tableName} WHERE [${dbConfig.FIELD_PRODUCTION_ID}] COLLATE SQL_Latin1_General_CP1_CI_AS IN (${placeholders})`
} else if (this.dbService.type === 'postgresql') { } else if (this.dbService.type === 'postgresql') {
// PostgreSQL: 使用双引号保护中文标识符UPPER 实现不区分大小写 // PostgreSQL: 使用双引号保护中文标识符UPPER 实现不区分大小写
// 注意getTableName() 已返回带双引号的表名,不应再加引号 const pgPlaceholders = batch.map((_, i) => `UPPER($${i + 1})`).join(', ')
const pgPlaceholders = uniqueProductionIds.map((_, i) => `UPPER($${i + 1})`).join(', ')
sql = `SELECT DISTINCT "${dbConfig.FIELD_PRODUCTION_ID}", "${dbConfig.FIELD_ORDER_NUMBER}" FROM ${tableName} WHERE UPPER("${dbConfig.FIELD_PRODUCTION_ID}") IN (${pgPlaceholders})` sql = `SELECT DISTINCT "${dbConfig.FIELD_PRODUCTION_ID}", "${dbConfig.FIELD_ORDER_NUMBER}" FROM ${tableName} WHERE UPPER("${dbConfig.FIELD_PRODUCTION_ID}") IN (${pgPlaceholders})`
} else { } else {
const idPlaceholders = uniqueProductionIds.map(() => 'UPPER(?)').join(', ') const idPlaceholders = batch.map(() => 'UPPER(?)').join(', ')
// P0: Use DISTINCT to prevent duplicates from one-to-many relationships // MySQL: 使用 UPPER 确保不区分大小写。
// MySQL: 使用 UPPER 确保不区分大小写
sql = `SELECT DISTINCT \`${dbConfig.FIELD_PRODUCTION_ID}\`, \`${dbConfig.FIELD_ORDER_NUMBER}\` FROM \`${tableName}\` WHERE UPPER(\`${dbConfig.FIELD_PRODUCTION_ID}\`) IN (${idPlaceholders})` sql = `SELECT DISTINCT \`${dbConfig.FIELD_PRODUCTION_ID}\`, \`${dbConfig.FIELD_ORDER_NUMBER}\` FROM \`${tableName}\` WHERE UPPER(\`${dbConfig.FIELD_PRODUCTION_ID}\`) IN (${idPlaceholders})`
} }
const result = await this.dbService.query(sql, params) const result = await this.dbService.query(sql, params)
const mappings = new Map<string, string>()
for (const row of result.rows) { for (const row of result.rows) {
const keys = Object.keys(row) const keys = Object.keys(row)
const prodId = row[keys[0]] as string const prodId = row[keys[0]] as string
@@ -188,6 +197,7 @@ export class OrderNumberResolver {
mappings.set(prodId, orderNum) mappings.set(prodId, orderNum)
} }
} }
}
return mappings return mappings
} catch (error) { } catch (error) {
@@ -235,13 +245,14 @@ export class OrderNumberResolver {
// Build results while preserving original input order // Build results while preserving original input order
// Note: Multiple productionIDs mapping to the same order number is VALID (not an error) // Note: Multiple productionIDs mapping to the same order number is VALID (not an error)
const results: OrderMapping[] = [] const results: OrderMapping[] = []
const processedInputs = new Set<string>()
for (const input of inputs) { for (const input of inputs) {
// Skip if this exact input was already processed // Skip if this exact input was already processed
const alreadyProcessed = results.some((r) => r.input === input) if (processedInputs.has(input)) {
if (alreadyProcessed) {
continue continue
} }
processedInputs.add(input)
const mapping: OrderMapping = { input, resolved: false } const mapping: OrderMapping = { input, resolved: false }

View File

@@ -20,6 +20,7 @@ export interface CleanerInput {
queryBatchSize?: number queryBatchSize?: number
processConcurrency?: number processConcurrency?: number
sessionRefreshOrderThreshold?: number sessionRefreshOrderThreshold?: number
enableRowProtection?: boolean
onProgress?: (message: string, progress?: number, extra?: Partial<CleanerProgress>) => void onProgress?: (message: string, progress?: number, extra?: Partial<CleanerProgress>) => void
} }

View File

@@ -118,7 +118,8 @@ export const validationConfigSchema = z.object({
export const cleanerConfigSchema = z.object({ export const cleanerConfigSchema = z.object({
queryBatchSize: z.number().int().min(1).max(100).default(100), queryBatchSize: z.number().int().min(1).max(100).default(100),
processConcurrency: z.number().int().min(1).max(20).default(1), processConcurrency: z.number().int().min(1).max(20).default(1),
sessionRefreshOrderThreshold: z.number().int().positive().default(160) sessionRefreshOrderThreshold: z.number().int().positive().default(160),
enableRowProtection: z.boolean().default(true)
}) })
export type CleanerConfig = z.infer<typeof cleanerConfigSchema> export type CleanerConfig = z.infer<typeof cleanerConfigSchema>

View File

@@ -62,12 +62,12 @@ const formatDateTime = (dateStr: string) => {
return dateStr // Return original if invalid return dateStr // Return original if invalid
} }
// Use UTC methods to display the time as stored in database (without timezone conversion) // Use local time for display
const year = date.getUTCFullYear() const year = date.getFullYear()
const month = String(date.getUTCMonth() + 1).padStart(2, '0') const month = String(date.getMonth() + 1).padStart(2, '0')
const day = String(date.getUTCDate()).padStart(2, '0') const day = String(date.getDate()).padStart(2, '0')
const hours = String(date.getUTCHours()).padStart(2, '0') const hours = String(date.getHours()).padStart(2, '0')
const minutes = String(date.getUTCMinutes()).padStart(2, '0') const minutes = String(date.getMinutes()).padStart(2, '0')
return `${year}-${month}-${day} ${hours}:${minutes}` return `${year}-${month}-${day} ${hours}:${minutes}`
} }

View File

@@ -8,6 +8,8 @@ interface CleanerExecutionBarProps {
setDryRun: (value: boolean) => void setDryRun: (value: boolean) => void
headless: boolean headless: boolean
setHeadless: (value: boolean) => void setHeadless: (value: boolean) => void
enableRowProtection: boolean
setEnableRowProtection: (value: boolean) => void
processConcurrency: number processConcurrency: number
updateProcessConcurrency: (value: number) => void updateProcessConcurrency: (value: number) => void
showSettingsMenu: boolean showSettingsMenu: boolean
@@ -30,7 +32,9 @@ export function CleanerExecutionBar({
setShowSettingsMenu, setShowSettingsMenu,
handleExecuteDeletion, handleExecuteDeletion,
isRunning, isRunning,
executeButtonRef executeButtonRef,
enableRowProtection,
setEnableRowProtection
}: CleanerExecutionBarProps): React.JSX.Element { }: CleanerExecutionBarProps): React.JSX.Element {
return ( return (
<div className="bg-white border-t border-slate-200 p-4 flex justify-between items-center shadow-[0_-4px_6px_-1px_rgba(0,0,0,0.05)] z-10 flex-shrink-0"> <div className="bg-white border-t border-slate-200 p-4 flex justify-between items-center shadow-[0_-4px_6px_-1px_rgba(0,0,0,0.05)] z-10 flex-shrink-0">
@@ -82,6 +86,22 @@ export function CleanerExecutionBar({
</button> </button>
</div> </div>
</div> </div>
<div className="border-t border-slate-100 pt-3">
<div className="flex items-center justify-between">
<div>
<div className="text-sm font-medium text-slate-800"></div>
<div className="text-xs text-slate-500 mt-0.5">
2000-7999
</div>
</div>
<button
onClick={() => setEnableRowProtection(!enableRowProtection)}
className={`transition-colors flex-shrink-0 ml-4 ${enableRowProtection ? 'text-green-500' : 'text-slate-300'}`}
>
{enableRowProtection ? <ToggleRight size={32} /> : <ToggleLeft size={32} />}
</button>
</div>
</div>
<div className="border-t border-slate-100 pt-3 space-y-3"> <div className="border-t border-slate-100 pt-3 space-y-3">
<div> <div>
<div className="text-sm font-medium text-slate-800"></div> <div className="text-sm font-medium text-slate-800"></div>

View File

@@ -123,6 +123,7 @@ export async function runCleanerExecution(params: {
queryBatchSize: number queryBatchSize: number
processConcurrency: number processConcurrency: number
sessionRefreshOrderThreshold: number sessionRefreshOrderThreshold: number
enableRowProtection: boolean
selectedManagers: string[] selectedManagers: string[]
}): Promise<CleanerReportData> { }): Promise<CleanerReportData> {
const cleanerDataResult = await window.electron.validation.getCleanerData({ const cleanerDataResult = await window.electron.validation.getCleanerData({
@@ -154,7 +155,8 @@ export async function runCleanerExecution(params: {
headless: params.headless, headless: params.headless,
queryBatchSize: params.queryBatchSize, queryBatchSize: params.queryBatchSize,
processConcurrency: params.processConcurrency, processConcurrency: params.processConcurrency,
sessionRefreshOrderThreshold: params.sessionRefreshOrderThreshold sessionRefreshOrderThreshold: params.sessionRefreshOrderThreshold,
enableRowProtection: params.enableRowProtection
}) })
const cleanerRunData = response.success ? (response.data as CleanerRunPayload | null) : null const cleanerRunData = response.success ? (response.data as CleanerRunPayload | null) : null

View File

@@ -63,6 +63,9 @@ export function useCleaner() {
const [queryBatchSize, setQueryBatchSize] = useState(100) const [queryBatchSize, setQueryBatchSize] = useState(100)
const [processConcurrency, setProcessConcurrency] = useState(1) const [processConcurrency, setProcessConcurrency] = useState(1)
const [sessionRefreshOrderThreshold, setSessionRefreshOrderThreshold] = useState(160) const [sessionRefreshOrderThreshold, setSessionRefreshOrderThreshold] = useState(160)
const [enableRowProtection, setEnableRowProtection] = useState(() =>
getStoredBoolean('cleaner_enableRowProtection', true)
)
const [showSettingsMenu, setShowSettingsMenu] = useState(false) const [showSettingsMenu, setShowSettingsMenu] = useState(false)
// Inline editing state for manager field (Admin only) // Inline editing state for manager field (Admin only)
@@ -158,6 +161,10 @@ export function useCleaner() {
sessionStorage.setItem('cleaner_headless', headless.toString()) sessionStorage.setItem('cleaner_headless', headless.toString())
}, [headless]) }, [headless])
useEffect(() => {
sessionStorage.setItem('cleaner_enableRowProtection', enableRowProtection.toString())
}, [enableRowProtection])
const updateProcessConcurrency = async (value: number) => { const updateProcessConcurrency = async (value: number) => {
const clamped = Math.max(1, Math.min(20, value)) const clamped = Math.max(1, Math.min(20, value))
setProcessConcurrency(clamped) setProcessConcurrency(clamped)
@@ -380,6 +387,7 @@ export function useCleaner() {
queryBatchSize, queryBatchSize,
processConcurrency, processConcurrency,
sessionRefreshOrderThreshold, sessionRefreshOrderThreshold,
enableRowProtection,
selectedManagers: Array.from(selectedManagers) selectedManagers: Array.from(selectedManagers)
}) })
setReportData(result) setReportData(result)
@@ -445,6 +453,8 @@ export function useCleaner() {
setProcessConcurrency, setProcessConcurrency,
sessionRefreshOrderThreshold, sessionRefreshOrderThreshold,
setSessionRefreshOrderThreshold, setSessionRefreshOrderThreshold,
enableRowProtection,
setEnableRowProtection,
updateProcessConcurrency, updateProcessConcurrency,
showSettingsMenu, showSettingsMenu,
setShowSettingsMenu, setShowSettingsMenu,

View File

@@ -42,6 +42,8 @@ const CleanerPage: React.FC = () => {
setHeadless, setHeadless,
processConcurrency, processConcurrency,
updateProcessConcurrency, updateProcessConcurrency,
enableRowProtection,
setEnableRowProtection,
showSettingsMenu, showSettingsMenu,
setShowSettingsMenu, setShowSettingsMenu,
filteredResults, filteredResults,
@@ -124,6 +126,8 @@ const CleanerPage: React.FC = () => {
setDryRun={setDryRun} setDryRun={setDryRun}
headless={headless} headless={headless}
setHeadless={setHeadless} setHeadless={setHeadless}
enableRowProtection={enableRowProtection}
setEnableRowProtection={setEnableRowProtection}
processConcurrency={processConcurrency} processConcurrency={processConcurrency}
updateProcessConcurrency={updateProcessConcurrency} updateProcessConcurrency={updateProcessConcurrency}
showSettingsMenu={showSettingsMenu} showSettingsMenu={showSettingsMenu}

View File

@@ -46,8 +46,8 @@ describe('PostgreSqlDialect', () => {
}) })
describe('currentTimestamp', () => { describe('currentTimestamp', () => {
it('should return CURRENT_TIMESTAMP', () => { it('should return explicit UTC timestamp expression', () => {
expect(dialect.currentTimestamp()).toBe('CURRENT_TIMESTAMP') expect(dialect.currentTimestamp()).toBe("(NOW() AT TIME ZONE 'UTC')")
}) })
}) })

View File

@@ -0,0 +1,104 @@
import { describe, it, expect, beforeEach, vi } from 'vitest'
import {
DiscreteMaterialPlanDAO,
type MaterialPlanRecord
} from '../../../../src/main/services/database/discrete-material-plan-dao'
import type { IDatabaseService } from '../../../../src/main/services/database'
vi.mock('../../../../src/main/services/logger', () => ({
createLogger: vi.fn(() => ({
info: vi.fn(),
debug: vi.fn(),
warn: vi.fn(),
error: vi.fn()
})),
getRequestId: vi.fn(() => 'test-request-id'),
trackDuration: vi.fn(async (fn) => ({ result: await fn(), durationMs: 1, isSlow: false }))
}))
vi.mock('../../../../src/main/services/database', () => ({
create: vi.fn()
}))
function createRecord(sourceNumber: string, index: number): MaterialPlanRecord {
return {
factory: '工厂A',
materialStatus: '已审批',
planNumber: `PLAN-${index}`,
sourceNumber,
materialType: '标准',
productCode: 'P001',
productName: '产品A',
productUnit: 'PCS',
productPlanQuantity: 1,
useDepartment: '',
remark: '',
creator: '',
createDate: new Date('2026-04-28T00:00:00Z'),
approver: '',
approveDate: new Date('2026-04-28T00:00:00Z'),
sequenceNumber: index,
materialCode: `MAT-${index}`,
materialName: '物料A',
specification: '',
model: '',
drawingNumber: '',
materialQuality: '',
planQuantity: 1,
unit: 'PCS',
requiredDate: new Date('2026-04-28T00:00:00Z'),
warehouse: '',
unitUsage: 1,
cumulativeOutputQuantity: 0,
bomVersion: ''
}
}
describe('DiscreteMaterialPlanDAO', () => {
let mockDbService: IDatabaseService
beforeEach(async () => {
vi.clearAllMocks()
mockDbService = {
type: 'sqlserver',
connect: vi.fn(),
disconnect: vi.fn(),
isConnected: vi.fn(() => true),
query: vi.fn(async (_sql, params = []) => {
const rows = JSON.parse(params[1] || '[]')
return {
rows: [{ deletedCount: 0, insertedCount: rows.length }],
columns: ['deletedCount', 'insertedCount'],
rowCount: 1
}
}),
transaction: vi.fn()
}
const database = await import('../../../../src/main/services/database')
vi.mocked(database.create).mockResolvedValue(mockDbService)
})
it('splits SQL Server replace operations by source number batches', async () => {
const records = Array.from({ length: 151 }, (_, index) =>
createRecord(`SC-${String(index).padStart(4, '0')}`, index)
)
const dao = new DiscreteMaterialPlanDAO()
const result = await dao.replaceBySourceNumbers(records, 1000)
expect(result).toEqual({ deleted: 0, inserted: 151 })
expect(mockDbService.query).toHaveBeenCalledTimes(7)
const firstParams = vi.mocked(mockDbService.query).mock.calls[0][1] || []
const sixthParams = vi.mocked(mockDbService.query).mock.calls[5][1] || []
const seventhParams = vi.mocked(mockDbService.query).mock.calls[6][1] || []
expect(JSON.parse(firstParams[0])).toHaveLength(25)
expect(JSON.parse(sixthParams[0])).toHaveLength(25)
expect(JSON.parse(seventhParams[0])).toHaveLength(1)
expect(JSON.parse(firstParams[1])).toHaveLength(25)
expect(JSON.parse(sixthParams[1])).toHaveLength(25)
expect(JSON.parse(seventhParams[1])).toHaveLength(1)
})
})

View File

@@ -23,7 +23,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(true) expect(result).toBe(true)
@@ -34,7 +35,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '', pendingQty: '',
materialCode: 'MAT999', materialCode: 'MAT999',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(false) expect(result).toBe(false)
@@ -45,7 +47,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 5000, rowNumber: 5000,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(false) expect(result).toBe(false)
@@ -56,7 +59,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '5', pendingQty: '5',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(false) expect(result).toBe(false)
@@ -67,7 +71,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: ' ', pendingQty: ' ',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(true) // whitespace-only is treated as empty after trim expect(result).toBe(true) // whitespace-only is treated as empty after trim
@@ -80,7 +85,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 1999, rowNumber: 1999,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
).toBe(true) ).toBe(true)
@@ -90,7 +96,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 2000, rowNumber: 2000,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
).toBe(false) ).toBe(false)
@@ -100,7 +107,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 7999, rowNumber: 7999,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
).toBe(false) ).toBe(false)
@@ -110,10 +118,47 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 8000, rowNumber: 8000,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
).toBe(true) ).toBe(true)
}) })
it('should allow deletion when row protection is disabled (row in 2000-7999)', () => {
const result = cleaner.shouldDeleteMaterial({
rowNumber: 5000,
pendingQty: '',
materialCode: 'MAT001',
deleteSet,
enableRowProtection: false
})
expect(result).toBe(true)
})
it('should still block deletion when row protection is disabled but pendingQty is not empty', () => {
const result = cleaner.shouldDeleteMaterial({
rowNumber: 5000,
pendingQty: '10',
materialCode: 'MAT001',
deleteSet,
enableRowProtection: false
})
expect(result).toBe(false)
})
it('should still block deletion when row protection is disabled but material not in delete set', () => {
const result = cleaner.shouldDeleteMaterial({
rowNumber: 5000,
pendingQty: '',
materialCode: 'MAT999',
deleteSet,
enableRowProtection: false
})
expect(result).toBe(false)
})
}) })
describe('getSkipReason()', () => { describe('getSkipReason()', () => {
@@ -125,7 +170,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 3000, rowNumber: 3000,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(reason).toBe('行号在 2000-7999 范围内(受保护)') expect(reason).toBe('行号在 2000-7999 范围内(受保护)')
@@ -136,13 +182,15 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
const reason = cleaner.getSkipReason({ const reason = cleaner.getSkipReason({
rowNumber: 100, rowNumber: 100,
pendingQty: '', pendingQty: '',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(result).toBe(true) expect(result).toBe(true)
@@ -154,7 +202,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '', pendingQty: '',
materialCode: 'MAT999', materialCode: 'MAT999',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(reason).toBe('物料不在删除清单中') expect(reason).toBe('物料不在删除清单中')
@@ -165,7 +214,8 @@ describe('CleanerService - Helper Methods', () => {
rowNumber: 100, rowNumber: 100,
pendingQty: '10', pendingQty: '10',
materialCode: 'MAT001', materialCode: 'MAT001',
deleteSet deleteSet,
enableRowProtection: true
}) })
expect(reason).toBe('累计待发数量不为空') expect(reason).toBe('累计待发数量不为空')

View File

@@ -97,6 +97,14 @@ describe('ExtractorService', () => {
recordsImported: 0, recordsImported: 0,
uniqueSourceNumbers: 0, uniqueSourceNumbers: 0,
errors: [] errors: []
} as ImportResult),
importFromRecords: vi.fn().mockResolvedValue({
success: true,
recordsRead: 0,
recordsDeleted: 0,
recordsImported: 0,
uniqueSourceNumbers: 0,
errors: []
} as ImportResult) } as ImportResult)
} }
@@ -173,6 +181,65 @@ describe('ExtractorService', () => {
expect(Array.isArray(result.errors)).toBe(true) expect(Array.isArray(result.errors)).toBe(true)
}) })
it('should import parsed records directly instead of re-reading merged Excel', async () => {
mockExtractorCoreInstance.downloadAllBatches.mockResolvedValue({
downloadedFiles: ['./file1.xlsx'],
errors: []
})
mockExcelParserInstance.parse = vi.fn().mockImplementation(() => {
mockExcelParserInstance._lastOrders = [
{
orderInfo: {
factory: '工厂A',
planNumber: 'PLAN001',
productionOrder: 'ORD001',
productCode: 'P001',
productName: '产品A',
plannedQuantity: '10',
unit: 'PCS'
},
materials: [
{
sequence: 1,
materialCode: 'MAT001',
materialName: '物料A',
quantity: 2,
unit: 'PCS'
}
]
}
]
return Promise.resolve()
})
mockDataImportInstance.importFromRecords.mockResolvedValue({
success: true,
recordsRead: 1,
recordsDeleted: 0,
recordsImported: 1,
uniqueSourceNumbers: 1,
errors: []
} as ImportResult)
const service = new ExtractorService(mockAuthService, './test-downloads')
vi.spyOn(service as any, 'saveMergedOrders').mockResolvedValue(undefined)
const result = await service.extract({
orderNumbers: ['ORD001'],
onProgress: vi.fn(),
onLog: vi.fn()
})
expect(result.importResult?.success).toBe(true)
expect(mockDataImportInstance.importFromRecords).toHaveBeenCalledTimes(1)
expect(mockDataImportInstance.importFromExcel).not.toHaveBeenCalled()
expect(mockDataImportInstance.importFromRecords.mock.calls[0][0][0]).toMatchObject({
planNumber: 'PLAN001',
sourceNumber: 'ORD001',
materialCode: 'MAT001',
planQuantity: 2
})
})
}) })
describe('mergeFiles()', () => { describe('mergeFiles()', () => {
@@ -196,6 +263,7 @@ describe('ExtractorService', () => {
] ]
const service = new ExtractorService(mockAuthService, './test-downloads') const service = new ExtractorService(mockAuthService, './test-downloads')
vi.spyOn(service as any, 'saveMergedOrders').mockResolvedValue(undefined)
// @ts-ignore - accessing private method for testing // @ts-ignore - accessing private method for testing
const result = await service.mergeFiles(['./file1.xlsx'], ['ORD001']) const result = await service.mergeFiles(['./file1.xlsx'], ['ORD001'])
@@ -231,6 +299,7 @@ describe('ExtractorService', () => {
}) })
const service = new ExtractorService(mockAuthService, './test-downloads') const service = new ExtractorService(mockAuthService, './test-downloads')
vi.spyOn(service as any, 'saveMergedOrders').mockResolvedValue(undefined)
// @ts-ignore - accessing private method for testing // @ts-ignore - accessing private method for testing
const result = await service.mergeFiles( const result = await service.mergeFiles(

View File

@@ -173,6 +173,25 @@ describe('OrderNumberResolver', () => {
// Should be optimized to query unique values only // Should be optimized to query unique values only
expect(mockDbService.query).toHaveBeenCalledTimes(1) expect(mockDbService.query).toHaveBeenCalledTimes(1)
}) })
it('splits large mapping queries into bounded batches', async () => {
const largeInput = Array.from({ length: 1001 }, (_, i) => `22A${i}`)
vi.mocked(mockDbService.query).mockImplementation(async (_sql, params = []) => ({
rows: params.map((prodId, i) => ({
总排号: prodId,
: `SC7020260212${String(i).padStart(5, '0')}`
})),
columns: ['总排号', '生产订单号'],
rowCount: params.length
}))
await resolver.mapProductionIdsToOrderNumbers(largeInput)
expect(mockDbService.query).toHaveBeenCalledTimes(2)
expect(vi.mocked(mockDbService.query).mock.calls[0][1]).toHaveLength(1000)
expect(vi.mocked(mockDbService.query).mock.calls[1][1]).toHaveLength(1)
})
}) })
describe('error handling', () => { describe('error handling', () => {