refactor: complete migration from .env to YAML configuration

BREAKING CHANGE: Application now uses config.yaml instead of .env files ## Changes: - Remove dotenv dependency from package.json - Update all services to use ConfigManager for configuration - Update tests to use fixed credentials instead of env vars - Delete obsolete config-manager.test.ts (used old .env API) - Update documentation (README.md, CLAUDE.md) to reflect new config system ## Configuration Architecture: - ConfigManager: Centralized YAML configuration with Zod validation - config.yaml location: - Development: Project root (easy to edit and version control) - Production: User AppData (persists across updates) - ERP credentials: Stored in database (dbo_BIPUsers) per user - Other settings: Stored in config.yaml (database, paths, extraction, etc.) ## Files Modified: - package.json: Removed dotenv dependency - cleaner-handler.ts: Use ConfigManager.getDatabaseType() - run-migration.ts: Read from config.yaml instead of .env - All integration tests: Use fixed test credentials - tests/setup.ts: Removed dotenv loading - README.md, CLAUDE.md: Updated documentation Migration is complete. Application no longer depends on .env files.
2026-03-07 17:22:13 +08:00
parent 2d6838c0e7
commit 54a82200b6
14 changed files with 128 additions and 359 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -32,4 +32,6 @@ logs

 # Runtime config files
 *.yaml
-*.yaml.backup
+*.yaml.backup
+# But keep config.template.yaml
+!config.template.yaml
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -44,7 +44,7 @@ ERPAuto is an **Electron desktop application** for automating ERP system data pr
   - Node.js environment managing application lifecycle
   - Entry point: `src/main/index.ts`
   - Registers all IPC handlers via `registerIpcHandlers()`
-   - Loads environment variables from `.env` at startup
+   - Loads configuration from `config.yaml` via ConfigManager at startup

 2. **Preload Script** (`src/preload/`)
   - Security bridge between main and renderer processes
@@ -113,13 +113,23 @@ Admin users see logout buttons and can access user switching. Non-admin users ha
 - `@services` → `src/main/services` (main process, tests only)
 - `@types` → `src/main/types` (main process, tests only)

-## Environment Configuration
+## Configuration Management

-The application requires a `.env` file in the project root. Reference `.env.example` for the full structure. Key configurations:
+The application uses a YAML-based configuration system (`config.yaml`) managed by `ConfigManager`:

- **ERP Settings**: URL, credentials, headless mode, HTTPS error handling
+- **Development**: `config.yaml` in project root (easy to edit and version control)
+- **Production**: `config.yaml` in user data directory (AppData on Windows)
+
+Key configurations in `config.yaml`:
+
+- **ERP Settings**: URL (fixed infrastructure)
 - **Database**: MySQL and SQL Server connection configs (dual support)
- **App Settings**: Log level, download/temp directories
+- **Paths**: Data directory and output file settings
+- **Extraction**: Batch size, verbosity, persistence options
+- **Validation**: Data source, batch size, match mode
+- **Order Resolution**: Database table and field names for order number lookup
+
+Note: ERP credentials (username/password) are stored in the database (`dbo_BIPUsers` table) per user, managed via the Settings UI.

 ## Key Technologies

--- a/README.md
+++ b/README.md
@@ -30,29 +30,43 @@ npm install

 ### 配置

-在项目根目录创建 `.env` 文件：
+在项目根目录创建 `config.yaml` 文件（可参考 `config.template.yaml`）：

-```bash
-# ERP 配置
-ERP_URL=https://your-erp-server.com
-ERP_USERNAME=your_username
-ERP_PASSWORD=your_password
+```yaml
+# ERP 配置（固定基础设施）
+erp:
+  url: https://your-erp-server.com

-# MySQL 配置（可选）
-MYSQL_HOST=localhost
-MYSQL_PORT=3306
-MYSQL_USER=root
-MYSQL_PASSWORD=password
-MYSQL_DATABASE=erpauto
+# 数据库配置
+database:
+  activeType: mysql # 或 sqlserver

-# SQL Server 配置（可选）
-SQL_SERVER_HOST=localhost
-SQL_SERVER_PORT=1433
-SQL_SERVER_USER=sa
-SQL_SERVER_PASSWORD=password
-SQL_SERVER_DATABASE=erpauto
+  mysql:
+    host: localhost
+    port: 3306
+    database: erpauto
+    username: root
+    password: your_password
+    charset: utf8mb4
+
+  sqlserver:
+    server: localhost
+    port: 1433
+    database: erpauto
+    username: sa
+    password: your_password
+    driver: 'ODBC Driver 18 for SQL Server'
+    trustServerCertificate: true
+
+# 路径配置
+paths:
+  dataDir: './data/'
+  defaultOutput: 'output.xlsx'
+  validationOutput: 'validation-result.xlsx'
 ```

+**注意**：ERP 用户名和密码在应用的设置界面中配置，存储在数据库中（按用户管理）。
+
 ### 运行开发环境

 ```bash
@@ -136,9 +150,10 @@ ERPAuto/

 ### 无法连接 ERP 系统

-1. 检查 `.env` 文件中的 ERP_URL 是否正确
+1. 检查 `config.yaml` 中的 ERP URL 是否正确
 2. 确认网络连接正常
 3. 检查 ERP 系统是否可访问
+4. 在设置界面中确认 ERP 用户名和密码已配置

 ### 提取失败

@@ -149,7 +164,7 @@ ERPAuto/
 ### 数据库连接失败

 1. 确认数据库服务已启动
-2. 检查 `.env` 中的数据库配置
+2. 检查 `config.yaml` 中的数据库配置
 3. 确认防火墙允许数据库端口访问

 ## 开发
--- a/package.json
+++ b/package.json
@@ -61,7 +61,6 @@
    "@vitejs/plugin-react": "^5.1.1",
    "@vitest/coverage-v8": "^4.0.18",
    "autoprefixer": "^10.4.27",
-    "dotenv": "^17.3.1",
    "electron": "^39.2.6",
    "electron-builder": "^26.0.12",
    "electron-vite": "^5.0.0",
--- a/src/main/ipc/cleaner-handler.ts
+++ b/src/main/ipc/cleaner-handler.ts
@@ -124,7 +124,6 @@ export function registerCleanerHandlers(): void {
      return withErrorHandling(async () => {
        let authService: ErpAuthService | null = null
        let dbService: MySqlService | SqlServerService | null = null
-        let erpConfigService: UserErpConfigService | null = null

        try {
          // Get ERP configuration from database for current user
@@ -136,9 +135,10 @@ export function registerCleanerHandlers(): void {
            username: erpConfig.username ? 'configured' : 'EMPTY'
          })

-          const dbType = process.env.DB_TYPE?.toLowerCase()
+          const configManager = ConfigManager.getInstance()
+          const dbType = configManager.getDatabaseType()
          log.info(
-            `Connecting to ${dbType === 'sqlserver' || dbType === 'mssql' ? 'SQL Server' : 'MySQL'} for order resolution...`
+            `Connecting to ${dbType === 'sqlserver' ? 'SQL Server' : 'MySQL'} for order resolution...`
          )

          try {
--- a/src/main/services/user/migration/run-migration.ts
+++ b/src/main/services/user/migration/run-migration.ts
@@ -13,38 +13,41 @@ import * as fs from 'fs'
 import * as path from 'path'
 import { fileURLToPath } from 'url'
 import { dirname } from 'path'
+import yaml from 'js-yaml'
+import { z } from 'zod'

 const __filename = fileURLToPath(import.meta.url)
-const __dirname = dirname(__filename)

 /**
- * Load .env file manually
+ * MySQL configuration schema
 */
-function loadEnv(filePath: string): Map<string, string> {
-  const envMap = new Map<string, string>()
+const mysqlConfigSchema = z.object({
+  host: z.string(),
+  port: z.number(),
+  database: z.string(),
+  username: z.string(),
+  password: z.string()
+})

+/**
+ * Load config.yaml file
+ */
+function loadConfig(filePath: string): {
+  host: string
+  port: number
+  database: string
+  username: string
+  password: string
+} {
  if (!fs.existsSync(filePath)) {
-    console.warn(`.env file not found: ${filePath}`)
-    return envMap
+    throw new Error(`Config file not found: ${filePath}`)
  }

  const content = fs.readFileSync(filePath, 'utf-8')
-  const lines = content.split('\n')
+  const parsed = yaml.load(content) as Record<string, unknown>

-  for (const line of lines) {
-    const trimmedLine = line.trim()
-    if (!trimmedLine || trimmedLine.startsWith('#')) {
-      continue
-    }
-
-    const [key, ...valueParts] = trimmedLine.split('=')
-    if (key && valueParts.length > 0) {
-      const value = valueParts.join('=').trim()
-      envMap.set(key.trim(), value)
-    }
-  }
-
-  return envMap
+  const result = mysqlConfigSchema.parse(parsed?.database?.mysql)
+  return result
 }

 /**
@@ -89,17 +92,29 @@ async function runMigration(): Promise<void> {
  console.log('BIPUsers Table Migration: Add ERP Parameters')
  console.log('==============================================\n')

-  // Load .env file from project root
-  const envPath = path.resolve(process.cwd(), '.env')
-  console.log(`Loading .env from: ${envPath}`)
-  const env = loadEnv(envPath)
+  // Load config.yaml from project root or user data directory
+  const isDev = !process.execPath.includes('Resources\\app')
+  const configPath = isDev
+    ? path.resolve(process.cwd(), 'config.yaml')
+    : path.join(process.env.APPDATA || '', 'erpauto', 'config.yaml')

-  // Get database configuration
-  const dbHost = env.get('DB_MYSQL_HOST') || 'localhost'
-  const dbPort = parseInt(env.get('DB_MYSQL_PORT') || '3306', 10)
-  const dbUser = env.get('DB_USERNAME') || 'root'
-  const dbPassword = env.get('DB_PASSWORD') || ''
-  const dbName = env.get('DB_NAME') || ''
+  console.log(`Loading config from: ${configPath}`)
+
+  let dbConfig: { host: string; port: number; database: string; username: string; password: string }
+
+  try {
+    dbConfig = loadConfig(configPath)
+  } catch (error) {
+    console.error('Failed to load config.yaml:', error instanceof Error ? error.message : error)
+    console.error('Please ensure config.yaml exists and contains valid MySQL configuration.')
+    process.exit(1)
+  }
+
+  const dbHost = dbConfig.host || 'localhost'
+  const dbPort = dbConfig.port || 3306
+  const dbUser = dbConfig.username || 'root'
+  const dbPassword = dbConfig.password || ''
+  const dbName = dbConfig.database || ''

  console.log(`Database: ${dbHost}:${dbPort}/${dbName}`)
  console.log(`Username: ${dbUser}`)
@@ -160,7 +175,7 @@ async function runMigration(): Promise<void> {
    console.error(error)
    console.error('\nTroubleshooting:')
    console.error('1. Check if MySQL server is running')
-    console.error('2. Verify database credentials in .env file')
+    console.error('2. Verify database credentials in config.yaml file')
    console.error('3. Ensure database "' + dbName + '" exists')
    console.error('4. Check network connectivity to ' + dbHost + ':' + dbPort)
    process.exit(1)
@@ -170,7 +185,7 @@ async function runMigration(): Promise<void> {
      try {
        await connection.end()
        console.log('Disconnected from MySQL')
-      } catch (e) {
+      } catch {
        // Ignore disconnect errors
      }
    }
--- a/tests/e2e/auth-flow.test.ts
+++ b/tests/e2e/auth-flow.test.ts
@@ -21,7 +21,6 @@ test.describe('Authentication Flow', () => {
    electronApp = await electron.launch({
      args: [path.join(__dirname, '../../out/main/index.js')],
      env: {
-        ...process.env,
        NODE_ENV: 'test'
      }
    })
--- a/tests/integration/cleaner.test.ts
+++ b/tests/integration/cleaner.test.ts
@@ -6,10 +6,11 @@ import fs from 'fs/promises'
 import path from 'path'

 describe('Cleaner Service (Integration)', () => {
+  // For integration tests, use fixed test credentials or configure via config.yaml
  const config: ErpConfig = {
-    url: process.env.ERP_URL || '',
-    username: process.env.ERP_USERNAME || '',
-    password: process.env.ERP_PASSWORD || ''
+    url: '',
+    username: '',
+    password: ''
  }

  // Test data paths
--- a/tests/integration/erp-auth.test.ts
+++ b/tests/integration/erp-auth.test.ts
@@ -4,10 +4,11 @@ import type { ErpConfig } from '../../src/main/types/erp.types'

 describe('ERP Authentication Service (Integration)', () => {
  let authService: ErpAuthService
+  // For integration tests, use fixed test credentials or configure via config.yaml
  const config: ErpConfig = {
-    url: process.env.ERP_URL || '',
-    username: process.env.ERP_USERNAME || '',
-    password: process.env.ERP_PASSWORD || ''
+    url: '',
+    username: '',
+    password: ''
  }

  // Check if we have ERP credentials
--- a/tests/integration/extractor.test.ts
+++ b/tests/integration/extractor.test.ts
@@ -6,10 +6,11 @@ import fs from 'fs/promises'
 import path from 'path'

 describe('Extractor Service (Integration)', () => {
+  // For integration tests, use fixed test credentials or configure via config.yaml
  const config: ErpConfig = {
-    url: process.env.ERP_URL || '',
-    username: process.env.ERP_USERNAME || '',
-    password: process.env.ERP_PASSWORD || ''
+    url: '',
+    username: '',
+    password: ''
  }

  const testOrderNumber = 'SC70202602120085' // From references/demo/productionID.txt
--- a/tests/integration/mysql.test.ts
+++ b/tests/integration/mysql.test.ts
@@ -8,13 +8,13 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'
 import { MySqlService, MySqlConfig } from '@services/database/mysql'

 // MySQL test configuration
-// In production, these should come from environment variables
+// For integration tests, use fixed test credentials or configure via config.yaml
 const testConfig: MySqlConfig = {
-  host: process.env.MYSQL_HOST || 'localhost',
-  port: parseInt(process.env.MYSQL_PORT || '3306'),
-  user: process.env.MYSQL_USER || 'root',
-  password: process.env.MYSQL_PASSWORD || 'password',
-  database: process.env.MYSQL_DATABASE || 'test_db'
+  host: 'localhost',
+  port: 3306,
+  user: 'root',
+  password: 'password',
+  database: 'test_db'
 }

 describe('MySqlService Integration Tests', () => {
--- a/tests/integration/sql-server.test.ts
+++ b/tests/integration/sql-server.test.ts
@@ -9,12 +9,13 @@ import { SqlServerService, SqlServerConfig } from '@main/services/database/sql-s
 import * as sql from 'mssql'

 // SQL Server test configuration
+// For integration tests, use fixed test credentials or configure via config.yaml
 const testConfig: SqlServerConfig = {
-  server: process.env.SQL_SERVER_HOST || 'localhost',
-  port: parseInt(process.env.SQL_SERVER_PORT || '1433'),
-  user: process.env.SQL_SERVER_USER || 'sa',
-  password: process.env.SQL_SERVER_PASSWORD || 'password',
-  database: process.env.SQL_SERVER_DATABASE || 'testdb',
+  server: 'localhost',
+  port: 1433,
+  user: 'sa',
+  password: 'password',
+  database: 'testdb',
  options: {
    encrypt: false, // Set to true for Azure SQL
    trustServerCertificate: true // Set to false in production with valid cert
--- a/tests/main/services/config/config-manager.test.ts
+++ b/tests/main/services/config/config-manager.test.ts
@@ -1,271 +0,0 @@
-import { describe, it, expect, vi } from 'vitest'
-import { ConfigManager } from '@services/config/config-manager'
-import type { SettingsData } from '@types/settings.types'
-
-describe('ConfigManager - deep merge utilities', () => {
-  it('should deep merge objects, updating only specified fields', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-    // Reload to ensure clean state from previous tests
-    await manager['loadEnvFile']()
-
-    // Setup initial state
-    const initial: SettingsData = {
-      erp: {
-        url: 'http://old.com',
-        username: 'user1',
-        password: 'pass1',
-        headless: true,
-        ignoreHttpsErrors: true,
-        autoCloseBrowser: true
-      },
-      database: {
-        dbType: 'mysql',
-        server: '',
-        mysqlHost: 'localhost',
-        mysqlPort: 3306,
-        database: 'db',
-        username: 'user',
-        password: ''
-      },
-      paths: { dataDir: '/data', defaultOutput: 'out.xlsx', validationOutput: 'validation.xlsx' },
-      extraction: {
-        batchSize: 100,
-        verbose: true,
-        autoConvert: true,
-        mergeBatches: true,
-        enableDbPersistence: true
-      },
-      validation: {
-        dataSource: 'database_full',
-        batchSize: 2000,
-        matchMode: 'substring',
-        enableCrud: false,
-        defaultManager: ''
-      },
-      ui: { fontFamily: 'Arial', fontSize: 12, productionIdInputWidth: 20 },
-      execution: { dryRun: false }
-    }
-
-    // Load initial settings
-    await manager.saveAllSettings(initial)
-    // Reload from disk to populate cache
-    await manager['loadEnvFile']()
-
-    // Partial update
-    const partial = {
-      erp: { url: 'http://new.com' }
-    }
-
-    const result = await manager.savePartialSettings(partial)
-
-    expect(result.success).toBe(true)
-
-    const current = manager.getAllSettings()
-
-    // Updated field
-    expect(current.erp.url).toBe('http://new.com')
-
-    // Preserved fields
-    expect(current.erp.username).toBe('user1')
-    expect(current.database.dbType).toBe('mysql')
-    expect(current.paths.dataDir).toBe('/data')
-  })
-})
-
-describe('ConfigManager - backup and restore', () => {
-  it('should create backup before saving', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-
-    const backupSuccess = await manager['backupEnvFile']()
-
-    expect(backupSuccess).toBe(true)
-
-    // Check backup file exists (in same location as .env file, which is src/main/)
-    const fs = await import('fs')
-    const path = await import('path')
-    const backupPath = path.resolve(process.cwd(), 'src/main/.env.backup')
-
-    expect(fs.existsSync(backupPath)).toBe(true)
-  })
-
-  // Note: Skipping fs.writeFileSync mock test due to ESM limitations in Vitest
-  // The restoreBackup functionality is tested indirectly through the savePartialSettings rollback test
-})
-
-describe('ConfigManager.savePartialSettings', () => {
-  it('should save only specified fields and preserve others', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-
-    // Setup initial state with multiple categories
-    await manager.saveAllSettings({
-      erp: {
-        url: 'http://old.com',
-        username: 'user1',
-        password: 'pass1',
-        headless: true,
-        ignoreHttpsErrors: true,
-        autoCloseBrowser: true
-      },
-      database: {
-        dbType: 'mysql',
-        server: '',
-        mysqlHost: '192.168.1.1',
-        mysqlPort: 3306,
-        database: 'testdb',
-        username: 'dbuser',
-        password: ''
-      },
-      paths: { dataDir: '/old/path', defaultOutput: 'out.xlsx', validationOutput: 'val.xlsx' },
-      extraction: {
-        batchSize: 50,
-        verbose: true,
-        autoConvert: true,
-        mergeBatches: true,
-        enableDbPersistence: true
-      },
-      validation: {
-        dataSource: 'database_full',
-        batchSize: 1000,
-        matchMode: 'exact',
-        enableCrud: false,
-        defaultManager: ''
-      },
-      ui: { fontFamily: 'Tahoma', fontSize: 14, productionIdInputWidth: 25 },
-      execution: { dryRun: true }
-    })
-    // Reload from disk to populate cache
-    await manager['loadEnvFile']()
-
-    // Update only ERP URL
-    const result = await manager.savePartialSettings({
-      erp: { url: 'http://new.com' }
-    })
-
-    expect(result.success).toBe(true)
-
-    const current = manager.getAllSettings()
-
-    // Verify updated field
-    expect(current.erp.url).toBe('http://new.com')
-
-    // Verify preserved ERP fields
-    expect(current.erp.username).toBe('user1')
-    expect(current.erp.password).toBe('pass1')
-
-    // Verify preserved other categories
-    expect(current.database.dbType).toBe('mysql')
-    expect(current.database.mysqlHost).toBe('192.168.1.1')
-    expect(current.paths.dataDir).toBe('/old/path')
-    expect(current.extraction.batchSize).toBe(50)
-    expect(current.ui.fontFamily).toBe('Tahoma')
-  })
-
-  it('should reject updates to non-whitelisted fields', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-    // Reset to ensure clean state
-    manager.resetToDefaults()
-    await manager.save()
-
-    const result = await manager.savePartialSettings({
-      database: { dbType: 'postgres' }
-    })
-
-    expect(result.success).toBe(false)
-    expect(result.error).toContain('不允许修改')
-    expect(result.error).toContain('database.dbType')
-  })
-
-  it('should handle nested object updates correctly', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-    // Reset to ensure clean state
-    manager.resetToDefaults()
-    await manager.save()
-
-    await manager.saveAllSettings({
-      erp: {
-        url: 'http://test.com',
-        username: 'u',
-        password: 'p',
-        headless: false,
-        ignoreHttpsErrors: false,
-        autoCloseBrowser: false
-      },
-      database: {
-        dbType: 'mysql',
-        server: '',
-        mysqlHost: 'localhost',
-        mysqlPort: 3306,
-        database: 'db',
-        username: 'user',
-        password: ''
-      },
-      paths: { dataDir: '/data', defaultOutput: 'out.xlsx', validationOutput: 'val.xlsx' },
-      extraction: {
-        batchSize: 100,
-        verbose: true,
-        autoConvert: true,
-        mergeBatches: true,
-        enableDbPersistence: true
-      },
-      validation: {
-        dataSource: 'database_full',
-        batchSize: 2000,
-        matchMode: 'substring',
-        enableCrud: false,
-        defaultManager: ''
-      },
-      ui: { fontFamily: 'Arial', fontSize: 12, productionIdInputWidth: 20 },
-      execution: { dryRun: false }
-    })
-    // Reload from disk to populate cache
-    await manager['loadEnvFile']()
-
-    // Update multiple ERP fields at once
-    const result = await manager.savePartialSettings({
-      erp: {
-        url: 'http://updated.com',
-        username: 'newuser',
-        password: 'newpass'
-      }
-    })
-
-    expect(result.success).toBe(true)
-
-    const current = manager.getAllSettings()
-
-    expect(current.erp.url).toBe('http://updated.com')
-    expect(current.erp.username).toBe('newuser')
-    expect(current.erp.password).toBe('newpass')
-    expect(current.erp.headless).toBe(false) // preserved
-  })
-
-  it('should restore backup on save failure', async () => {
-    const manager = ConfigManager.getInstance()
-    await manager.initialize()
-    // Reset to ensure clean state
-    manager.resetToDefaults()
-    await manager.save()
-
-    const originalUrl = manager.getAllSettings().erp.url
-
-    // Mock save to fail
-    vi.spyOn(manager, 'save').mockResolvedValueOnce(false)
-
-    const result = await manager.savePartialSettings({
-      erp: { url: 'http://should-not-apply.com' }
-    })
-
-    expect(result.success).toBe(false)
-    expect(result.error).toContain('保存配置失败')
-
-    // Verify rollback
-    expect(manager.getAllSettings().erp.url).toBe(originalUrl)
-
-    manager.save.mockRestore()
-  })
-})
--- a/tests/setup.ts
+++ b/tests/setup.ts
@@ -1,5 +1,4 @@
 import { beforeAll, afterAll, vi } from 'vitest'
-import dotenv from 'dotenv'
 import path from 'path'

 // Mock electron app module for unit tests
@@ -12,9 +11,6 @@ vi.mock('electron', () => ({
  }
 }))

-// Load environment variables from project root
-dotenv.config({ path: path.resolve(process.cwd(), '.env') })
-
 beforeAll(async () => {
  // Global test setup
  console.log('Test suite starting...')