Add complete documentation for the configuration system including: - Architecture design with Mermaid diagrams (5 diagrams) - Detailed explanation of all 6 configuration modules - Complete environment variable reference table - Configuration loading and validation flow - Usage examples and migration guide - Troubleshooting section Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1180 lines
30 KiB
Markdown
1180 lines
30 KiB
Markdown
# 配置管理文档
|
||
|
||
## 目录
|
||
|
||
- [概述](#概述)
|
||
- [架构设计](#架构设计)
|
||
- [配置模块详解](#配置模块详解)
|
||
- [环境变量配置](#环境变量配置)
|
||
- [配置加载流程](#配置加载流程)
|
||
- [配置验证](#配置验证)
|
||
- [使用示例](#使用示例)
|
||
- [迁移指南](#迁移指南)
|
||
- [故障排除](#故障排除)
|
||
- [附录](#附录)
|
||
|
||
---
|
||
|
||
## 概述
|
||
|
||
### 设计理念
|
||
|
||
本项目采用 **12-Factor App** 配置管理原则,通过环境变量管理所有配置项。这种方式具有以下优势:
|
||
|
||
- **环境隔离**: 开发、测试、生产环境使用不同的 `.env` 文件
|
||
- **安全性**: 敏感信息(密码、密钥)不会被提交到版本控制
|
||
- **灵活性**: 无需修改代码即可调整配置
|
||
- **可移植性**: 配置与代码分离,便于部署和迁移
|
||
|
||
### 演进历史
|
||
|
||
**V1.0 - JSON 配置文件**
|
||
- 配置存储在 `config/user_settings.json`
|
||
- 需要手动编辑 JSON 文件
|
||
- 容易误提交敏感信息到代码库
|
||
|
||
**V2.0 - 环境变量(当前)**
|
||
- 配置通过 `.env` 文件管理
|
||
- 自动加载机制,导入即生效
|
||
- 支持多数据库、多环境配置
|
||
|
||
### 主要特性
|
||
|
||
- **自动加载**: `env_loader.py` 导入时自动加载 `.env` 文件
|
||
- **类型安全**: 使用 Python `dataclass` 定义配置结构
|
||
- **配置验证**: 每个配置模块都有独立的验证方法
|
||
- **向后兼容**: 仍支持从 JSON 文件加载配置
|
||
- **多数据库支持**: 同时支持 SQL Server 和 MySQL
|
||
- **热更新**: 支持运行时更新配置并保存
|
||
|
||
---
|
||
|
||
## 架构设计
|
||
|
||
### 分层架构
|
||
|
||
```
|
||
┌─────────────────────────────────────────┐
|
||
│ Application Layer │
|
||
│ (Playwright Scripts, GUI, etc.) │
|
||
└─────────────────┬───────────────────────┘
|
||
│
|
||
┌─────────────────▼───────────────────────┐
|
||
│ ConfigLoader │
|
||
│ - load() │
|
||
│ - save() │
|
||
│ - save_to_env() │
|
||
└─────────────────┬───────────────────────┘
|
||
│
|
||
┌─────────────────▼───────────────────────┐
|
||
│ AppConfig │
|
||
│ ┌───────────────────────────────────┐ │
|
||
│ │ ERPConfig │ │
|
||
│ │ DatabaseConfig │ │
|
||
│ │ PathConfig │ │
|
||
│ │ ExtractionConfig │ │
|
||
│ │ ValidationConfig │ │
|
||
│ └───────────────────────────────────┘ │
|
||
└─────────────────┬───────────────────────┘
|
||
│
|
||
┌─────────────────▼───────────────────────┐
|
||
│ EnvLoader │
|
||
│ - load_env_file() │
|
||
│ - get_env() / get_env_bool() │
|
||
│ - save_env_file() │
|
||
└─────────────────┬───────────────────────┘
|
||
│
|
||
┌─────────────────▼───────────────────────┐
|
||
│ .env File │
|
||
│ (Environment Variables) │
|
||
└─────────────────────────────────────────┘
|
||
```
|
||
|
||
### 配置类层次结构
|
||
|
||
```mermaid
|
||
classDiagram
|
||
class AppConfig {
|
||
+erp: ERPConfig
|
||
+database: DatabaseConfig
|
||
+paths: PathConfig
|
||
+extraction: ExtractionConfig
|
||
+validation: ValidationConfig
|
||
+from_env() AppConfig
|
||
+validate() list[str]
|
||
+to_dict() dict
|
||
}
|
||
|
||
class ERPConfig {
|
||
+url: str
|
||
+username: str
|
||
+password: str
|
||
+headless: bool
|
||
+ignore_https_errors: bool
|
||
+auto_close_browser: bool
|
||
+from_env() ERPConfig
|
||
+validate() list[str]
|
||
}
|
||
|
||
class DatabaseConfig {
|
||
+db_type: DatabaseType
|
||
+server: str
|
||
+database: str
|
||
+username: str
|
||
+password: str
|
||
+sqlserver: SQLServerConfig
|
||
+mysql: MySQLConfig
|
||
+from_env() DatabaseConfig
|
||
+validate() list[str]
|
||
}
|
||
|
||
class PathConfig {
|
||
+data_dir: str
|
||
+production_id_file: str
|
||
+default_output: str
|
||
+validation_output: str
|
||
+from_env() PathConfig
|
||
+validate() list[str]
|
||
}
|
||
|
||
class ExtractionConfig {
|
||
+batch_size: int
|
||
+verbose: bool
|
||
+auto_convert: bool
|
||
+merge_batches: bool
|
||
+enable_db_persistence: bool
|
||
+from_env() ExtractionConfig
|
||
+validate() list[str]
|
||
}
|
||
|
||
class ValidationConfig {
|
||
+data_source: str
|
||
+use_database: bool
|
||
+batch_size: int
|
||
+enable_crud_operations: bool
|
||
+default_manager: str
|
||
+match_mode: str
|
||
+from_env() ValidationConfig
|
||
+validate() list[str]
|
||
}
|
||
|
||
class ConfigLoader {
|
||
+load(use_env) AppConfig
|
||
+load_from_json(file) AppConfig
|
||
+save(config, file) bool
|
||
+save_to_env(config, file) bool
|
||
}
|
||
|
||
class EnvLoader {
|
||
+load_env_file() void
|
||
+get_env(key, default) str
|
||
+get_env_bool(key, default) bool
|
||
+get_env_int(key, default) int
|
||
+save_env_file(file, env_dict) bool
|
||
+update_env_file(file, **kwargs) bool
|
||
}
|
||
|
||
AppConfig *-- ERPConfig
|
||
AppConfig *-- DatabaseConfig
|
||
AppConfig *-- PathConfig
|
||
AppConfig *-- ExtractionConfig
|
||
AppConfig *-- ValidationConfig
|
||
ConfigLoader ..> AppConfig : loads
|
||
EnvLoader ..> AppConfig : supports
|
||
```
|
||
|
||
### 模块职责
|
||
|
||
| 模块 | 文件 | 职责 |
|
||
|------|------|------|
|
||
| **schema.py** | `config/schema.py` | 定义所有配置类的数据结构 |
|
||
| **env_loader.py** | `config/env_loader.py` | 环境变量加载和类型转换 |
|
||
| **loader.py** | `config/loader.py` | 主配置加载器,协调各模块 |
|
||
| **defaults.py** | `config/defaults.py` | 默认配置值定义 |
|
||
| **user_settings.py** | `config/user_settings.py` | 向后兼容层(可选) |
|
||
|
||
### 设计模式
|
||
|
||
1. **Builder Pattern**: `AppConfig.from_env()` 逐步构建配置对象
|
||
2. **Factory Pattern**: `ConfigLoader` 根据参数创建不同来源的配置
|
||
3. **Singleton Pattern**: 配置在应用启动时加载一次,全局共享
|
||
4. **Validator Pattern**: 每个配置类都有独立的 `validate()` 方法
|
||
|
||
---
|
||
|
||
## 配置模块详解
|
||
|
||
### AppConfig - 主配置容器
|
||
|
||
**文件**: `config/schema.py:273-351`
|
||
|
||
```python
|
||
@dataclass
|
||
class AppConfig:
|
||
"""应用总配置"""
|
||
erp: ERPConfig
|
||
database: DatabaseConfig
|
||
paths: PathConfig
|
||
extraction: ExtractionConfig
|
||
validation: ValidationConfig
|
||
```
|
||
|
||
**方法**:
|
||
- `from_env()`: 从环境变量创建配置
|
||
- `validate()`: 验证所有子配置
|
||
- `to_dict()`: 转换为字典(用于保存到 JSON)
|
||
|
||
---
|
||
|
||
### ERPConfig - ERP 系统配置
|
||
|
||
**文件**: `config/schema.py:21-54`
|
||
|
||
```python
|
||
@dataclass
|
||
class ERPConfig:
|
||
url: str # ERP 系统地址
|
||
username: str # 登录用户名
|
||
password: str # 登录密码
|
||
headless: bool = True # 无头模式
|
||
ignore_https_errors: bool = True # 忽略 HTTPS 错误
|
||
auto_close_browser: bool = True # 自动关闭浏览器
|
||
```
|
||
|
||
**验证规则**:
|
||
- URL 不能为空
|
||
- 用户名不能为空
|
||
- 密码不能为空
|
||
|
||
**默认值**:
|
||
```python
|
||
url: "https://68.11.34.30:8082/"
|
||
username: "BLDpengqiangqiang"
|
||
headless: True
|
||
```
|
||
|
||
---
|
||
|
||
### DatabaseConfig - 数据库配置
|
||
|
||
**文件**: `config/schema.py:94-149`
|
||
|
||
```python
|
||
@dataclass
|
||
class DatabaseConfig:
|
||
db_type: DatabaseType = DatabaseType.SQLSERVER
|
||
server: str = "" # SQL Server 服务器
|
||
database: str = "" # 数据库名
|
||
username: str = "" # 用户名
|
||
password: str = "" # 密码
|
||
sqlserver: Optional[SQLServerConfig] = None # SQL Server 特定配置
|
||
mysql: Optional[MySQLConfig] = None # MySQL 特定配置
|
||
```
|
||
|
||
**支持的数据库类型**:
|
||
- `sqlserver`: SQL Server (默认)
|
||
- `mysql`: MySQL
|
||
|
||
**SQL Server 特定配置**:
|
||
```python
|
||
@dataclass
|
||
class SQLServerConfig:
|
||
driver: str = "ODBC Driver 18 for SQL Server"
|
||
trust_server_certificate: str = "yes"
|
||
```
|
||
|
||
**MySQL 特定配置**:
|
||
```python
|
||
@dataclass
|
||
class MySQLConfig:
|
||
host: str = ""
|
||
port: int = 3306
|
||
charset: str = "utf8mb4"
|
||
```
|
||
|
||
**验证规则**:
|
||
- SQL Server: 服务器、数据库、用户名、密码都不能为空
|
||
- MySQL: 主机、数据库、用户名、密码都不能为空
|
||
|
||
**默认值**:
|
||
```python
|
||
# SQL Server
|
||
server: "192.168.110.114"
|
||
database: "CompanyDB"
|
||
username: "peng"
|
||
|
||
# MySQL
|
||
host: "192.168.31.83"
|
||
database: "BLD_DB"
|
||
username: "remote_user"
|
||
```
|
||
|
||
---
|
||
|
||
### PathConfig - 文件路径配置
|
||
|
||
**文件**: `config/schema.py:153-180`
|
||
|
||
```python
|
||
@dataclass
|
||
class PathConfig:
|
||
data_dir: str # 数据目录
|
||
production_id_file: str # 生产 ID 文件
|
||
default_output: str = "离散备料计划维护_合并.xlsx"
|
||
validation_output: str = "物料状态校验结果.xlsx"
|
||
```
|
||
|
||
**默认值**:
|
||
```python
|
||
data_dir: "D:/python/playwrite/data/"
|
||
production_id_file: "ProductionID.txt"
|
||
```
|
||
|
||
---
|
||
|
||
### ExtractionConfig - 数据提取配置
|
||
|
||
**文件**: `config/schema.py:184-213`
|
||
|
||
```python
|
||
@dataclass
|
||
class ExtractionConfig:
|
||
batch_size: int = 100 # 批处理大小
|
||
verbose: bool = True # 详细输出
|
||
auto_convert: bool = True # 自动转换
|
||
merge_batches: bool = True # 合并批次
|
||
enable_db_persistence: bool = False # 启用数据库持久化
|
||
```
|
||
|
||
**验证规则**:
|
||
- `batch_size` 必须 > 0
|
||
- `batch_size` 不应超过 1000
|
||
|
||
---
|
||
|
||
### ValidationConfig - 物料校验配置
|
||
|
||
**文件**: `config/schema.py:217-269`
|
||
|
||
```python
|
||
@dataclass
|
||
class ValidationConfig:
|
||
data_source: str = "database_full" # 数据源
|
||
use_database: bool = True # 使用数据库
|
||
batch_size: int = 2000 # 批处理大小
|
||
enable_crud_operations: bool = False # 启用 CRUD 操作
|
||
default_manager: str = "" # 默认管理员
|
||
match_mode: str = "substring" # 匹配模式
|
||
```
|
||
|
||
**数据源选项**:
|
||
- `database_full`: 完整数据库查询
|
||
- `database_filtered`: 过滤后的数据库查询
|
||
- `excel_existing`: 现有 Excel 文件
|
||
- `excel_full`: 完整 Excel 数据
|
||
|
||
**匹配模式**:
|
||
- `substring`: 子字符串匹配
|
||
- `exact`: 精确匹配
|
||
|
||
**验证规则**:
|
||
- `data_source` 必须在有效选项中
|
||
- `batch_size` 必须 > 0 且 ≤ 2000(SQL Server 参数限制)
|
||
- `match_mode` 必须是 "substring" 或 "exact"
|
||
|
||
---
|
||
|
||
## 环境变量配置
|
||
|
||
### .env 文件结构
|
||
|
||
`.env` 文件位于项目根目录,使用 `KEY=VALUE` 格式:
|
||
|
||
```bash
|
||
# ===========================
|
||
# ERP 系统配置
|
||
# ===========================
|
||
ERP_URL=https://68.11.34.30:8082/
|
||
ERP_USERNAME=BLDpengqiangqiang
|
||
ERP_PASSWORD=your_password_here
|
||
ERP_HEADLESS=true
|
||
ERP_IGNORE_HTTPS_ERRORS=true
|
||
ERP_AUTO_CLOSE_BROWSER=true
|
||
|
||
# ===========================
|
||
# 数据库配置 - SQL Server
|
||
# ===========================
|
||
DB_TYPE=sqlserver
|
||
DB_SERVER=192.168.110.114
|
||
DB_NAME=CompanyDB
|
||
DB_USERNAME=peng
|
||
DB_PASSWORD=your_password_here
|
||
DB_SQLSERVER_DRIVER=ODBC Driver 18 for SQL Server
|
||
DB_TRUST_SERVER_CERTIFICATE=yes
|
||
|
||
# ===========================
|
||
# 数据库配置 - MySQL
|
||
# ===========================
|
||
# DB_TYPE=mysql
|
||
# DB_NAME=BLD_DB
|
||
# DB_USERNAME=remote_user
|
||
# DB_PASSWORD=your_password_here
|
||
# DB_MYSQL_HOST=192.168.31.83
|
||
# DB_MYSQL_PORT=3306
|
||
# DB_MYSQL_CHARSET=utf8mb4
|
||
```
|
||
|
||
### 完整环境变量列表
|
||
|
||
| 分类 | 变量名 | 类型 | 默认值 | 说明 |
|
||
|------|--------|------|--------|------|
|
||
| **ERP** | `ERP_URL` | string | `https://68.11.34.30:8082/` | ERP 系统地址 |
|
||
| | `ERP_USERNAME` | string | `BLDpengqiangqiang` | 登录用户名 |
|
||
| | `ERP_PASSWORD` | string | *必填* | 登录密码 |
|
||
| | `ERP_HEADLESS` | bool | `true` | 无头模式 |
|
||
| | `ERP_IGNORE_HTTPS_ERRORS` | bool | `true` | 忽略 HTTPS 错误 |
|
||
| | `ERP_AUTO_CLOSE_BROWSER` | bool | `true` | 自动关闭浏览器 |
|
||
| **数据库** | `DB_TYPE` | enum | `sqlserver` | 数据库类型 (sqlserver/mysql) |
|
||
| | `DB_SERVER` | string | `192.168.110.114` | SQL Server 服务器地址 |
|
||
| | `DB_NAME` | string | `CompanyDB` | 数据库名称 |
|
||
| | `DB_USERNAME` | string | `peng` | 数据库用户名 |
|
||
| | `DB_PASSWORD` | string | *必填* | 数据库密码 |
|
||
| **SQL Server** | `DB_SQLSERVER_DRIVER` | string | `ODBC Driver 18 for SQL Server` | ODBC 驱动名称 |
|
||
| | `DB_TRUST_SERVER_CERTIFICATE` | string | `yes` | 信任服务器证书 |
|
||
| **MySQL** | `DB_MYSQL_HOST` | string | `192.168.31.83` | MySQL 主机地址 |
|
||
| | `DB_MYSQL_PORT` | int | `3306` | MySQL 端口 |
|
||
| | `DB_MYSQL_CHARSET` | string | `utf8mb4` | 字符集 |
|
||
| **路径** | `PATH_DATA_DIR` | string | `D:/python/playwrite/data/` | 数据目录 |
|
||
| | `PATH_PRODUCTION_ID_FILE` | string | `ProductionID.txt` | 生产 ID 文件名 |
|
||
| | `PATH_DEFAULT_OUTPUT` | string | `离散备料计划维护_合并.xlsx` | 默认输出文件名 |
|
||
| | `PATH_VALIDATION_OUTPUT` | string | `物料状态校验结果.xlsx` | 校验输出文件名 |
|
||
| **提取** | `EXTRACTION_BATCH_SIZE` | int | `100` | 批处理大小 |
|
||
| | `EXTRACTION_VERBOSE` | bool | `true` | 详细输出 |
|
||
| | `EXTRACTION_AUTO_CONVERT` | bool | `true` | 自动转换 |
|
||
| | `EXTRACTION_MERGE_BATCHES` | bool | `true` | 合并批次 |
|
||
| | `EXTRACTION_ENABLE_DB_PERSISTENCE` | bool | `false` | 数据库持久化 |
|
||
| **校验** | `VALIDATION_DATA_SOURCE` | string | `database_full` | 数据源 |
|
||
| | `VALIDATION_USE_DATABASE` | bool | `true` | 使用数据库 |
|
||
| | `VALIDATION_BATCH_SIZE` | int | `2000` | 批处理大小 |
|
||
| | `VALIDATION_ENABLE_CRUD` | bool | `false` | 启用 CRUD 操作 |
|
||
| | `VALIDATION_DEFAULT_MANAGER` | string | `""` | 默认管理员 |
|
||
| | `VALIDATION_MATCH_MODE` | string | `substring` | 匹配模式 |
|
||
|
||
### 布尔值格式
|
||
|
||
支持以下布尔值格式(不区分大小写):
|
||
|
||
| 真 | 假 |
|
||
|----|-----|
|
||
| `true` | `false` |
|
||
| `1` | `0` |
|
||
| `yes` | `no` |
|
||
| `on` | `off` |
|
||
|
||
---
|
||
|
||
## 配置加载流程
|
||
|
||
### 自动加载机制
|
||
|
||
`env_loader.py` 在导入时自动执行:
|
||
|
||
```python
|
||
# env_loader.py:201
|
||
load_env_file() # 模块导入时自动调用
|
||
```
|
||
|
||
这意味着只需导入配置模块,`.env` 文件就会自动加载:
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
# .env 文件已自动加载
|
||
config = ConfigLoader.load()
|
||
```
|
||
|
||
### 配置加载流程图
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A[Application Start] --> B[env_loader.py imported]
|
||
B --> C[Auto-load .env file]
|
||
C --> D[ConfigLoader.load called]
|
||
D --> E{use_env parameter?}
|
||
E -->|True| F[Load from environment variables]
|
||
E -->|False| G[Load from JSON file]
|
||
F --> H[AppConfig.from_env]
|
||
G --> I[Parse JSON and create AppConfig]
|
||
H --> J[Create config objects]
|
||
I --> J
|
||
J --> K[Validate configuration]
|
||
K --> L{Validation passed?}
|
||
L -->|Yes| M[Return AppConfig]
|
||
L -->|No| N[Return errors list]
|
||
```
|
||
|
||
### 配置优先级
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
A[Default Values] --> B[JSON Config File]
|
||
B --> C[Environment Variables]
|
||
C --> D[Runtime Override]
|
||
D --> E[Final Config]
|
||
|
||
style A fill:#e1f5fe
|
||
style B fill:#fff9c4
|
||
style C fill:#c8e6c9
|
||
style D fill:#ffccbc
|
||
style E fill:#f3e5f5
|
||
```
|
||
|
||
**优先级说明**:
|
||
1. **环境变量** (最高优先级)
|
||
2. **JSON 配置文件**
|
||
3. **代码中的默认值** (最低优先级)
|
||
|
||
### 数据库配置选择流程
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A[DatabaseConfig] --> B{DB_TYPE value}
|
||
B -->|sqlserver| C[Use SQLServerConfig]
|
||
B -->|mysql| D[Use MySQLConfig]
|
||
C --> E[Load SQL Server settings]
|
||
D --> F[Load MySQL settings]
|
||
E --> G[Create connection]
|
||
F --> G
|
||
```
|
||
|
||
### 使用 ConfigLoader
|
||
|
||
**基本用法**:
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
|
||
# 从环境变量加载(推荐)
|
||
config = ConfigLoader.load(use_env=True)
|
||
|
||
# 从 JSON 文件加载(向后兼容)
|
||
config = ConfigLoader.load(use_env=False)
|
||
config = ConfigLoader.load_from_json("config/user_settings.json")
|
||
```
|
||
|
||
**保存配置**:
|
||
|
||
```python
|
||
# 保存到 JSON 文件
|
||
ConfigLoader.save(config, "config/user_settings.json")
|
||
|
||
# 保存到 .env 文件
|
||
ConfigLoader.save_to_env(config, ".env")
|
||
```
|
||
|
||
---
|
||
|
||
## 配置验证
|
||
|
||
### 验证流程图
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A[AppConfig.validate] --> B[erp.validate]
|
||
A --> C[database.validate]
|
||
A --> D[paths.validate]
|
||
A --> E[extraction.validate]
|
||
A --> F[validation.validate]
|
||
|
||
B --> G{Has errors?}
|
||
C --> G
|
||
D --> G
|
||
E --> G
|
||
F --> G
|
||
|
||
G -->|Yes| H[Collect all errors]
|
||
G -->|No| I[Validation passed]
|
||
H --> J[Return error list]
|
||
I --> K[Return empty list]
|
||
```
|
||
|
||
### 验证方法
|
||
|
||
**完整验证**:
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
|
||
config = ConfigLoader.load()
|
||
errors = config.validate()
|
||
|
||
if errors:
|
||
print("配置验证失败:")
|
||
for error in errors:
|
||
print(f" - {error}")
|
||
else:
|
||
print("配置验证通过")
|
||
```
|
||
|
||
**单独验证某个模块**:
|
||
|
||
```python
|
||
# 仅验证 ERP 配置
|
||
erp_errors = config.erp.validate()
|
||
|
||
# 仅验证数据库配置
|
||
db_errors = config.database.validate()
|
||
```
|
||
|
||
### 验证规则汇总
|
||
|
||
| 配置类 | 验证规则 |
|
||
|--------|----------|
|
||
| **ERPConfig** | URL、用户名、密码不能为空 |
|
||
| **DatabaseConfig** | 根据数据库类型验证相应字段 |
|
||
| **PathConfig** | 数据目录、生产 ID 文件不能为空 |
|
||
| **ExtractionConfig** | 批次大小 > 0 且 ≤ 1000 |
|
||
| **ValidationConfig** | 数据源有效、批次大小 > 0 且 ≤ 2000、匹配模式有效 |
|
||
|
||
---
|
||
|
||
## 使用示例
|
||
|
||
### 示例 1: 基本配置加载
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
|
||
# 加载配置
|
||
config = ConfigLoader.load()
|
||
|
||
# 访问配置值
|
||
print(f"ERP URL: {config.erp.url}")
|
||
print(f"数据库类型: {config.database.db_type.value}")
|
||
print(f"数据目录: {config.paths.data_dir}")
|
||
```
|
||
|
||
### 示例 2: 在数据库连接中使用
|
||
|
||
```python
|
||
from db.connection import get_connection
|
||
from config.loader import ConfigLoader
|
||
|
||
# 获取配置
|
||
config = ConfigLoader.load()
|
||
db_config = config.database
|
||
|
||
# 创建数据库连接
|
||
with get_connection(db_config) as db:
|
||
results = db.execute_query("SELECT * FROM table")
|
||
```
|
||
|
||
### 示例 3: 修改和保存配置
|
||
|
||
```python
|
||
from config.env_loader import update_env_file
|
||
from config.loader import ConfigLoader
|
||
|
||
# 更新环境变量
|
||
update_env_file(
|
||
ERP_HEADLESS="false",
|
||
EXTRACTION_BATCH_SIZE="200",
|
||
DB_TYPE="mysql"
|
||
)
|
||
|
||
# 重新加载配置
|
||
config = ConfigLoader.load()
|
||
```
|
||
|
||
### 示例 4: 在 Playwright 脚本中使用
|
||
|
||
```python
|
||
from playwright.sync_api import sync_playwright
|
||
from config.loader import ConfigLoader
|
||
|
||
config = ConfigLoader.load()
|
||
|
||
with sync_playwright() as p:
|
||
browser = p.chromium.launch(
|
||
headless=config.erp.headless
|
||
)
|
||
context = browser.new_context(
|
||
ignore_https_errors=config.erp.ignore_https_errors
|
||
)
|
||
page = context.new_page()
|
||
page.goto(config.erp.url)
|
||
# ... 继续自动化操作
|
||
```
|
||
|
||
### 示例 5: 条件配置
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
|
||
config = ConfigLoader.load()
|
||
|
||
# 根据配置决定行为
|
||
if config.database.db_type.value == "mysql":
|
||
print("使用 MySQL 连接")
|
||
# MySQL 特定逻辑
|
||
else:
|
||
print("使用 SQL Server 连接")
|
||
# SQL Server 特定逻辑
|
||
|
||
if config.extraction.verbose:
|
||
print("详细模式已启用")
|
||
```
|
||
|
||
### 示例 6: GUI 配置管理
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
from config.env_loader import update_env_file
|
||
|
||
def on_save_button_clicked():
|
||
"""保存 GUI 设置"""
|
||
config = ConfigLoader.load()
|
||
|
||
# 从 GUI 获取值
|
||
config.erp.headless = not headless_checkbox.isChecked()
|
||
config.extraction.batch_size = batch_size_spinbox.value()
|
||
|
||
# 保存到 .env
|
||
ConfigLoader.save_to_env(config)
|
||
```
|
||
|
||
---
|
||
|
||
## 迁移指南
|
||
|
||
### 从 JSON 配置迁移到 .env
|
||
|
||
**旧格式** (`config/user_settings.json`):
|
||
|
||
```json
|
||
{
|
||
"erp": {
|
||
"url": "https://68.11.34.30:8082/",
|
||
"username": "BLDpengqiangqiang",
|
||
"password": "your_password",
|
||
"headless": true,
|
||
"ignore_https_errors": true,
|
||
"auto_close_browser": true
|
||
},
|
||
"database": {
|
||
"db_type": "sqlserver",
|
||
"server": "192.168.110.114",
|
||
"database": "CompanyDB",
|
||
"username": "peng",
|
||
"password": "your_password",
|
||
"sqlserver": {
|
||
"driver": "ODBC Driver 18 for SQL Server",
|
||
"trust_server_certificate": "yes"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**新格式** (`.env`):
|
||
|
||
```bash
|
||
# ERP 配置
|
||
ERP_URL=https://68.11.34.30:8082/
|
||
ERP_USERNAME=BLDpengqiangqiang
|
||
ERP_PASSWORD=your_password
|
||
ERP_HEADLESS=true
|
||
ERP_IGNORE_HTTPS_ERRORS=true
|
||
ERP_AUTO_CLOSE_BROWSER=true
|
||
|
||
# 数据库配置
|
||
DB_TYPE=sqlserver
|
||
DB_SERVER=192.168.110.114
|
||
DB_NAME=CompanyDB
|
||
DB_USERNAME=peng
|
||
DB_PASSWORD=your_password
|
||
DB_SQLSERVER_DRIVER=ODBC Driver 18 for SQL Server
|
||
DB_TRUST_SERVER_CERTIFICATE=yes
|
||
```
|
||
|
||
### 迁移步骤
|
||
|
||
1. **导出现有配置**
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
|
||
# 从 JSON 加载
|
||
config = ConfigLoader.load(use_env=False)
|
||
|
||
# 保存到 .env
|
||
ConfigLoader.save_to_env(config, ".env")
|
||
```
|
||
|
||
2. **验证新配置**
|
||
|
||
```python
|
||
# 重新加载(现在从环境变量)
|
||
new_config = ConfigLoader.load(use_env=True)
|
||
|
||
# 验证
|
||
errors = new_config.validate()
|
||
if errors:
|
||
for error in errors:
|
||
print(f"错误: {error}")
|
||
```
|
||
|
||
3. **更新代码**
|
||
|
||
将所有配置加载代码改为使用环境变量:
|
||
|
||
```python
|
||
# 旧代码
|
||
config = ConfigLoader.load(use_env=False)
|
||
|
||
# 新代码
|
||
config = ConfigLoader.load() # use_env=True 是默认值
|
||
```
|
||
|
||
4. **备份并删除旧文件**
|
||
|
||
```bash
|
||
# 备份
|
||
cp config/user_settings.json config/user_settings.json.bak
|
||
|
||
# 删除或重命名
|
||
mv config/user_settings.json config/user_settings.json.deprecated
|
||
```
|
||
|
||
### 向后兼容性
|
||
|
||
如果需要同时支持两种配置方式:
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
from pathlib import Path
|
||
|
||
# 检查是否存在 .env 文件
|
||
if Path(".env").exists():
|
||
config = ConfigLoader.load(use_env=True)
|
||
else:
|
||
# 回退到 JSON 配置
|
||
config = ConfigLoader.load(use_env=False)
|
||
```
|
||
|
||
---
|
||
|
||
## 故障排除
|
||
|
||
### 常见问题
|
||
|
||
#### 1. 环境变量未加载
|
||
|
||
**症状**: 配置值都是默认值,不是 .env 文件中的值
|
||
|
||
**可能原因**:
|
||
- `.env` 文件不在项目根目录
|
||
- `.env` 文件格式错误
|
||
- 环境变量名称拼写错误
|
||
|
||
**解决方案**:
|
||
```python
|
||
# 检查 .env 文件是否被加载
|
||
from config.env_loader import get_env
|
||
print(get_env("ERP_URL")) # 应该输出 .env 中的值
|
||
|
||
# 检查文件是否存在
|
||
from pathlib import Path
|
||
print(Path(".env").exists())
|
||
|
||
# 手动指定 .env 文件路径
|
||
from config.env_loader import load_env_file
|
||
load_env_file("/path/to/your/.env")
|
||
```
|
||
|
||
#### 2. 配置验证失败
|
||
|
||
**症状**: `config.validate()` 返回错误列表
|
||
|
||
**常见错误**:
|
||
```
|
||
- ERP 密码不能为空
|
||
- 数据库密码不能为空
|
||
- 批次大小必须大于 0
|
||
```
|
||
|
||
**解决方案**:
|
||
```python
|
||
# 检查具体哪个模块有问题
|
||
config = ConfigLoader.load()
|
||
|
||
print("ERP 验证:", config.erp.validate())
|
||
print("数据库验证:", config.database.validate())
|
||
print("路径验证:", config.paths.validate())
|
||
|
||
# 更新缺失的配置
|
||
from config.env_loader import update_env_file
|
||
update_env_file(
|
||
ERP_PASSWORD="your_actual_password",
|
||
DB_PASSWORD="your_actual_db_password"
|
||
)
|
||
```
|
||
|
||
#### 3. 数据库连接失败
|
||
|
||
**症状**: `pyodbc.Error` 或连接超时
|
||
|
||
**可能原因**:
|
||
- 数据库配置错误
|
||
- 网络不通
|
||
- 驱动未安装
|
||
|
||
**调试步骤**:
|
||
```python
|
||
# 1. 检查配置
|
||
from config.loader import ConfigLoader
|
||
config = ConfigLoader.load()
|
||
|
||
print(f"数据库类型: {config.database.db_type.value}")
|
||
print(f"服务器: {config.database.server}")
|
||
print(f"数据库: {config.database.database}")
|
||
|
||
# 2. 测试连接
|
||
from db.connection import get_connection
|
||
try:
|
||
with get_connection(config.database) as db:
|
||
result = db.execute_query("SELECT 1")
|
||
print("连接成功:", result)
|
||
except Exception as e:
|
||
print("连接失败:", e)
|
||
|
||
# 3. 检查 SQL Server 驱动
|
||
import pyodbc
|
||
print("可用驱动:", pyodbc.drivers())
|
||
```
|
||
|
||
#### 4. 路径错误
|
||
|
||
**症状**: `FileNotFoundError` 或文件找不到
|
||
|
||
**解决方案**:
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
from pathlib import Path
|
||
|
||
config = ConfigLoader.load()
|
||
|
||
# 使用绝对路径
|
||
data_dir = Path(config.paths.data_dir).absolute()
|
||
print(f"数据目录: {data_dir}")
|
||
print(f"目录存在: {data_dir.exists()}")
|
||
|
||
# 确保目录存在
|
||
data_dir.mkdir(parents=True, exist_ok=True)
|
||
|
||
# 检查路径分隔符(Windows 使用反斜杠或正斜杠都可以)
|
||
# 推荐使用 pathlib 处理路径
|
||
```
|
||
|
||
#### 5. 布尔值不生效
|
||
|
||
**症状**: 设置了 `ERP_HEADLESS=false` 但浏览器仍然无头模式运行
|
||
|
||
**原因**: 布尔值格式不正确
|
||
|
||
**解决方案**:
|
||
```bash
|
||
# 正确的布尔值格式
|
||
ERP_HEADLESS=false # 小写
|
||
ERP_HEADLESS=False # 首字母大写
|
||
ERP_HEADLESS=0 # 数字
|
||
ERP_HEADLESS=no # no/off
|
||
|
||
# 错误的格式
|
||
ERP_HEADLESS=False # 如果有引号会被当作字符串
|
||
```
|
||
|
||
### 调试技巧
|
||
|
||
**打印所有配置**:
|
||
|
||
```python
|
||
from config.loader import ConfigLoader
|
||
import json
|
||
|
||
config = ConfigLoader.load()
|
||
print(json.dumps(config.to_dict(), indent=2, ensure_ascii=False))
|
||
```
|
||
|
||
**追踪配置加载**:
|
||
|
||
```python
|
||
from config.env_loader import load_env_file, get_env
|
||
import os
|
||
|
||
# 手动加载并打印
|
||
load_env_file()
|
||
|
||
# 检查特定环境变量
|
||
print("ERP_URL from env:", os.getenv("ERP_URL"))
|
||
print("ERP_URL from get_env:", get_env("ERP_URL"))
|
||
```
|
||
|
||
**验证测试**:
|
||
|
||
```bash
|
||
# 运行配置测试
|
||
python tests/test_env_config.py
|
||
```
|
||
|
||
### 错误信息解读
|
||
|
||
| 错误信息 | 原因 | 解决方案 |
|
||
|----------|------|----------|
|
||
| `ERP URL 不能为空` | `ERP_URL` 未设置 | 检查 .env 文件中的 ERP_URL |
|
||
| `无效的数据源: xxx` | `VALIDATION_DATA_SOURCE` 值错误 | 使用有效值: database_full, database_filtered, excel_existing, excel_full |
|
||
| `批次大小不应超过 2000` | `VALIDATION_BATCH_SIZE` 太大 | 减小批次大小(SQL Server 参数限制) |
|
||
| `无效的匹配模式: xxx` | `VALIDATION_MATCH_MODE` 值错误 | 使用 substring 或 exact |
|
||
|
||
---
|
||
|
||
## 附录
|
||
|
||
### A. 环境变量快速参考
|
||
|
||
#### ERP 配置
|
||
```bash
|
||
ERP_URL=https://68.11.34.30:8082/
|
||
ERP_USERNAME=BLDpengqiangqiang
|
||
ERP_PASSWORD=your_password
|
||
ERP_HEADLESS=true
|
||
ERP_IGNORE_HTTPS_ERRORS=true
|
||
ERP_AUTO_CLOSE_BROWSER=true
|
||
```
|
||
|
||
#### SQL Server 配置
|
||
```bash
|
||
DB_TYPE=sqlserver
|
||
DB_SERVER=192.168.110.114
|
||
DB_NAME=CompanyDB
|
||
DB_USERNAME=peng
|
||
DB_PASSWORD=your_password
|
||
DB_SQLSERVER_DRIVER=ODBC Driver 18 for SQL Server
|
||
DB_TRUST_SERVER_CERTIFICATE=yes
|
||
```
|
||
|
||
#### MySQL 配置
|
||
```bash
|
||
DB_TYPE=mysql
|
||
DB_NAME=BLD_DB
|
||
DB_USERNAME=remote_user
|
||
DB_PASSWORD=your_password
|
||
DB_MYSQL_HOST=192.168.31.83
|
||
DB_MYSQL_PORT=3306
|
||
DB_MYSQL_CHARSET=utf8mb4
|
||
```
|
||
|
||
#### 路径配置
|
||
```bash
|
||
PATH_DATA_DIR=D:/python/playwrite/data/
|
||
PATH_PRODUCTION_ID_FILE=ProductionID.txt
|
||
PATH_DEFAULT_OUTPUT=离散备料计划维护_合并.xlsx
|
||
PATH_VALIDATION_OUTPUT=物料状态校验结果.xlsx
|
||
```
|
||
|
||
#### 提取配置
|
||
```bash
|
||
EXTRACTION_BATCH_SIZE=100
|
||
EXTRACTION_VERBOSE=true
|
||
EXTRACTION_AUTO_CONVERT=true
|
||
EXTRACTION_MERGE_BATCHES=true
|
||
EXTRACTION_ENABLE_DB_PERSISTENCE=false
|
||
```
|
||
|
||
#### 校验配置
|
||
```bash
|
||
VALIDATION_DATA_SOURCE=database_full
|
||
VALIDATION_USE_DATABASE=true
|
||
VALIDATION_BATCH_SIZE=2000
|
||
VALIDATION_ENABLE_CRUD=false
|
||
VALIDATION_DEFAULT_MANAGER=
|
||
VALIDATION_MATCH_MODE=substring
|
||
```
|
||
|
||
### B. 配置文件模板
|
||
|
||
创建 `.env.example` 文件(不含敏感信息):
|
||
|
||
```bash
|
||
# ===========================
|
||
# ERP 系统配置
|
||
# ===========================
|
||
ERP_URL=https://your-erp-system.com/
|
||
ERP_USERNAME=your_username
|
||
ERP_PASSWORD=your_password_here
|
||
ERP_HEADLESS=true
|
||
ERP_IGNORE_HTTPS_ERRORS=true
|
||
ERP_AUTO_CLOSE_BROWSER=true
|
||
|
||
# ===========================
|
||
# 数据库配置
|
||
# ===========================
|
||
DB_TYPE=sqlserver
|
||
DB_SERVER=your_server_address
|
||
DB_NAME=your_database_name
|
||
DB_USERNAME=your_db_username
|
||
DB_PASSWORD=your_db_password_here
|
||
DB_SQLSERVER_DRIVER=ODBC Driver 18 for SQL Server
|
||
DB_TRUST_SERVER_CERTIFICATE=yes
|
||
|
||
# ===========================
|
||
# 路径配置
|
||
# ===========================
|
||
PATH_DATA_DIR=./data/
|
||
PATH_PRODUCTION_ID_FILE=ProductionID.txt
|
||
PATH_DEFAULT_OUTPUT=output.xlsx
|
||
PATH_VALIDATION_OUTPUT=validation_result.xlsx
|
||
|
||
# ===========================
|
||
# 数据提取配置
|
||
# ===========================
|
||
EXTRACTION_BATCH_SIZE=100
|
||
EXTRACTION_VERBOSE=true
|
||
EXTRACTION_AUTO_CONVERT=true
|
||
EXTRACTION_MERGE_BATCHES=true
|
||
EXTRACTION_ENABLE_DB_PERSISTENCE=false
|
||
|
||
# ===========================
|
||
# 校验配置
|
||
# ===========================
|
||
VALIDATION_DATA_SOURCE=database_full
|
||
VALIDATION_USE_DATABASE=true
|
||
VALIDATION_BATCH_SIZE=2000
|
||
VALIDATION_ENABLE_CRUD=false
|
||
VALIDATION_DEFAULT_MANAGER=
|
||
VALIDATION_MATCH_MODE=substring
|
||
```
|
||
|
||
### C. 相关文档链接
|
||
|
||
- [数据库架构文档](./DATABASE_ARCHITECTURE.md)
|
||
- [项目 README](../README.md)
|
||
- [开发指南](./DEVELOPMENT.md)
|
||
- [API 文档](./API.md)
|
||
|
||
### D. 更新日志
|
||
|
||
**v2.0.0** (2025-01-XX)
|
||
- 从 JSON 配置迁移到环境变量
|
||
- 添加自动加载机制
|
||
- 支持多数据库配置(SQL Server/MySQL)
|
||
- 添加配置验证功能
|
||
|
||
**v1.0.0** (2024-XX-XX)
|
||
- 初始版本
|
||
- JSON 配置文件支持
|
||
|
||
---
|
||
|
||
**文档版本**: 2.0.0
|
||
**最后更新**: 2025-02-09
|
||
**维护者**: Development Team
|