Commit Graph

20 Commits

Author SHA1 Message Date
Misaka
a520c9a05c fix: add SQL Server to MySQL syntax conversion for parameter placeholders and table names
Automatically convert SQL Server syntax to MySQL-compatible format in
execute_query() and execute_update() methods:

- Parameter placeholders: ? -> %s
- Table names: [dbo].[TableName] -> dbo_TableName
- Column names: [ColumnName] -> ColumnName

This fixes the "Not all parameters were used in the SQL statement" error
that occurred when running queries originally written for SQL Server against
MySQL database.

The conversion is transparent to existing code, allowing SQL queries
throughout the codebase to work with MySQL without modification.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 21:49:08 +08:00
Misaka
f2bbcfc426 Merge branch 'mysql' into dev
Resolved merge conflict in utils/material_status_validator.py by keeping
both deduplication statistics and error messages for empty results.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 20:44:52 +08:00
Misaka_Company
e5f6f4ee0c feat: add MaterialCode-based deduplication to material validation queries
Implement database-level deduplication using ROW_NUMBER() window function
to remove duplicate material records based on MaterialCode field.

Changes:
- Add query_all_distinct_by_material_code() to deduplicate full table results
- Add query_by_source_numbers_distinct() to deduplicate filtered results
- Update validator to use new deduplication methods with statistics logging
- Retention strategy: keep earliest record (CreateDate ASC, SequenceNumber ASC)

Benefits:
- Reduces data redundancy from 54,358 to 1,679 records (96.91% reduction)
- Improves validation result readability by removing duplicate materials
- Maintains backward compatibility with existing query methods

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 16:22:35 +08:00
Misaka
85be5e166b fix: support MySQL in production order query by using dynamic placeholders
- Use conn.get_placeholder() instead of hardcoded '?'
- Adapt table and column names for both MySQL and SQL Server
- Load database type from config to handle dialect differences
- Fix ProgrammingError when running with MySQL database

Co-Authored-By: Gemini 2.0 Flash <gemini-cli@google.com>
2026-02-13 18:19:13 +08:00
Misaka_Company
6d81d5cb76 feat: add computer name-based silent login (无感登录)
Implement automatic authentication based on computer name to enable
passwordless login for registered computers with fallback to manual
authentication.

Changes:
- Add SessionManager.login_by_computer_name() for silent login attempt
- Add BIPUsersDAO.authenticate_by_computer_name() for computer name lookup
- Update BIPUsersDAO.create_user() to support optional computer_name parameter
- Update main_ui.py to try silent login before showing login dialog
- Display current computer name in login dialog for user reference

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 16:44:47 +08:00
Misaka_Company
72468ddf23 fix: correct SQL parameter count in DiscreteMaterialPlanDAO batch insert
Fix parameter mismatch error when saving discrete material plan data.
Changed placeholder count from 28 to 29 to match the 29 columns in the
INSERT statement (Factory through BOMVersion).

Resolves error: "The SQL contains 28 parameter markers, but 29 parameters were supplied"

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 16:18:17 +08:00
Misaka_Company
cff5e78c1d refactor: optimize material deletion with exact code matching and database queries
- Change from fuzzy material name matching to exact material code matching
- Migrate from MaterialsTypeToBeDeleted to MaterialsToBeDeleted table
- Add should_delete_material() for on-demand database queries
- Remove in-memory material list loading to reduce memory footprint
- Update documentation to reflect new matching logic
- Remove main scripts from git tracking (contain user-specific config)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 10:53:53 +08:00
Misaka
aaa46ef282 feat: migrate configuration to .env environment variables
This commit implements a complete migration from JSON-based configuration
to .env environment variables, providing better security and flexibility.

Key Changes:
- Add python-dotenv dependency for environment variable support
- Create config/env_loader.py with type conversion utilities
- Add from_env() class methods to all config dataclasses
- Update ConfigLoader to prioritize environment variables
- Add save_to_env() method for .env file management
- Implement database connection factory pattern
- Add base DAO and connection classes for better abstraction
- Support both SQL Server and MySQL with unified interface
- Create migration script (scripts/migrate_to_env.py)
- Update GUI to read/write .env files
- Add comprehensive migration documentation

New Files:
- config/env_loader.py - Environment variable loader
- db/base_connection.py - Base database connection interface
- db/base_dao.py - Base DAO with common utilities
- db/connection_factory.py - Factory for creating connections
- db/mysql_connection.py - MySQL-specific connection
- db/sqlserver_connection.py - SQL Server-specific connection
- db/table_name_converter.py - SQL dialect converter
- scripts/migrate_to_env.py - Configuration migration tool
- docs/ENV_MIGRATION.md - Complete migration guide
- .env.example - Environment variable template

Testing:
- Verified MySQL connection (8.0.44)
- Tested all DAO operations
- Confirmed 150 tables accessible
- Validated configuration loading

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 22:39:14 +08:00
Misaka_Company
04b99292ad feat: implement user authentication and permission-based data filtering
Add user login system with role-based access control:
- Admin users see all data and can filter by any manager
- Regular users only see records where ManagerName matches their username

New components:
- BIPUsersDAO: user authentication and management
- SessionManager: singleton session state management
- LoginDialog: modal login UI for app startup
- init_users.sql: initial user setup script

Permission enforcement:
- Material validation tab: hide manager filter for non-admin, force filter by current user
- Material type management: hide filter UI for non-admin, filter at database level
- Window title and status bar display current user info

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 18:17:31 +08:00
Misaka_Company
cd21e54abf feat: add material deletion confirmation with checkbox UI
- Add MaterialsToBeDeletedDAO for managing material deletion records by MaterialCode
- Add MaterialValidationResult dataclass for enhanced validation results
- Add CheckboxTreeview component with selectable checkbox functionality
- Refactor material validation UI with new columns: select, material name, code, spec, model, manager
- Add double-click to edit manager name functionality
- Add select all/deselect all buttons
- Add confirm deletion button to write selected records to database
- Prioritize MaterialsToBeDeleted.ManagerName over type-based matching when displaying
- Export results to Excel with selection state

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 16:40:28 +08:00
Misaka_Company
0ef1b86f12 refactor: rename MaterialsToBeDeleted to MaterialsTypeToBeDeleted
Rename database table and related classes from MaterialsToBeDeleted to MaterialsTypeToBeDeleted for better semantic clarity.

Changes:
- Rename MaterialsToBeDeletedDAO class to MaterialsTypeToBeDeletedDAO
- Update all SQL queries to reference MaterialsTypeToBeDeleted table
- Update import statement in material_status_validator.py
- Update documentation comments

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 14:15:51 +08:00
Misaka_Company
2096a51f55 feat: add database-driven material validation with multi-mode support
This commit enhances the material validation functionality to support
database-driven workflows alongside the existing Excel-based approach.

## New Features

### DAO Layer
- Add ProductionContractDataDAO for querying production contract data
- Add MaterialsToBeDeletedDAO with full CRUD operations
- Enhance DiscreteMaterialPlanDAO with query_all(), query_by_source_numbers(),
  and get_unique_material_names() methods

### Validation Modes
Support for 4 validation modes in MaterialValidationTab:
1. Database Full - Query all materials from DiscreteMaterialPlanData
2. Database Filtered - Query by ProductionID.txt file
3. Excel Existing - Validate from existing Excel file
4. Excel Full - Complete workflow with ERP extraction

### Configuration
- Add ValidationConfig dataclass with data_source, batch_size, match_mode,
  enable_crud_operations, and default_manager fields
- Update ConfigLoader to support validation configuration
- Add validation settings section in SettingsTab GUI

### Query Chain
Implementation of full query chain:
ProductionID.txt (总排号) → productionContractData (生产订单号) →
DiscreteMaterialPlanData (SourceNumber) → MaterialName →
MaterialsToBeDeleted comparison

## Backward Compatibility
All existing Excel-based validation methods remain unchanged, ensuring
no breaking changes for existing workflows.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 13:08:31 +08:00
Misaka_Company
53a1e33e45 feat: add SQL Server database persistence for extracted material plan data
Add optional database persistence feature that automatically saves extracted
discrete material plan data to SQL Server. Users can enable this feature in
the settings tab.

Changes:
- Add enable_db_persistence flag to ExtractionConfig (default: disabled)
- Create DiscreteMaterialPlanDAO for database operations with REPLACE pattern
- Update progress tracking to include database persistence stage (90-100%)
- Add database persistence checkbox in settings UI
- Remove verbose logging checkbox from data extraction UI (config-only now)
- Update extraction workflow to save merged DataFrame to database

Progress weights adjusted:
- download: 65% -> 60%
- database: 10% (new stage)
- Other stages adjusted accordingly

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 12:21:58 +08:00
Misaka_Company
2ca53f6a55 style: normalize code formatting across the codebase
- Standardize quote style (single to double quotes)
- Improve code formatting consistency
- Apply formatting to utilities, GUI components, and tools
- Update imports and docstrings for consistency

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 14:56:35 +08:00
Misaka_Company
13bc4520bf refactor: centralize configuration management with type-safe schema
Major changes:
- Add dataclass-based configuration schema with validation (config/schema.py)
- Create centralized config loader and default values (config/defaults.py, config/loader.py)
- Remove duplicate database_config.py, merge into unified structure
- Consolidate browser settings into ERP config
- Add batch_size parameter support to extractor

Bug fixes:
- Fix settings save error by updating config paths (browser.* → erp.*)
- Fix batch_size not being applied in data extraction

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 14:27:28 +08:00
Misaka_Company
c9e68a2bf0 fix: handle large dataset queries and type conversion
- Add batch processing to production order query to handle SQL Server's 2100 parameter limit
- Fix material name column index from Q to R column
- Add string type conversion to prevent TypeError in material matching

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 11:28:33 +08:00
Misaka_Company
c6b97443e5 feat: add material status validation tool
Add MaterialStatusValidator tool to check material status and match materials to be deleted:
- Add get_all_materials_to_delete() function to fetch all materials from database
- Add utils/material_status_validator.py with MaterialStatusValidator class
- Add validate_material_status.py script to run the validation
- Delete obsolete materialDelete.py file

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 10:11:13 +08:00
Misaka_Company
c176063bfa feat: add material matching logic for discrete material plan cleaner
- Add db/materials_to_delete.py for querying materials to delete by manager
- Add manager_name parameter to DiscreteMaterialPlanCleaner
- Implement material keyword matching logic in process_order
- Update file paths from orderID.txt to ProductionID.txt

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 16:50:14 +08:00
Misaka_Company
902ff9b831 refactor: use database for production order queries
- Add db/production_order_query.py component for querying production orders
- Replace file-based orderID.txt with database-driven approach
- Read ProductionID.txt (总排号) and query [26年压力表合同数据] table
- Update both extraction and cleaning scripts to use new component
- Change parameter: order_id_file → production_id_file

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 16:06:40 +08:00
Misaka_Company
bbf84b7f8c docs: update project structure and add database documentation
- Add tests/ directory to .gitignore
- Document database connectivity with pyodbc and config
- Add project structure and file organization guidelines
- Update installation steps to use requirements.txt
- Document new dependencies: pandas, openpyxl, pyodbc
2026-01-23 15:00:55 +08:00