Track materials that are not in the delete list and display them in
a new "未处理物料" (Unmatched Materials) section in the report.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
Wrap the order processing logic in try-finally to guarantee cleanup
of the popup page regardless of whether processing succeeds or fails.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add ExecutionConfig for dryrun settings in config schema
- Create DeleteProgressWindow widget for real-time progress display
- Integrate delete execution flow in MaterialValidationTab with threading
- Add dryrun checkbox for admin users in settings
- Add progress callback support to DiscreteMaterialPlanCleaner
- Add markdown report generation with statistics
- Include tkinterweb and markdown2 dependencies for report rendering
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
Add dryrun boolean parameter to DiscreteMaterialPlanCleaner.
When dryrun=True, the save_button_locator.click() is skipped,
allowing preview of changes without actually saving.
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add get_materials_to_delete_by_managers() function in db/materials_to_delete.py
- Supports querying by single manager, multiple managers, or all managers
- Uses IN clause for multi-manager queries
- Refactor DiscreteMaterialPlanCleaner to accept manager_names parameter
- Accepts str, List[str], or None (for all managers)
- Automatically normalizes parameter types in __init__
- Updates preload_data() with appropriate log messages
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
- Add identify_input_type() function to detect input format:
- production_id: 2-digit + 1-letter + serial (e.g., 25A1, 25A12345)
- order_number: SC + 14 digits (e.g., SC70202602120120)
- Refactor MaterialStatusValidator to intelligently handle both input
types in validate_from_database() and validate_from_database_enhanced()
- Update log messages to reflect the new dual-input capability
Co-Authored-By: Claude (glm-5) <noreply@anthropic.com>
This change adds robust wait mechanisms to prevent race conditions when
performing delete and save operations in the ERP system.
Changes:
- Add row number change detection after delete to confirm completion
- Add save button disappearance detection to confirm save completion
- Add row number range check (7000-8000) to skip specific rows
- Enable previously commented save button click
- Improve logging with detailed status messages
- Add timeout handling (10s for delete, 60s for save)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Apply consistent code formatting including:
- Single quotes to double quotes for string literals
- Multi-line formatting for long expressions
- Trailing whitespace cleanup
- Blank line adjustments for PEP 8 compliance
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Resolved merge conflict in utils/material_status_validator.py by keeping
both deduplication statistics and error messages for empty results.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add centralized logging mechanism that simultaneously outputs to console
and GUI log components, improving code maintainability and consistency.
Changes:
- Add gui/log_config.py for centralized logging configuration
- Add gui/widgets/log_handler.py as bridge between logging and LogText
- Integrate unified logging into DataExtractionTab and MaterialValidationTab
- Initialize logging system in MainWindow on startup
- Improve error messages in material_status_validator for empty results
- Add documentation for logging mechanism and refactoring
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Rename utility files to use English names for better cross-platform compatibility:
- utils/离散备料计划维护数据提取.py → utils/discrete_material_plan_extractor.py
- utils/离散备料计划维护数据清理.py → utils/discrete_material_plan_cleaner.py
Update all import statements across the codebase and documentation references.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Restructure conditional flow to check status before detail count
- Add handling for orders with no material plan data
- Add handling for completed orders (status=完成)
- Add warning for orders with unrecognized status
- Fix deletion condition to check for non-zero pending quantity
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implement database-level deduplication using ROW_NUMBER() window function
to remove duplicate material records based on MaterialCode field.
Changes:
- Add query_all_distinct_by_material_code() to deduplicate full table results
- Add query_by_source_numbers_distinct() to deduplicate filtered results
- Update validator to use new deduplication methods with statistics logging
- Retention strategy: keep earliest record (CreateDate ASC, SequenceNumber ASC)
Benefits:
- Reduces data redundancy from 54,358 to 1,679 records (96.91% reduction)
- Improves validation result readability by removing duplicate materials
- Maintains backward compatibility with existing query methods
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Added validation to check if cumulative pending quantity is empty before allowing material deletion, with appropriate emoji indicators for delete eligibility.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Change from fuzzy material name matching to exact material code matching
- Migrate from MaterialsTypeToBeDeleted to MaterialsToBeDeleted table
- Add should_delete_material() for on-demand database queries
- Remove in-memory material list loading to reduce memory footprint
- Update documentation to reflect new matching logic
- Remove main scripts from git tracking (contain user-specific config)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add MaterialsToBeDeletedDAO for managing material deletion records by MaterialCode
- Add MaterialValidationResult dataclass for enhanced validation results
- Add CheckboxTreeview component with selectable checkbox functionality
- Refactor material validation UI with new columns: select, material name, code, spec, model, manager
- Add double-click to edit manager name functionality
- Add select all/deselect all buttons
- Add confirm deletion button to write selected records to database
- Prioritize MaterialsToBeDeleted.ManagerName over type-based matching when displaying
- Export results to Excel with selection state
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Rename database table and related classes from MaterialsToBeDeleted to MaterialsTypeToBeDeleted for better semantic clarity.
Changes:
- Rename MaterialsToBeDeletedDAO class to MaterialsTypeToBeDeletedDAO
- Update all SQL queries to reference MaterialsTypeToBeDeleted table
- Update import statement in material_status_validator.py
- Update documentation comments
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit enhances the material validation functionality to support
database-driven workflows alongside the existing Excel-based approach.
## New Features
### DAO Layer
- Add ProductionContractDataDAO for querying production contract data
- Add MaterialsToBeDeletedDAO with full CRUD operations
- Enhance DiscreteMaterialPlanDAO with query_all(), query_by_source_numbers(),
and get_unique_material_names() methods
### Validation Modes
Support for 4 validation modes in MaterialValidationTab:
1. Database Full - Query all materials from DiscreteMaterialPlanData
2. Database Filtered - Query by ProductionID.txt file
3. Excel Existing - Validate from existing Excel file
4. Excel Full - Complete workflow with ERP extraction
### Configuration
- Add ValidationConfig dataclass with data_source, batch_size, match_mode,
enable_crud_operations, and default_manager fields
- Update ConfigLoader to support validation configuration
- Add validation settings section in SettingsTab GUI
### Query Chain
Implementation of full query chain:
ProductionID.txt (总排号) → productionContractData (生产订单号) →
DiscreteMaterialPlanData (SourceNumber) → MaterialName →
MaterialsToBeDeleted comparison
## Backward Compatibility
All existing Excel-based validation methods remain unchanged, ensuring
no breaking changes for existing workflows.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add optional database persistence feature that automatically saves extracted
discrete material plan data to SQL Server. Users can enable this feature in
the settings tab.
Changes:
- Add enable_db_persistence flag to ExtractionConfig (default: disabled)
- Create DiscreteMaterialPlanDAO for database operations with REPLACE pattern
- Update progress tracking to include database persistence stage (90-100%)
- Add database persistence checkbox in settings UI
- Remove verbose logging checkbox from data extraction UI (config-only now)
- Update extraction workflow to save merged DataFrame to database
Progress weights adjusted:
- download: 65% -> 60%
- database: 10% (new stage)
- Other stages adjusted accordingly
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Break down query phase into 3 detailed steps (read file, query database, complete)
- Split download batch process into 7 granular steps per batch (clear, fill, search, wait, select, configure, download)
- Divide conversion phase into 2 steps per file plus 3 final steps (merge, cleanup)
- Add 'action' parameter to all progress reports for better categorization
- Improve user feedback with more detailed status messages and completion indicators
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Major changes:
- Add dataclass-based configuration schema with validation (config/schema.py)
- Create centralized config loader and default values (config/defaults.py, config/loader.py)
- Remove duplicate database_config.py, merge into unified structure
- Consolidate browser settings into ERP config
- Add batch_size parameter support to extractor
Bug fixes:
- Fix settings save error by updating config paths (browser.* → erp.*)
- Fix batch_size not being applied in data extraction
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add ProgressInfo and ProgressCalculator classes for structured progress tracking
- Implement RealtimeOutput for immediate log display without buffering
- Add progress callback support to DiscreteMaterialPlanExtractor
- Use queue-based thread-safe communication for GUI progress updates
- Fix log output issue where all content appeared at once after completion
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add batch processing to production order query to handle SQL Server's 2100 parameter limit
- Fix material name column index from Q to R column
- Add string type conversion to prevent TypeError in material matching
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add MaterialStatusValidator tool to check material status and match materials to be deleted:
- Add get_all_materials_to_delete() function to fetch all materials from database
- Add utils/material_status_validator.py with MaterialStatusValidator class
- Add validate_material_status.py script to run the validation
- Delete obsolete materialDelete.py file
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changed footer detection to look ahead at the next row instead of the current row, and simplified to only check for '制单人'.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add db/materials_to_delete.py for querying materials to delete by manager
- Add manager_name parameter to DiscreteMaterialPlanCleaner
- Implement material keyword matching logic in process_order
- Update file paths from orderID.txt to ProductionID.txt
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add db/production_order_query.py component for querying production orders
- Replace file-based orderID.txt with database-driven approach
- Read ProductionID.txt (总排号) and query [26年压力表合同数据] table
- Update both extraction and cleaning scripts to use new component
- Change parameter: order_id_file → production_id_file
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace the class-based locator strategy with a text-based filter using regex to accurately identify the material code input field. This change targets elements matching "材料编码" followed by 11 digits, increasing reliability. Also uncommented page2.pause() to assist with debugging the iteration loop.
Refine element selection by introducing a specific parent container `.card-table-side-box` to enhance locator accuracy. Replace the previous text-parsing approach for material codes with direct input value retrieval. Additionally, add support for extracting cumulative pending and outbound quantities to improve data completeness.
Added logic to extract and validate the "Material Preparation Status" (备料状态) before modifying discrete material plans.
The cleanup process now only proceeds if the status is "审批通过" (Approval Passed). Orders with other statuses are skipped to prevent errors during modification. This change also includes code cleanup by removing commented-out debug statements.
Remove manual exit prompt in main script to allow automated execution.
Add logic in DiscreteMaterialPlanCleaner to navigate the "Material Plan" popup, handling nested iframes and waiting for data to load. Implement iteration over material details to extract codes and names, handling cases where no details exist.
Implement a new automation script and utility to handle the cleaning of
discrete material plan maintenance data.
- Add main_clean.py as the primary entry point, configuring the cleaner
instance and reading order IDs.
- Add DiscreteMaterialPlanCleaner in utils/ utilizing Playwright to automate
browser interactions.
- Implement logic for user authentication (login/logout), UI setup for
queries, and processing individual orders.
- Include verbose logging and debug mode support for troubleshooting.
