Commit Graph

19 Commits

Author SHA1 Message Date
Misaka_Company
3180ccacb8 feat: enhance progress reporting granularity in data extraction workflow
- Break down query phase into 3 detailed steps (read file, query database, complete)
- Split download batch process into 7 granular steps per batch (clear, fill, search, wait, select, configure, download)
- Divide conversion phase into 2 steps per file plus 3 final steps (merge, cleanup)
- Add 'action' parameter to all progress reports for better categorization
- Improve user feedback with more detailed status messages and completion indicators

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 15:17:18 +08:00
Misaka_Company
2ca53f6a55 style: normalize code formatting across the codebase
- Standardize quote style (single to double quotes)
- Improve code formatting consistency
- Apply formatting to utilities, GUI components, and tools
- Update imports and docstrings for consistency

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 14:56:35 +08:00
Misaka_Company
13bc4520bf refactor: centralize configuration management with type-safe schema
Major changes:
- Add dataclass-based configuration schema with validation (config/schema.py)
- Create centralized config loader and default values (config/defaults.py, config/loader.py)
- Remove duplicate database_config.py, merge into unified structure
- Consolidate browser settings into ERP config
- Add batch_size parameter support to extractor

Bug fixes:
- Fix settings save error by updating config paths (browser.* → erp.*)
- Fix batch_size not being applied in data extraction

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 14:27:28 +08:00
Misaka_Company
885d87dff8 feat: add progress callback mechanism for real-time updates in data extraction
- Add ProgressInfo and ProgressCalculator classes for structured progress tracking
- Implement RealtimeOutput for immediate log display without buffering
- Add progress callback support to DiscreteMaterialPlanExtractor
- Use queue-based thread-safe communication for GUI progress updates
- Fix log output issue where all content appeared at once after completion

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 14:06:30 +08:00
Misaka_Company
c9e68a2bf0 fix: handle large dataset queries and type conversion
- Add batch processing to production order query to handle SQL Server's 2100 parameter limit
- Fix material name column index from Q to R column
- Add string type conversion to prevent TypeError in material matching

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 11:28:33 +08:00
Misaka_Company
c6b97443e5 feat: add material status validation tool
Add MaterialStatusValidator tool to check material status and match materials to be deleted:
- Add get_all_materials_to_delete() function to fetch all materials from database
- Add utils/material_status_validator.py with MaterialStatusValidator class
- Add validate_material_status.py script to run the validation
- Delete obsolete materialDelete.py file

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 10:11:13 +08:00
Misaka_Company
a78afe383f fix: adjust footer detection logic in excel converter
Changed footer detection to look ahead at the next row instead of the current row, and simplified to only check for '制单人'.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 08:39:18 +08:00
Misaka_Company
c176063bfa feat: add material matching logic for discrete material plan cleaner
- Add db/materials_to_delete.py for querying materials to delete by manager
- Add manager_name parameter to DiscreteMaterialPlanCleaner
- Implement material keyword matching logic in process_order
- Update file paths from orderID.txt to ProductionID.txt

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 16:50:14 +08:00
Misaka_Company
902ff9b831 refactor: use database for production order queries
- Add db/production_order_query.py component for querying production orders
- Replace file-based orderID.txt with database-driven approach
- Read ProductionID.txt (总排号) and query [26年压力表合同数据] table
- Update both extraction and cleaning scripts to use new component
- Change parameter: order_id_file → production_id_file

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 16:06:40 +08:00
Misaka_Company
4b0a882b69 refactor: improve material code input selection
Replace the class-based locator strategy with a text-based filter using regex to accurately identify the material code input field. This change targets elements matching "材料编码" followed by 11 digits, increasing reliability. Also uncommented page2.pause() to assist with debugging the iteration loop.
2026-01-23 14:30:35 +08:00
Misaka_Company
ebd53b168e refactor: improve locator precision and data extraction
Refine element selection by introducing a specific parent container `.card-table-side-box` to enhance locator accuracy. Replace the previous text-parsing approach for material codes with direct input value retrieval. Additionally, add support for extracting cumulative pending and outbound quantities to improve data completeness.
2026-01-23 13:52:52 +08:00
Misaka_Company
cf88800444 feat: Check preparation status before cleaning data
Added logic to extract and validate the "Material Preparation Status" (备料状态) before modifying discrete material plans.

The cleanup process now only proceeds if the status is "审批通过" (Approval Passed). Orders with other statuses are skipped to prevent errors during modification. This change also includes code cleanup by removing commented-out debug statements.
2026-01-23 11:11:22 +08:00
Misaka_Company
201f65d715 feat: implement material plan data cleaning automation
Remove manual exit prompt in main script to allow automated execution.
Add logic in DiscreteMaterialPlanCleaner to navigate the "Material Plan" popup, handling nested iframes and waiting for data to load. Implement iteration over material details to extract codes and names, handling cases where no details exist.
2026-01-22 18:14:56 +08:00
Misaka_Company
8708709cac feat: add discrete material plan data cleaning tool
Implement a new automation script and utility to handle the cleaning of
discrete material plan maintenance data.

- Add main_clean.py as the primary entry point, configuring the cleaner
  instance and reading order IDs.
- Add DiscreteMaterialPlanCleaner in utils/ utilizing Playwright to automate
  browser interactions.
- Implement logic for user authentication (login/logout), UI setup for
  queries, and processing individual orders.
- Include verbose logging and debug mode support for troubleshooting.
2026-01-22 15:55:38 +08:00
Misaka_Company
92d4572985 Refactor ExcelConverter to streamline logging and enhance summary reporting 2026-01-16 15:59:25 +08:00
Misaka_Company
bb9348d12b Refactor DiscreteMaterialPlanExtractor to use logout function from auth module and update headless mode in main script 2026-01-16 15:20:05 +08:00
Misaka_Company
93f986260e Add logout functionality to DiscreteMaterialPlanExtractor for improved session management 2026-01-16 14:59:18 +08:00
Misaka_Company
51cba4caee Refactor main script to utilize DiscreteMaterialPlanExtractor for data extraction and streamline order processing 2026-01-16 13:51:20 +08:00
Misaka_Company
4d441be304 Add ExcelConverter module and demo script for Excel data transformation 2026-01-16 12:51:06 +08:00