---
name: excel-report-converter
description: Generate Python scripts to convert report-style Excel files to database-record format. Use for converting Excel files with multiple stacked reports into flat database tables, analyzing report structure, creating custom conversion scripts for specific Excel report formats, and transforming hierarchical report data (header + detail lines + footer) into normalized database records
---

# Excel Report Converter

Generate Python scripts to convert report-style Excel files (multiple reports per worksheet) into database-record format (flat table).

## When to Use

Use this skill when:
- User provides an Excel file with multiple similar reports stacked vertically in one worksheet
- Each report has hierarchical structure: header information + detail data rows + footer information
- User wants to convert to database-record format where header fields are repeated for each detail row
- The report structure is consistent across all reports in the file

## Workflow

### Step 1: Extract and Analyze Report Structure

Use the `excel-to-markdown` skill to convert the Excel file to markdown format for analysis:

```bash
# Convert Excel to markdown to analyze structure
python3 scripts/excel_to_markdown.py input.xlsx -o /tmp/analysis.md --show-rows --show-cols
```

Read the markdown file and identify:
1. **Report delimiters**: How to identify where each report starts/ends (e.g., specific title in column 1)
2. **Header structure**: Which rows contain header information and what fields are in each column
3. **Detail table**: Which row contains column headers and where detail data starts/ends
4. **Footer structure**: Which rows contain footer information and where the data is located
5. **Row separators**: Are there empty rows between reports? After the last row?

### Step 2: Generate Conversion Script

Create a Python script with the following structure:

```python
#!/usr/bin/env python3
"""
Convert report-style Excel files to database-record format
Customized for: [describe the report format]
"""

import argparse
import sys
from pathlib import Path
from openpyxl import load_workbook
from openpyxl.utils import get_column_letter

class ReportParser:
    def __init__(self, worksheet):
        self.ws = worksheet
        self.max_row = worksheet.max_row
        self.max_col = worksheet.max_column

    def find_report_blocks(self):
        """Identify start and end rows for each report block"""
        # Implement block detection logic
        # Look for report delimiters (e.g., title in column 1)
        # Exclude trailing empty rows
        pass

    def parse_header(self, start_row):
        """Extract header fields from the report header section"""
        # Map cell positions to field names
        pass

    def parse_detail_rows(self, start_row, end_row):
        """Extract detail rows from the report detail section"""
        # Identify column header row
        # Extract data until empty row or footer starts
        pass

    def parse_footer(self, end_row):
        """Extract footer fields from the report footer section"""
        # Map cell positions to field names
        pass

    def parse_report(self, start_row, end_row):
        """Parse complete report: header + details + footer"""
        header = self.parse_header(start_row)
        details = self.parse_detail_rows(start_row, end_row)
        footer = self.parse_footer(end_row)
        return header, details, footer

def convert_to_database_format(input_file, output_file, sheet_name=None):
    """Convert report format to database-record format"""
    wb = load_workbook(input_file, data_only=True)
    ws = wb[sheet_name] if sheet_name else wb.active

    parser = ReportParser(ws)
    blocks = parser.find_report_blocks()

    # Collect all records
    all_records = []
    for start_row, end_row in blocks:
        header, details, footer = parser.parse_report(start_row, end_row)
        # Merge header + each detail row + footer
        for detail in details:
            record = {**header, **detail, **footer}
            all_records.append(record)

    # Write output
    from openpyxl import Workbook
    wb_out = Workbook()
    ws_out = wb_out.active

    # Write header row
    fields = list(all_records[0].keys())
    for col_idx, field_name in enumerate(fields, start=1):
        ws_out.cell(row=1, column=col_idx, value=field_name)

    # Write data rows
    for row_idx, record in enumerate(all_records, start=2):
        for col_idx, field_name in enumerate(fields, start=1):
            ws_out.cell(row=row_idx, column=col_idx, value=record.get(field_name))

    wb_out.save(output_file)

def main():
    parser = argparse.ArgumentParser(description='Convert report-style Excel to database-record format')
    parser.add_argument('input_file', help='Input Excel file')
    parser.add_argument('output_file', help='Output Excel file')
    parser.add_argument('--sheet', help='Worksheet name (default: active)')
    args = parser.parse_args()

    convert_to_database_format(args.input_file, args.output_file, args.sheet)

if __name__ == '__main__':
    main()
```

**Key Implementation Points:**

- **Column mapping**: Use fixed column indices (1-indexed) or use openpyxl's column letters
- **Empty row detection**: Check if all cells in a row are None/empty to identify boundaries
- **Date handling**: Excel dates may be numeric - use appropriate conversion if needed
- **Merged cells**: Check for merged cells in header/footer sections
- **Data types**: Preserve original data types (strings, numbers, dates) from source

### Step 3: Test and Verify

Run the generated script to convert the file:

```bash
python3 scripts/custom_converter.py input.xlsx output.xlsx
```

Use `excel-to-markdown` to verify the output:

```bash
# For large files, only verify first 10 records
python3 scripts/excel_to_markdown.py output.xlsx -o /tmp/verify.md --show-rows --show-cols --rows 1:11
```

Read `/tmp/verify.md` and verify:
1. All records are extracted (count should match total detail rows across all reports)
2. Header fields are correctly populated for each record
3. Detail fields are correctly mapped
4. Footer fields are correctly populated
5. No data loss or corruption

**Important:** For large output files with many records, use `--rows 1:11` to only convert and verify the first 10 data records (plus header row). This avoids processing time and memory issues when verifying large exports.

### Step 4: Iterate if Needed

If verification reveals issues:
1. Identify the specific problem (wrong column index, missing field, incorrect row detection)
2. Fix the script accordingly
3. Re-run the conversion
4. Re-verify with excel-to-markdown
5. Repeat until output is correct

## Common Patterns

### Report Block Detection

Most report-style Excel files use one of these patterns:

**Pattern A: Title-based delimiters**
```
Row 1:  Report Title
Row 2:  [header data]
...
Row N:  [footer data]
Row N+1: [empty or next report title]
```

Look for a specific value in column 1 (e.g., "离散备料计划", "Purchase Order")

**Pattern B: Fixed-size reports**
All reports have the same number of rows. Calculate block size and divide evenly.

### Header/Detail/Footer Separation

**Typical structure:**
- Rows 1-4: Header information (labels and values in specific columns)
- Row 5: Empty separator
- Row 6: Detail table column headers
- Rows 7+: Detail data rows
- Row N-1: Footer row 1 (creator, date, approver)
- Row N: Footer row 2 (printer, print date)
- Row N+1: Empty separator or next report

### Column Mapping Strategies

**Strategy 1: Fixed column positions**
Use when report format is consistent:
```python
factory = ws.cell(row=2, column=3).value  # C列
order_no = ws.cell(row=2, column=9).value  # I列
```

**Strategy 2: Search by label**
Use when column positions may vary:
```python
# Find column by searching for label in first row
for col in range(1, max_col + 1):
    if ws.cell(row=header_row, column=col).value == "Order No":
        order_no_col = col
        break
```

### Field Merging Strategy

When creating database records:
1. Parse header fields once per report
2. Parse footer fields once per report
3. For each detail row, create a merged record: `{**header, **detail_row, **footer}`
4. This repeats header/footer fields for each detail line (database normalization)

## Example Report Structure Analysis

When analyzing the markdown output, look for:

```
Row 1: | Report Title | | | ...
Row 2: | Field: | | Value | | Field: | | Value | ...
Row 3: | Field: | | Value | | Field: | | Value | ...
Row 4: | (empty or separator)
Row 5: | Col1 | Col2 | Col3 | ...  (detail table headers)
Row 6: | val1 | val2 | val3 | ...  (first detail row)
Row 7: | val1 | val2 | val3 | ...  (second detail row)
...
Row N: | Creator: | | Name | | Date: | | 2025-01-01 |
Row N+1: | Printer: | | Name | | Print Date: | | 2025-01-02 |
Row N+2: | (empty) or next report title
```

Map this structure to the script functions:
- `parse_header(start_row)`: Extract fields from rows 2-3
- `parse_detail_rows(start_row, end_row)`: Extract rows 6+ until empty/footer
- `parse_footer(end_row)`: Extract fields from rows N to N+1

## Resources

The `excel-to-markdown` skill provides Excel-to-markdown conversion for structure analysis.

This skill does not include bundled scripts or references - each conversion script is generated dynamically based on the specific report format being analyzed.