HH/docs/STAFF_CSV_IMPORT_GUIDE.md

# Staff CSV Import Guide

This guide explains how to import staff data with hierarchy information from CSV files into the PX360 system.

## Overview

The staff CSV import command allows you to bulk import staff records from CSV files, including:
- Basic staff information (name, ID, job title)
- Organizational details (department, section, subsection)
- Manager-reportee relationships (hierarchy)
- Personal information (country, gender)

## CSV Format

### Required Columns

| Column | Description | Example |
|--------|-------------|---------|
| Staff ID | Unique employee ID | `4` |
| Name | Full name of staff | `ABDULAZIZ SALEH ALHAMMADI` |
| Location | Work location | `Nuzha` |
| Department | Department name | `Human Resource` |
| Section | Department section | `Employee Relations` |
| Subsection | Department subsection | (can be empty) |
| AlHammadi Job Title | Job title | `Chief Operating Officer` |
| Country | Country | `Saudi Arabia` |
| Gender | Gender (Male/Female) | `Male` |
| Manager | Manager ID and name | `2 - MOHAMMAD SALEH AL HAMMADI` |

### Manager Field Format

The Manager field must be in the format: `ID - Name`

Example: `2 - MOHAMMAD SALEH AL HAMMADI`

The system will:
1. Extract the ID from the beginning
2. Look for a staff record with that employee_id
3. Create a manager-reportee relationship

### Sample CSV File

```csv
Staff ID,Name,Location,Department,Section,Subsection,AlHammadi Job Title,Country,Gender,Manager
4,ABDULAZIZ SALEH ALHAMMADI,Nuzha,Senior Management Offices,COO Office,,Chief Operating Officer,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI
1049,VARGHESE   NINAN,Nuzha,Human Resource ,Employee Relations,,Porter,India,Male,1053 - MAJID SALEM SAEED ALNAHDI
1053,MAJID SALEM SAEED ALNAHDI,Nuzha,Human Resource ,Administration,,Human Resources Manager ,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI
1086,QAMAR   KHALIFAH,Nuzha,Corporate Administration ,Quality Management ,,Quality Management Director,Eritrea,Female,8639 - REEMA  SALEH MOHAMMED  AL HAMMADI
1103,EMALYN HAWARI  HARADJI,Nuzha,Human Resource ,Administration,,Porter,Philippines,Female,1053 - MAJID SALEM SAEED ALNAHDI
```

## Department Mapping

The command automatically maps CSV department names to system department codes:

| CSV Department Name | System Department Code |
|---------------------|------------------------|
| Senior Management Offices | ADM-005 |
| Human Resource | ADM-005 |
| Human Resource (with trailing space) | ADM-005 |
| Corporate Administration | ADM-005 |
| Emergency | EMR-001 |
| Outpatient | OUT-002 |
| Inpatient | INP-003 |
| Diagnostics | DIA-004 |
| Administration | ADM-005 |

If a department is not found in the mapping, it defaults to `ADM-005` (Administration).

## Usage

### Basic Command

```bash
python manage.py import_staff_csv <csv_file> --hospital-code <hospital_code>
```

### Command Options

| Option | Required | Default | Description |
|--------|----------|---------|-------------|
| `csv_file` | Yes | - | Path to CSV file to import |
| `--hospital-code` | Yes | - | Hospital code to assign staff to |
| `--staff-type` | No | `admin` | Staff type (physician/nurse/admin/other) |
| `--skip-existing` | No | False | Skip staff with existing employee_id |
| `--update-existing` | No | False | Update existing staff records |
| `--create-users` | No | False | Create user accounts for imported staff |
| `--dry-run` | No | False | Preview without making changes |

### Examples

#### 1. Import with dry-run (preview)

```bash
python manage.py import_staff_csv sample_staff_data.csv --hospital-code HH --dry-run
```

#### 2. Import staff for a specific hospital

```bash
python manage.py import_staff_csv staff_data.csv --hospital-code HH
```

#### 3. Import with staff type specification

```bash
python manage.py import_staff_csv physicians.csv --hospital-code HH --staff-type physician
```

#### 4. Import and skip existing records

```bash
python manage.py import_staff_csv staff_data.csv --hospital-code HH --skip-existing
```

#### 5. Import and update existing records

```bash
python manage.py import_staff_csv staff_data.csv --hospital-code HH --update-existing
```

#### 6. Import with user account creation

```bash
python manage.py import_staff_csv staff_data.csv --hospital-code HH --create-users
```

## Import Process

The import process works in two passes:

### First Pass: Create/Update Staff Records

1. Read and parse the CSV file
2. For each row:
   - Check if staff with employee_id already exists
   - Create new staff record or update existing one
   - Map department, set all fields
   - Skip manager linking (done in second pass)

### Second Pass: Link Manager Relationships

1. Iterate through all processed staff records
2. For each staff member with a manager_id:
   - Look up the manager by employee_id
   - Create the report_to relationship
   - Log success or warning if manager not found

## Output and Results

### Successful Import Example

```
============================================================
Staff CSV Import Command
============================================================
✓ Found hospital: Alhammadi Hospital (HH)
✓ Found 15 departments in hospital

Configuration:
  CSV file: sample_staff_data.csv
  Hospital: Alhammadi Hospital
  Staff type: admin
  Skip existing: False
  Update existing: False
  Create user accounts: False
  Dry run: False

Reading CSV file...
✓ Found 5 staff records in CSV
  [1] ✓ Created: ABDULAZIZ SALEH ALHAMMADI
  [2] ✓ Created: VARGHESE   NINAN
  [3] ✓ Created: MAJID SALEM SAEED ALNAHDI
  [4] ✓ Created: QAMAR   KHALIFAH
  [5] ✓ Created: EMALYN HAWARI  HARADJI

Linking manager relationships...
  [1] ⚠ Manager not found: 2 for ABDULAZIZ SALEH ALHAMMADI
  [2] ✓ Linked VARGHESE   NINAN → MAJID SALEM SAEED ALNAHDI
  [3] ⚠ Manager not found: 2 for MAJID SALEM SAEED ALNAHDI
  [4] ⚠ Manager not found: 8639 for QAMAR   KHALIFAH
  [5] ✓ Linked EMALYN HAWARI  HARADJI → MAJID SALEM SAEED ALNAHDI

============================================================
Import Summary:
  Staff records created: 5
  Staff records updated: 0
  Staff records skipped: 0
  Manager relationships linked: 2
  Errors: 0
============================================================
Import completed successfully!
```

### Status Indicators

- `✓` - Success
- `⊘` - Skipped
- `✗` - Error
- `⚠` - Warning

## Handling Managers Not Found

When a manager ID in the CSV doesn't exist in the system (either because:
1. The manager hasn't been imported yet
2. The manager works at a different location
3. The manager ID is incorrect

The import will:
- Still create the staff record
- Log a warning about the missing manager
- Continue with other records

To resolve this, you can:
1. Import all staff first (including managers)
2. Re-run the import with `--update-existing` to link the relationships
3. Or manually link managers in the admin interface

## Best Practices

### 1. Use Dry-Run First

Always test your CSV with dry-run mode:

```bash
python manage.py import_staff_csv staff_data.csv --hospital-code HH --dry-run
```

### 2. Import All Staff at Once

To ensure manager relationships are linked correctly, import all staff records in a single CSV file.

### 3. Validate Manager IDs

Ensure all manager IDs in the CSV correspond to staff IDs that exist in the same CSV file.

### 4. Check Department Mapping

Verify that department names in your CSV match the expected mapping or update the `DEPARTMENT_MAPPING` in the command file if needed.

### 5. Handle Special Characters

The CSV file should use UTF-8 encoding to handle Arabic names and special characters correctly.

## Troubleshooting

### Issue: "CSV file not found"

**Solution:** Ensure the CSV file path is correct relative to the project root.

### Issue: "Hospital with code 'XXX' not found"

**Solution:** 
1. Check available hospitals: `python manage.py shell -c "from apps.organizations.models import Hospital; [print(f'{h.code} - {h.name}') for h in Hospital.objects.all()]"`
2. Use the correct hospital code

### Issue: "No valid staff data found in CSV"

**Solution:** 
1. Verify CSV has the correct headers
2. Check CSV encoding (should be UTF-8)
3. Ensure data is properly formatted

### Issue: Manager relationships not linking

**Solution:**
1. Verify manager IDs exist in the CSV
2. Re-run import with `--update-existing`
3. Check that both manager and reportee are in the same hospital

### Issue: Department not found

**Solution:**
1. Check the department mapping in the command
2. Ensure departments are seeded for the hospital: `python manage.py seed_departments`
3. Update `DEPARTMENT_MAPPING` if needed

## Viewing Imported Data

### Via Django Admin

1. Go to `/admin/organizations/staff/`
2. View all imported staff records
3. Click on a staff member to see:
   - Basic information
   - Department and hierarchy
   - Manager (report_to) relationship
   - Direct reports

### Via Shell

```python
from apps.organizations.models import Staff

# View all imported staff
staff = Staff.objects.all()
for s in staff:
    print(f"{s.employee_id} - {s.get_full_name()} - {s.department_name}")
    if s.report_to:
        print(f"  Manager: {s.report_to.get_full_name()}")
    print()

# View hierarchy
def print_hierarchy(staff, level=0):
    indent = "  " * level
    print(f"{indent}• {staff.get_full_name()} ({staff.job_title})")
    for report in staff.direct_reports.all():
        print_hierarchy(report, level + 1)

# Start with top-level staff (no manager)
top_level = Staff.objects.filter(report_to__isnull=True)
for staff in top_level:
    print_hierarchy(staff)
```

## Related Commands

- `python manage.py seed_departments` - Seed departments for hospitals
- `python manage.py seed_staff` - Seed sample staff data (for testing)

## Model Fields

The Staff model includes these new fields for CSV import:

- `department_name` - Stores original department name from CSV
- `section` - Department section
- `subsection` - Department subsection
- `country` - Staff country
- `gender` - Staff gender
- `report_to` - Foreign key to manager (self-referential)

## API Integration

The imported staff data is available via the API:

```python
# Get all staff
GET /api/organizations/staff/

# Get staff with hierarchy
GET /api/organizations/staff/?include=direct_reports

# Filter by department
GET /api/organizations/staff/?department_name=Human%20Resource

# Filter by manager
GET /api/organizations/staff/?report_to__employee_id=1053
```

## Notes

- The import command runs within a database transaction - if any error occurs, all changes are rolled back
- Duplicate employee_ids are handled based on `--skip-existing` and `--update-existing` flags
- The command is idempotent - running it multiple times with the same data won't create duplicates (when using `--skip-existing`)
- User account creation (`--create-users`) requires staff to have email addresses

## Support

For issues or questions:
1. Check this documentation
2. Review the import command file: `apps/organizations/management/commands/import_staff_csv.py`
3. Check the Staff model: `apps/organizations/models.py`