HH/docs/STAFF_HIERARCHY_INTEGRATION.md

# Staff Hierarchy Integration Guide

## Overview

This guide explains how to integrate staff data with management hierarchy into the PX360 system. The system supports importing staff data from CSV files, including manager relationships that create a complete organizational hierarchy.

## Key Features

### 1. Staff Model Enhancements

The Staff model has been enhanced with the following fields to support hierarchy and CSV data:

- **`name`**: Stores the original full name from CSV (preserves exact formatting)
- **`phone`**: Staff phone number (optional)
- **`location`**: Staff location/building (e.g., "Nuzha")
- **`report_to`**: Self-referential field for manager hierarchy
- **`country`**: Staff's country of origin
- **`department_name`**: Original department name from CSV
- **`section`**: Department section
- **`subsection`**: Department subsection

### 2. Manager Hierarchy

The system uses a self-referential relationship (`report_to`) to create management hierarchies:

```python
staff.report_to  # Points to their manager
staff.direct_reports  # Returns all staff who report to this person
```

## CSV Format

### Required Columns

The CSV file must contain the following columns (case-sensitive):

```csv
Staff ID,Name,Location,Department,Section,Subsection,AlHammadi Job Title,Country,Gender,Manager
```

### Optional Columns

- **Phone**: Staff phone number

### Example CSV Data

```csv
Staff ID,Name,Location,Department,Section,Subsection,AlHammadi Job Title,Country,Gender,Manager
4,ABDULAZIZ SALEH ALHAMMADI,Nuzha,Senior Management Offices,COO Office,,Chief Operating Officer,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI
1049,VARGHESE   NINAN,Nuzha,Human Resource ,Employee Relations,,Porter,India,Male,1053 - MAJID SALEM SAEED ALNAHDI
1053,MAJID SALEM SAEED ALNAHDI,Nuzha,Human Resource ,Administration,,Human Resources Manager ,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI
1086,QAMAR   KHALIFAH,Nuzha,Corporate Administration ,Quality Management ,,Quality Management Director,Eritrea,Female,8639 - REEMA  SALEH MOHAMMED  AL HAMMADI
```

### Column Details

| Column | Description | Example |
|--------|-------------|---------|
| Staff ID | Unique employee identifier | 1049 |
| Name | Full name (stored as-is) | VARGHESE NINAN |
| Location | Physical location/building | Nuzha |
| Department | Department name | Human Resource |
| Section | Department section | Employee Relations |
| Subsection | Department subsection | (blank) |
| AlHammadi Job Title | Job title | Porter |
| Country | Staff's country | India |
| Gender | Male/Female/Other | Male |
| Manager | Manager ID and name (format: "ID - Name") | 1053 - MAJID SALEM SAEED ALNAHDI |
| Phone (optional) | Contact phone number | +966123456789 |

### Manager Field Format

The Manager field uses the format: **`ID - Name`**

Example: `1053 - MAJID SALEM SAEED ALNAHDI`

The system extracts:
- **ID**: `1053` (used to link to manager record)
- **Name**: `MAJID SALEM SAEED ALNAHDI` (for display/logging)

## Import Command

### Basic Usage

```bash
python manage.py import_staff_csv <csv_file> --hospital-code <hospital_code>
```

### Command Options

| Option | Description | Default |
|--------|-------------|---------|
| `csv_file` | Path to CSV file (required) | - |
| `--hospital-code` | Hospital code to assign staff to (required) | - |
| `--staff-type` | Staff type: physician/nurse/admin/other | admin |
| `--skip-existing` | Skip staff with existing employee_id | False |
| `--update-existing` | Update existing staff records | False |
| `--create-users` | Create user accounts for imported staff | False |
| `--dry-run` | Preview without making changes | False |

### Example Commands

#### 1. Import with Dry Run (Preview)
```bash
python manage.py import_staff_csv staff_data.csv \
  --hospital-code ALHAMMADI \
  --dry-run
```

#### 2. Import New Staff Only
```bash
python manage.py import_staff_csv staff_data.csv \
  --hospital-code ALHAMMADI \
  --skip-existing
```

#### 3. Update Existing Staff
```bash
python manage.py import_staff_csv staff_data.csv \
  --hospital-code ALHAMMADI \
  --update-existing
```

#### 4. Import with User Account Creation
```bash
python manage.py import_staff_csv staff_data.csv \
  --hospital-code ALHAMMADI \
  --staff-type admin \
  --create-users
```

## Department Mapping

The system automatically maps CSV departments to internal department codes:

```python
DEPARTMENT_MAPPING = {
    'Senior Management Offices': 'ADM-005',
    'Human Resource': 'ADM-005',
    'Corporate Administration': 'ADM-005',
    'Emergency': 'EMR-001',
    'Outpatient': 'OUT-002',
    'Inpatient': 'INP-003',
    'Diagnostics': 'DIA-004',
    'Administration': 'ADM-005',
}
```

Unmapped departments default to **'ADM-005'** (Administration).

## Hierarchy Building Process

The import process uses a **two-pass algorithm**:

### Pass 1: Create/Update Staff Records
1. Parse each row from CSV
2. Create or update Staff record
3. Store mapping of employee_id → Staff object

### Pass 2: Link Manager Relationships
1. For each staff member with a manager_id
2. Find the manager using the mapping
3. Set `report_to` field to establish hierarchy

This ensures all records exist before linking relationships.

## Data Flow

```
CSV File → Import Command → Parse CSV → Create/Update Staff → Link Managers → Database
                                      ↓
                            Department Mapping
```

## Querying Hierarchy

### Get Staff's Manager
```python
staff = Staff.objects.get(employee_id='1049')
manager = staff.report_to
print(manager.name)  # "MAJID SALEM SAEED ALNAHDI"
```

### Get Direct Reports
```python
manager = Staff.objects.get(employee_id='1053')
direct_reports = manager.direct_reports.all()
for report in direct_reports:
    print(report.name)
```

### Get Full Hierarchy Chain
```python
def get_manager_chain(staff):
    chain = []
    current = staff
    while current.report_to:
        chain.append(current.report_to)
        current = current.report_to
    return chain

staff = Staff.objects.get(employee_id='1049')
chain = get_manager_chain(staff)
# Returns list of managers up the chain
```

## Admin Interface

The Django admin interface displays hierarchy information:

### Staff List View
- Shows: Name, Type, Job Title, Employee ID, Phone, Hospital, Department, Manager, Country, Status

### Staff Detail View
- **Personal Information**: Name, first/last name, location
- **Role**: Staff type, job title
- **Professional**: License, specialization, employee ID, email, phone
- **Organization**: Hospital, department, section, subsection, location
- **Hierarchy**: Manager (report_to)
- **Personal**: Country, gender
- **Account**: User account link

### Search Fields
You can search by:
- Original name (from CSV)
- First/last name
- Employee ID
- Job title
- Phone number
- Department/section

## API Endpoints

The Staff serializer includes hierarchy fields:

```json
{
  "id": "uuid",
  "name": "VARGHESE NINAN",
  "first_name": "VARGHESE",
  "last_name": "NINAN",
  "employee_id": "1049",
  "phone": "+966123456789",
  "location": "Nuzha",
  "department": "Human Resource",
  "report_to": {
    "id": "uuid",
    "name": "MAJID SALEM SAEED ALNAHDI",
    "employee_id": "1053"
  },
  "report_to_name": "MAJID SALEM SAEED ALNAHDI",
  "direct_reports_count": 2,
  "has_user_account": true
}
```

## Troubleshooting

### Issue: Manager Not Found
**Problem**: Manager ID in CSV doesn't match any imported staff.

**Solution**:
1. Ensure manager is also in the CSV file
2. Verify manager ID format matches (numbers only)
3. Use `--dry-run` to preview and identify issues

### Issue: Department Not Mapped
**Problem**: Department name doesn't match mapping.

**Solution**:
1. Add department to `DEPARTMENT_MAPPING` in `import_staff_csv.py`
2. Or use default mapping (ADM-005)

### Issue: Duplicate Employee IDs
**Problem**: Multiple records with same employee_id.

**Solution**:
- Use `--skip-existing` to skip duplicates
- Or use `--update-existing` to update existing records

### Issue: Name Parsing Issues
**Problem**: Name not split correctly into first/last name.

**Solution**:
- The system splits on first space: "John Doe" → First: "John", Last: "Doe"
- Original name is preserved in `name` field
- You can manually correct first/last name in admin if needed

## Best Practices

### 1. Prepare CSV File
- Remove extra spaces from headers
- Ensure consistent date/name formats
- Validate manager IDs exist in file
- Remove duplicates

### 2. Use Dry Run First
```bash
python manage.py import_staff_csv staff.csv --hospital-code ALHAMMADI --dry-run
```

### 3. Import in Stages
1. Dry run to check for issues
2. Import with `--skip-existing` to add new staff
3. Review results
4. Use `--update-existing` if updates needed

### 4. Verify Hierarchy
```python
# Check hierarchy after import
for staff in Staff.objects.all():
    if staff.report_to:
        print(f"{staff.name} → {staff.report_to.name}")
```

### 5. Back Up Data
Always backup before importing with `--update-existing`:
```bash
python manage.py dumpdata organizations.Staff > staff_backup.json
```

## Migration Notes

If you're migrating from an existing system:

1. **Export** existing staff data to CSV format
2. **Map** department names to internal codes
3. **Update** manager IDs to match CSV employee IDs
4. **Test** import with small sample first
5. **Validate** hierarchy relationships after import

## Support

For issues or questions:
1. Check Django logs for detailed error messages
2. Use `--dry-run` to preview imports
3. Review imported data in Django admin
4. Check CSV format matches requirements

## Related Documentation

- [Staff Model Documentation](STAFF_MODEL.md)
- [Department Structure](DEPARTMENT_STRUCTURE_UPDATE.md)
- [User Account Creation](STAFF_USER_ACCOUNT_FEATURE_COMPLETE.md)