HH/docs/STAFF_HIERARCHY_INTEGRATION_SUMMARY.md

Staff Hierarchy Integration - Complete Summary

Executive Summary

The PX360 system now includes comprehensive support for importing, storing, and visualizing staff hierarchy data from CSV files. This document provides a complete overview of the integration process and available features.

Your Data Structure

CSV Format

Staff ID,Name,Location,Department,Section,Subsection,AlHammadi Job Title,Country,Gender,Manager
4,ABDULAZIZ SALEH ALHAMMADI,Nuzha,Senior Management Offices,COO Office,,Chief Operating Officer,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI
1049,VARGHESE   NINAN,Nuzha,Human Resource ,Employee Relations,,Porter,India,Male,1053 - MAJID SALEM SAEED ALNAHDI
1053,MAJID SALEM SAEED ALNAHDI,Nuzha,Human Resource ,Administration,,Human Resources Manager ,Saudi Arabia,Male,2 - MOHAMMAD SALEH AL HAMMADI

Data Mapping

CSV Field	PX360 Field	Notes
Staff ID	employee_id	Unique identifier (required)
Name	first_name, last_name	Split into first/last name
Location	❌ Not stored	Physical location
Department	department	Department name (lookup/create)
Section	❌ Not stored	Sub-department
Subsection	❌ Not stored	Further subdivision
AlHammadi Job Title	job_title	Job title
Country	❌ Not stored	Country
Gender	❌ Not stored	Gender
Manager	report_to	Manager ID (extracted from "ID - NAME")

Manager Field Parsing

Format: "ID - NAME"

Example: 2 - MOHAMMAD SALEH AL HAMMADI

System Behavior:

1. Extracts ID: 2
2. Looks up Staff record with employee_id = "2"
3. Sets report_to field to that Staff record
4. Creates hierarchical relationship

Integration Architecture

1. Data Storage

Model: apps.organizations.models.Staff

Key Hierarchy Fields:

employee_id = CharField(unique=True)           # From CSV "Staff ID"
first_name = CharField()                         # From CSV "Name" (first part)
last_name = CharField()                          # From CSV "Name" (last part)
job_title = CharField()                          # From CSV "AlHammadi Job Title"
department = ForeignKey(Department)              # From CSV "Department"
report_to = ForeignKey('self')                   # From CSV "Manager" (ID part)

Relationship: report_to creates self-referential foreign key for hierarchy

2. Data Import

Command: python manage.py import_staff_csv path/to/file.csv

Location: apps/organizations/management/commands/import_staff_csv.py

Process:

1. Read CSV file
2. Parse each row
3. Extract manager ID from Manager field
4. Create/update Staff records
5. Set report_to relationships
6. Validate data integrity

3. Hierarchy Visualization Options

The system provides two visualization approaches:

Option A: HTML-Based Hierarchy (Original)

URL: /organizations/staff/hierarchy/

Characteristics:

• Server-side rendering
• Static tree structure
• Page reloads for interactions
• Print-friendly
• Good for reports

Technology: Django templates + Bootstrap

Documentation: docs/STAFF_HIERARCHY_VIEW.md

Option B: D3.js Interactive Hierarchy ⭐ NEW

URL: /organizations/staff/hierarchy/d3/

Characteristics:

• Client-side rendering
• Interactive tree visualization
• Instant interactions (no reloads)
• Multiple layouts
• Zoom & pan
• Rich interactivity

Technology: D3.js v7.9.0 + JavaScript

Documentation: docs/D3_HIERARCHY_INTEGRATION.md

Complete Workflow

Step 1: Prepare Data

1. Ensure CSV has correct headers
2. Verify all Manager IDs exist in Staff ID column
3. Check department names are consistent
4. Save as UTF-8 encoded CSV

Step 2: Import Data

# Option 1: Use import command
python manage.py import_staff_csv sample_staff_data.csv

# Option 2: Use Django Admin
# Navigate to /admin/organizations/staff/
# Use import feature

# Option 3: Manual entry
# Navigate to /organizations/staff/create/
# Enter data manually

Step 3: Verify Data

# Check import
from apps.organizations.models import Staff

# Count staff
print(f"Total staff: {Staff.objects.count()}")

# Check top managers
print(f"Top managers: {Staff.objects.filter(report_to__isnull=True).count()}")

# Check specific staff
staff = Staff.objects.get(employee_id="4")
print(f"{staff.first_name} reports to {staff.report_to}")
print(f"Direct reports: {staff.direct_reports.count()}")

Step 4: Visualize Hierarchy

HTML-Based View

URL: /organizations/staff/hierarchy/
- See tree structure
- Click to expand/collapse (page reloads)
- Search by name or ID
- Filter by hospital/department

D3.js Interactive View

URL: /organizations/staff/hierarchy/d3/
- Interactive tree visualization
- Click to expand/collapse (instant)
- Three layouts: horizontal, vertical, radial
- Zoom and pan support
- Real-time search
- Node sizing options
- Tooltips with details
- Double-click to view staff details

Feature Comparison

Feature	HTML	D3.js
Rendering	Server-side	Client-side
Interactivity	Basic (reload)	High (instant)
Layouts	1 fixed	3 switchable
Zoom/Pan	❌	✅
Animations	❌	✅
Tooltips	❌	✅
Search	Server-side	Client-side
Export	✅ Print	Limited
Best For	Reports	Exploration

Technical Implementation

1. Import Command

File: apps/organizations/management/commands/import_staff_csv.py

Key Functions:

• handle(): Main command entry point
• parse_manager(): Extract ID from "ID - NAME" format
• create_or_update_staff(): Create/update staff records
• create_department(): Auto-create departments
• set_manager_relationship(): Set report_to field

2. Hierarchy API

File: apps/organizations/views.py

Endpoint: /organizations/api/staff/hierarchy/

Method: GET

Parameters:

• hospital (optional): Filter by hospital
• department (optional): Filter by department

Response: JSON hierarchy tree with statistics

3. D3 Visualization

File: templates/organizations/staff_hierarchy_d3.html

Key Components:

• fetchHierarchyData(): Fetch data from API
• update(): Render/update visualization
• diagonal(): Create curved links
• click(): Toggle node expansion
• getNodeRadius(): Calculate node sizes

Features:

• Three tree layouts (horizontal, vertical, radial)
• Zoom and pan behavior
• Search with auto-navigation
• Expand/collapse animations
• Responsive sizing

Documentation Structure

Integration Guides

1. STAFF_HIERARCHY_IMPORT_GUIDE.md

   • How to import CSV data
   • Data mapping details
   • Troubleshooting import issues
   • Usage examples

2. D3_HIERARCHY_INTEGRATION.md

   • D3.js implementation details
   • Features and capabilities
   • User guide
   • Customization options
   • Troubleshooting

3. STAFF_HIERARCHY_VIEW.md (existing)

   • HTML-based hierarchy view
   • Original implementation

Code Documentation

• apps/organizations/models.py - Staff model definition
• apps/organizations/views.py - API endpoint
• apps/organizations/ui_views.py - View functions
• apps/organizations/urls.py - URL routing
• templates/organizations/staff_hierarchy.html - HTML view
• templates/organizations/staff_hierarchy_d3.html - D3 view
• apps/organizations/management/commands/import_staff_csv.py - Import command

Quick Start

Import Your Data

# 1. Prepare CSV file
# Ensure format matches your example

# 2. Import data
python manage.py import_staff_csv your_staff_data.csv

# 3. Verify import
python manage.py shell
>>> from apps.organizations.models import Staff
>>> Staff.objects.count()
>>> Staff.objects.filter(report_to__isnull=True).count()

View Hierarchy

HTML-Based View

1. Navigate to: /organizations/staff/hierarchy/
2. See tree structure with top-level managers
3. Click to expand/collapse nodes
4. Use search to find specific staff
5. Filter by hospital/department

D3 Interactive View

1. Navigate to: /organizations/staff/hierarchy/d3/
2. See interactive chart
3. Click nodes to expand/collapse
4. Use mouse wheel to zoom
5. Drag to pan
6. Search for staff (auto-navigate)
7. Change layout from dropdown
8. Adjust node sizes from dropdown
9. Double-click nodes for details

Example Data Flow

CSV Import

CSV Row:
  Staff ID: 1053
  Name: MAJID SALEM SAEED ALNAHDI
  Manager: 2 - MOHAMMAD SALEH AL HAMMADI
  Job Title: Human Resources Manager
  Department: Human Resource

↓ Import Command

Staff Record Created:
  employee_id: "1053"
  first_name: "MAJID SALEM SAEED"
  last_name: "ALNAHDI"
  job_title: "Human Resources Manager"
  report_to: Staff.objects.get(employee_id="2")

↓ Hierarchy Built

Tree Structure:
  Mohammad Saleh Al Hammadi (ID: 2)
  └── Majid Salem Saeed Alnahdi (ID: 1053)
      └── Varghese Ninan (ID: 1049)

D3 Visualization

API Request: GET /organizations/api/staff/hierarchy/

↓

Query Database:
  - Get all staff (filtered by permissions)
  - Build hierarchy tree
  - Calculate team sizes
  - Compute statistics

↓

Return JSON:
  {
    "hierarchy": [...],
    "statistics": {...}
  }

↓

D3.js Renders:
  - Interactive tree chart
  - With zoom, pan, search
  - Multiple layout options
  - Smooth animations

Common Use Cases

1. View Entire Organization

D3 View: /organizations/staff/hierarchy/d3/

• Click "Expand All"
• See complete organization structure
• Use zoom/pan to navigate
• Good for understanding full hierarchy

2. Find Specific Staff Member

D3 View: /organizations/staff/hierarchy/d3/

• Type name or ID in search box
• Press Enter
• Chart auto-navigates to staff
• Node highlighted in red
• Instant feedback

3. Print Hierarchy Report

HTML View: /organizations/staff/hierarchy/

• Expand desired sections
• Use browser print (Ctrl/Cmd + P)
• Static layout prints well
• Good for reports

4. Analyze Team Structure

D3 View: /organizations/staff/hierarchy/d3/

• Set node size to "Team Size"
• Larger nodes = more direct reports
• Identify managers with large teams
• Spot potential bottlenecks

5. Compare Departments

D3 View: /organizations/staff/hierarchy/d3/

• Use department filter (if available)
• Switch between departments
• Compare hierarchy structures
• Identify organizational patterns

Troubleshooting

Import Issues

Problem: "Manager ID not found"

• Solution: Ensure manager IDs exist in Staff ID column
• Check: Import managers before their reports

Problem: "Department not found"

• Solution: Create departments first, or let import auto-create
• Check: Department name spelling

Hierarchy Issues

Problem: Wrong relationships

• Solution: Verify report_to field in database
• Check: Manager IDs in CSV match Staff IDs

Problem: Staff missing from hierarchy

• Solution: Check staff status is "active"
• Check: Verify hospital/department permissions
• Check: RBAC settings

D3 View Issues

Problem: Chart not loading

• Solution: Check browser console for errors
• Check: API endpoint is accessible
• Check: D3.js CDN is reachable

Problem: Search not working

• Solution: Verify staff exist in hierarchy
• Check: JavaScript console for errors
• Try: Full name instead of partial

Best Practices

Data Quality

1. ✅ Use consistent Staff IDs
2. ✅ Complete manager chains
3. ✅ Standardized department names
4. ✅ Valid job titles

Hierarchy Structure

1. ✅ Clear reporting lines
2. ✅ Avoid circular references
3. ✅ Reasonable depth (5-10 levels)
4. ✅ Appropriate span of control (5-15 direct reports)

Visualization

1. ✅ Use D3 for exploration and analysis
2. ✅ Use HTML for printing and reports
3. ✅ Filter large hierarchies
4. ✅ Choose appropriate layout for your needs

API Reference

Hierarchy API

Endpoint: /organizations/api/staff/hierarchy/

Method: GET

Parameters:

• hospital (optional): Filter by hospital ID
• department (optional): Filter by department ID

Response:

{
  "hierarchy": [
    {
      "id": "uuid",
      "name": "Full Name",
      "employee_id": "EMP123",
      "job_title": "Job Title",
      "hospital": "Hospital Name",
      "department": "Department Name",
      "status": "active",
      "staff_type": "type",
      "team_size": 5,
      "children": [...]
    }
  ],
  "statistics": {
    "total_staff": 100,
    "top_managers": 3
  }
}

Future Enhancements

Potential Additions

1. Export Features

   • Export D3 chart as PNG/SVG
   • Export as PDF
   • Download hierarchy data as JSON

2. Advanced Filtering

   • Filter by staff type
   • Filter by status
   • Multi-criteria filters

3. Additional Visualizations

   • Sunburst chart
   • Treemap
   • Force-directed graph

4. Analytics Dashboard

   • Hierarchy depth analysis
   • Span of control metrics
   • Bottleneck identification
   • Organizational health metrics

5. Collaboration Features

   • Shareable hierarchy links
   • Embed in reports
   • Compare hierarchies over time

Support and Resources

Documentation

• Import Guide: docs/STAFF_HIERARCHY_IMPORT_GUIDE.md
• D3 Integration: docs/D3_HIERARCHY_INTEGRATION.md
• HTML View: docs/STAFF_HIERARCHY_VIEW.md

Code Files

• Import Command: apps/organizations/management/commands/import_staff_csv.py
• API: apps/organizations/views.py - StaffViewSet.hierarchy()
• Views: apps/organizations/ui_views.py
• Templates: templates/organizations/staff_hierarchy*.html

External Resources

• D3.js: https://d3js.org/
• D3 Tree: https://github.com/d3/d3-hierarchy/tree

Summary

The PX360 system now provides a complete solution for:

✅ Importing staff data with hierarchy from CSV
✅ Storing hierarchical relationships using report_to field
✅ Visualizing hierarchies in two powerful ways:

• HTML-based (static, print-friendly)
• D3.js-based (interactive, feature-rich)

Choose the right visualization for your needs:

• HTML View: For reports, printing, simple viewing
• D3 View: For exploration, analysis, interactive use

Access your hierarchy visualizations:

• HTML: /organizations/staff/hierarchy/
• D3: /organizations/staff/hierarchy/d3/

Import your data:

python manage.py import_staff_csv your_staff_data.csv

The system is ready to handle your staff hierarchy data with powerful visualization capabilities!