# Phase 11: Advanced Queue Management - Implementation Plan

**Project**: Hospital Management System v4  
**App**: Appointments  
**Date**: 2025-01-10  
**Status**: 🔄 IN PROGRESS (40% Complete - Phases 11.1-11.4 Done)  
**Dependencies**: Phase 10 (Smart Scheduling Engine) ✅ COMPLETED  
**Estimated Time**: 12-15 hours  
**Time Spent**: ~5 hours

---

## Executive Summary

Phase 11 enhances the queue management system with real-time calculations, dynamic positioning, WebSocket support for live updates, and advanced analytics. This builds upon the existing basic queue system to create an intelligent, automated queue management platform.

### Strategic Objectives

1. **Reduce Average Wait Time by 20%** through dynamic positioning
2. **Improve Patient Experience** with real-time queue updates
3. **Optimize Provider Utilization** through intelligent load balancing
4. **Enable Real-time Monitoring** via WebSocket connections
5. **Provide Actionable Analytics** for queue performance optimization

### Current State vs. Target State

| Feature | Current State | Target State |
|---------|---------------|--------------|
| Queue Positioning | Basic FIFO with manual priority | AI-powered dynamic positioning |
| Wait Time Estimation | Static calculation | Real-time, load-adjusted estimation |
| Updates | Manual refresh required | WebSocket real-time updates |
| Analytics | Basic metrics | Comprehensive performance analytics |
| Load Management | Manual overflow handling | Automated load balancing |

---

## Implementation Checklist

### Phase 11.1: Database Schema Enhancements ✅ COMPLETE

- [x] **Step 1: Create New Models (models.py)**
  - [x] Create `QueueConfiguration` model
    - [ ] Dynamic positioning rules (weights for priority, wait time, appointment time)
    - [ ] Capacity management (overflow queue settings)
    - [ ] Wait time estimation settings
    - [ ] Real-time update configuration
  - [ ] Create `QueueMetrics` model
    - [ ] Volume metrics (total entries, completed, no-shows)
    - [ ] Time metrics (average wait, max wait, service time)
    - [ ] Queue state metrics (peak size, average size)
    - [ ] Hourly granularity for trend analysis
  - [x] Add indexes for performance optimization

- [x] **Step 2: Run Migrations**
  - [x] Generate migration files (`0004_queueconfiguration_queuemetrics.py`)
  - [x] Review migration SQL
  - [x] Apply migrations to database
  - [x] Verify schema changes

### Phase 11.2: Queue Engine Implementation ✅ COMPLETE

- [x] **Step 1: Create Queue Engine Module**
  - [x] Create `appointments/queue/` directory
  - [x] Create `appointments/queue/__init__.py`
  - [x] Create `appointments/queue/queue_engine.py` (500+ lines)

- [x] **Step 2: Implement AdvancedQueueEngine Class**
  - [x] `add_to_queue()` - Intelligent patient addition
    - [ ] Calculate optimal position using weighted factors
    - [ ] Estimate wait time based on historical data
    - [ ] Auto-reposition existing entries if needed
    - [ ] Broadcast WebSocket update
    - [ ] Update queue metrics
  
  - [ ] `calculate_optimal_position()` - Multi-factor positioning
    - [ ] Priority score factor (configurable weight)
    - [ ] Wait time fairness factor (configurable weight)
    - [ ] Appointment time proximity factor (configurable weight)
    - [ ] Support both FIFO and dynamic modes
  
  - [ ] `calculate_estimated_wait_time()` - Smart estimation
    - [ ] Use historical service time data (last 7 days)
    - [ ] Apply load factor based on queue size
    - [ ] Account for time of day variations
    - [ ] Return timedelta object
  
  - [ ] `calculate_load_factor()` - Queue load analysis
    - [ ] Calculate utilization percentage
    - [ ] Apply multiplier based on load level
    - [ ] Return adjustment factor (1.0 - 2.0)
  
  - [ ] `reposition_queue_entries()` - Dynamic repositioning
    - [ ] Calculate priority scores for all waiting entries
    - [ ] Consider wait time accumulation
    - [ ] Consider appointment proximity
    - [ ] Update positions atomically
  
  - [ ] `get_next_patient()` - Serve next patient
    - [ ] Get highest priority waiting entry
    - [ ] Mark as called
    - [ ] Broadcast update
    - [ ] Return entry object
  
  - [ ] `broadcast_queue_update()` - WebSocket broadcasting
    - [ ] Get current queue status
    - [ ] Send to WebSocket group
    - [ ] Handle errors gracefully
  
  - [ ] `get_queue_status()` - Status snapshot
    - [ ] Current size and capacity
    - [ ] Average wait time
    - [ ] Top 10 waiting entries
    - [ ] Timestamp
  
  - [x] `update_queue_metrics()` - Analytics tracking
    - [x] Create/update hourly metrics
    - [x] Track peak queue size
    - [x] Calculate averages
    - [x] Store in QueueMetrics model

### Phase 11.3: WebSocket Support ✅ COMPLETE

- [x] **Step 1: Install Django Channels**
  - [x] Add `channels==4.0.0` to requirements.txt
  - [x] Add `channels-redis==4.1.0` to requirements.txt
  - [x] Add `daphne==4.0.0` to requirements.txt
  - [x] Add `channels` and `daphne` to INSTALLED_APPS

- [x] **Step 2: Configure Channels**
  - [x] Update `hospital_management/settings.py`
    - [x] Add ASGI_APPLICATION setting
    - [x] Configure CHANNEL_LAYERS with Redis (database 2)
  - [x] Update `hospital_management/asgi.py`
    - [x] Import ProtocolTypeRouter, URLRouter
    - [x] Configure WebSocket routing with AuthMiddleware
    - [x] Add AllowedHostsOriginValidator for security
  - [x] Create `hospital_management/routing.py`
    - [x] Define WebSocket URL patterns aggregator

- [x] **Step 3: Create Queue Consumers**
  - [x] Create `appointments/consumers.py` (300+ lines)
  - [x] Implement `QueueConsumer` class
    - [x] `connect()` - Join queue group, send initial status
    - [x] `disconnect()` - Leave queue group
    - [x] `receive()` - Handle client messages (get_status, ping)
    - [x] `queue_update()` - Handle broadcast events
    - [x] `position_change()` - Handle position changes
    - [x] `patient_called()` - Handle patient called events
    - [x] `get_queue_status()` - Async database query
  - [x] Implement `PatientQueueConsumer` class
    - [x] Patient-specific updates
    - [x] Personal position tracking
    - [x] Called notifications

- [x] **Step 4: Configure Routing**
  - [x] Create `appointments/routing.py`
  - [x] Define WebSocket URL patterns:
    - [x] `ws/appointments/queue/<queue_id>/`
    - [x] `ws/appointments/queue/<queue_id>/patient/<patient_id>/`
  - [x] Import in main routing.py

- [x] **Step 5: Set Up Redis**
  - [x] Configure Redis connection in settings (localhost:6379/2)
  - [x] Set capacity (1500) and expiry (10s) settings
  - [x] Ready for Redis connectivity testing

### Phase 11.4: API Endpoints ✅ COMPLETE

- [x] **Step 1: Create Queue API Views**
  - [x] Create `appointments/api/queue_views.py` (500+ lines)
  - [x] Implement `QueueManagementViewSet` (7 endpoints)
    - [x] `add_patient` - POST /api/appointments/queues/{id}/add_patient/
    - [x] `status` - GET /api/appointments/queues/{id}/status/
    - [x] `call_next` - POST /api/appointments/queues/{id}/call_next/
    - [x] `reposition` - POST /api/appointments/queues/{id}/reposition/
    - [x] `analytics` - GET /api/appointments/queues/{id}/analytics/?days=7
    - [x] `metrics` - GET /api/appointments/queues/{id}/metrics/
    - [x] `configuration` - GET/PUT /api/appointments/queues/{id}/configuration/
  - [x] Implement `QueueEntryViewSet` (5 endpoints)
    - [x] `mark_called` - POST /api/appointments/queue-entries/{id}/mark_called/
    - [x] `mark_in_progress` - POST /api/appointments/queue-entries/{id}/mark_in_progress/
    - [x] `mark_completed` - POST /api/appointments/queue-entries/{id}/mark_completed/
    - [x] `mark_no_show` - POST /api/appointments/queue-entries/{id}/mark_no_show/
    - [x] `remove` - DELETE /api/appointments/queue-entries/{id}/remove/

- [x] **Step 2: Add URL Patterns**
  - [x] Update `appointments/api/urls.py`
  - [x] Register QueueManagementViewSet with router
  - [x] Register QueueEntryViewSet with router
  - [x] WebSocket URLs already configured in routing.py

### Phase 11.5: Views & Templates 🔄 IN PROGRESS

- [ ] **Step 1: Create Queue Management Views**
  - [ ] Update `appointments/views.py`
    - [ ] `AdvancedQueueManagementView` - Main queue interface
    - [ ] `queue_status_htmx_view` - HTMX status updates
    - [ ] `add_to_queue_htmx_view` - HTMX patient addition
    - [ ] `call_next_patient_htmx_view` - HTMX next patient
    - [ ] `queue_analytics_view` - Analytics dashboard
    - [ ] `queue_metrics_dashboard_view` - Metrics dashboard
    - [ ] `queue_config_view` - Configuration interface

- [ ] **Step 2: Create Main Templates (4 templates)**
  - [ ] `templates/appointments/queue/advanced_queue_management.html`
    - [ ] Real-time queue display
    - [ ] WebSocket connection setup
    - [ ] Add patient form
    - [ ] Call next patient button
    - [ ] Queue configuration panel
  
  - [ ] `templates/appointments/queue/queue_analytics.html`
    - [ ] Performance metrics dashboard
    - [ ] Time-series charts
    - [ ] Provider comparison
    - [ ] Export functionality
  
  - [ ] `templates/appointments/queue/queue_monitor.html`
    - [ ] Public display screen
    - [ ] Large font queue positions
    - [ ] Auto-refresh display
    - [ ] Minimal UI for waiting room
  
  - [ ] `templates/appointments/queue/queue_config.html`
    - [ ] Configuration form
    - [ ] Weight adjustments (priority, wait time, appointment)
    - [ ] Capacity settings (max size, overflow)
    - [ ] Real-time update settings (WebSocket, intervals)
    - [ ] Load factor thresholds
    - [ ] Auto-reposition settings

- [ ] **Step 3: Create Partial Templates (8 partials)**
  - [ ] `templates/appointments/queue/partials/queue_list.html`
    - [ ] Current queue entries
    - [ ] Position, patient name, wait time
    - [ ] Status indicators
    - [ ] Action buttons
  
  - [ ] `templates/appointments/queue/partials/queue_stats.html`
    - [ ] Current size, capacity
    - [ ] Average wait time
    - [ ] Load indicator
    - [ ] Refresh timestamp
  
  - [ ] `templates/appointments/queue/partials/next_patient.html`
    - [ ] Next patient details
    - [ ] Call button
    - [ ] Skip button
    - [ ] Patient info
  
  - [ ] `templates/appointments/queue/partials/queue_metrics.html`
    - [ ] Hourly metrics chart
    - [ ] Peak times
    - [ ] No-show rates
    - [ ] Service time trends
  
  - [ ] `templates/appointments/queue/partials/add_patient_form.html`
    - [ ] Patient selection
    - [ ] Priority selection
    - [ ] Notes field
    - [ ] Submit button
  
  - [ ] `templates/appointments/queue/partials/position_indicator.html`
    - [ ] Large position number
    - [ ] Estimated wait time
    - [ ] Queue name
    - [ ] Status message
  
  - [ ] `templates/appointments/queue/partials/load_indicator.html`
    - [ ] Visual load gauge
    - [ ] Color-coded status (green/yellow/red)
    - [ ] Current utilization percentage
    - [ ] Load factor display
  
  - [ ] `templates/appointments/queue/partials/config_form.html`
    - [ ] Weight sliders
    - [ ] Threshold inputs
    - [ ] Toggle switches
    - [ ] Save/Reset buttons

**COMPLETE TEMPLATE LIST (4 Main + 8 Partials = 12 Total):**

**Main Templates:**
1. `advanced_queue_management.html` - Main queue interface with WebSocket
2. `queue_analytics.html` - Analytics dashboard with charts
3. `queue_monitor.html` - Public display for waiting room
4. `queue_config.html` - Configuration management

**Partial Templates:**
5. `partials/queue_list.html` - Queue entries table
6. `partials/queue_stats.html` - Real-time statistics
7. `partials/next_patient.html` - Next patient card
8. `partials/queue_metrics.html` - Metrics charts
9. `partials/add_patient_form.html` - Add patient form
10. `partials/position_indicator.html` - Position display
11. `partials/load_indicator.html` - Load gauge
12. `partials/config_form.html` - Configuration form

### Phase 11.6: Frontend JavaScript ✅

- [ ] **Step 1: Create WebSocket Client**
  - [ ] Create `static/js/queue_websocket.js`
    - [ ] WebSocket connection management
    - [ ] Auto-reconnect logic
    - [ ] Message handling
    - [ ] UI update functions
    - [ ] Error handling

- [ ] **Step 2: Create Queue UI Module**
  - [ ] Create `static/js/queue_ui.js`
    - [ ] Queue list rendering
    - [ ] Stats updates
    - [ ] Animation effects
    - [ ] Sound notifications (optional)
    - [ ] Toast messages

- [ ] **Step 3: Integrate with HTMX**
  - [ ] Add HTMX attributes to templates
  - [ ] Configure auto-refresh fallback
  - [ ] Add loading indicators
  - [ ] Handle HTMX events

### Phase 11.7: Admin Interface ✅

- [ ] **Step 1: Register Models in Admin**
  - [ ] Update `appointments/admin.py`
    - [ ] Register `QueueConfiguration` with custom admin
    - [ ] Register `QueueMetrics` with read-only admin
    - [ ] Add inline editing for queue config
    - [ ] Add filters and search

- [ ] **Step 2: Create Custom Admin Actions**
  - [ ] Add "Reposition Queue" action
  - [ ] Add "Clear Queue" action
  - [ ] Add "Export Metrics" action

### Phase 11.8: Signals & Automation ✅

- [ ] **Step 1: Create Queue Signals**
  - [ ] Update `appointments/signals.py`
    - [ ] `queue_entry_created` - Auto-position new entries
    - [ ] `queue_entry_called` - Update metrics
    - [ ] `queue_entry_completed` - Calculate service time
    - [ ] `queue_entry_no_show` - Update no-show metrics

- [ ] **Step 2: Connect Signals**
  - [ ] Connect to QueueEntry model signals
  - [ ] Add signal handlers
  - [ ] Test signal execution

### Phase 11.9: Testing ✅

- [ ] **Step 1: Unit Tests**
  - [ ] Create `appointments/tests/test_queue_engine.py`
    - [ ] Test optimal position calculation
    - [ ] Test wait time estimation
    - [ ] Test load factor calculation
    - [ ] Test dynamic repositioning
    - [ ] Test metrics updates

- [ ] **Step 2: Integration Tests**
  - [ ] Create `appointments/tests/test_queue_api.py`
    - [ ] Test API endpoints
    - [ ] Test WebSocket connections
    - [ ] Test concurrent updates
    - [ ] Test error handling

- [ ] **Step 3: Performance Tests**
  - [ ] Create `appointments/tests/test_queue_performance.py`
    - [ ] Test with 50+ concurrent connections
    - [ ] Test with 100+ queue entries
    - [ ] Measure response times
    - [ ] Test WebSocket broadcast latency

### Phase 11.10: Documentation ✅

- [ ] **Step 1: API Documentation**
  - [ ] Document queue API endpoints
  - [ ] Add request/response examples
  - [ ] Document WebSocket protocol
  - [ ] Add error codes reference

- [ ] **Step 2: User Guide**
  - [ ] Create queue management guide
  - [ ] Add screenshots
  - [ ] Document configuration options
  - [ ] Add troubleshooting section

- [ ] **Step 3: Technical Documentation**
  - [ ] Document queue engine algorithm
  - [ ] Explain dynamic positioning logic
  - [ ] Document WebSocket architecture
  - [ ] Add deployment notes

---

## Success Criteria

### Performance Metrics
- ✅ Dynamic positioning reduces average wait time by 20%
- ✅ WebSocket updates delivered in <1 second
- ✅ System handles 50+ concurrent queue updates
- ✅ Wait time estimates accurate within ±10 minutes
- ✅ Queue metrics updated in real-time
- ✅ Page load time <2 seconds for queue display
- ✅ WebSocket reconnection <3 seconds on disconnect

### Functional Requirements
- ✅ Real-time queue updates without page refresh
- ✅ Intelligent patient positioning based on multiple factors
- ✅ Accurate wait time estimation using historical data
- ✅ Automated load balancing and overflow handling
- ✅ Comprehensive analytics and reporting
- ✅ Multi-queue support with independent configurations
- ✅ Mobile-responsive queue display

### User Experience
- ✅ Intuitive queue management interface
- ✅ Clear visual indicators for queue status
- ✅ Smooth animations for position changes
- ✅ Audio/visual notifications for queue updates
- ✅ Easy configuration without technical knowledge

---

## Technical Architecture

### WebSocket Flow
```
Patient Added → Queue Engine → Calculate Position → Update DB
                                                    ↓
                                            Broadcast Update
                                                    ↓
                                    WebSocket Group (queue_123)
                                                    ↓
                            All Connected Clients Receive Update
                                                    ↓
                                        UI Auto-Updates
```

### Dynamic Positioning Algorithm
```python
position_score = (
    priority_score * priority_weight +
    wait_time_score * wait_time_weight +
    appointment_proximity_score * appointment_weight
)

# Default weights:
# priority_weight = 0.5 (50%)
# wait_time_weight = 0.3 (30%)
# appointment_weight = 0.2 (20%)
```

### Load Factor Calculation
```python
utilization = current_size / max_capacity

if utilization < 0.5:
    load_factor = 1.0  # Normal
elif utilization < 0.75:
    load_factor = 1.2  # Slightly slower
elif utilization < 0.9:
    load_factor = 1.5  # Significantly slower
else:
    load_factor = 2.0  # Overloaded
```

---

## File Structure

```
appointments/
├── queue/
│   ├── __init__.py
│   └── queue_engine.py          # AdvancedQueueEngine class
├── api/
│   └── queue_views.py            # Queue API endpoints
├── consumers.py                  # WebSocket consumer
├── routing.py                    # WebSocket routing
├── models.py                     # QueueConfiguration, QueueMetrics
├── views.py                      # Queue management views
├── urls.py                       # URL patterns
├── admin.py                      # Admin configuration
├── signals.py                    # Queue signals
└── templates/appointments/queue/
    ├── queue_management.html     # Main interface
    ├── queue_analytics.html      # Analytics dashboard
    ├── queue_monitor.html        # Public display
    ├── queue_config.html         # Configuration
    └── partials/
        ├── queue_list.html       # Queue entries list
        ├── queue_stats.html      # Statistics
        ├── next_patient.html     # Next patient card
        ├── queue_metrics.html    # Metrics charts
        ├── add_patient_form.html # Add patient form
        └── position_indicator.html # Position display

static/js/
├── queue_websocket.js            # WebSocket client
└── queue_ui.js                   # UI updates
```

---

## Dependencies

### Python Packages
```txt
channels==4.0.0
channels-redis==4.1.0
redis==5.0.1
```

### System Requirements
- Redis server (for WebSocket channel layer)
- PostgreSQL (for advanced queries and performance)

### Configuration
```python
# settings.py additions
INSTALLED_APPS = [
    # ...
    'channels',
]

ASGI_APPLICATION = 'hospital_management.asgi.application'

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            'hosts': [('127.0.0.1', 6379)],
        },
    },
}
```

---

## Migration Strategy

### Phase 1: Database (Day 1)
1. Create new models
2. Run migrations
3. Create default configurations for existing queues

### Phase 2: Backend (Days 2-3)
1. Implement queue engine
2. Create API endpoints
3. Set up WebSocket infrastructure

### Phase 3: Frontend (Days 4-5)
1. Create templates
2. Implement WebSocket client
3. Add HTMX integration

### Phase 4: Testing (Day 6)
1. Unit tests
2. Integration tests
3. Performance tests

### Phase 5: Deployment (Day 7)
1. Deploy Redis
2. Deploy application
3. Monitor and optimize

---

## Monitoring & Maintenance

### Key Metrics to Monitor
- WebSocket connection count
- Average message latency
- Queue processing time
- Redis memory usage
- Database query performance

### Alerts to Configure
- WebSocket disconnections >10%
- Queue wait time >60 minutes
- Redis connection failures
- Database slow queries

### Regular Maintenance
- Weekly: Review queue metrics
- Monthly: Optimize database indexes
- Quarterly: Review and adjust weights

---

## Next Phase Preview

**Phase 12: Intelligent Waitlist Automation** will build upon this queue system to:
- Automatically fill cancelled slots from waitlist
- Intelligent patient matching
- Multi-channel notifications
- Response tracking and analytics

---

## Notes

- All WebSocket connections should have heartbeat/ping-pong
- Implement graceful degradation if WebSocket fails (fall back to HTMX polling)
- Consider implementing queue snapshots for audit trail
- Add queue position history for analytics
- Consider implementing virtual queues for telemedicine

---

**Status**: Ready for implementation  
**Last Updated**: 2025-01-10  
**Next Review**: After Phase 11 completion